Cameron Neylon

As I mentioned a couple of weeks or so ago I've been playing around with
[Friendfeed](http://friendfeed.com). This is a 'lifestreaming' web
service which allows you to aggregate 'all' of the content you are
generating on the web into one place ([see here for
mine](http://friendfeed.com/cameronneylon)). This is interesting from my
perspective because it maps well onto our ideas about generating
multiple data streams from a research lab. This raw data then needs to
be pulled together and turned into some sort of narrative description of
what happened.

Friendfeed does quite a good job of the first bit. I can aggregate
content generated on this blog, photos [I have
uploaded](http://www.flickr.com/photos/24801682@N08/) to
[Flickr](http://www.flickr.com/ "Flickr"){rel="homepage"}, as well as
relevant streams from [my
LaBLog](http://chemtools.chem.soton.ac.uk/projects/blog/blogs.php/blog_id/10),
[papers](http://www.citeulike.org/user/CameronNeylon) I have looked at,
and other material that [I have
shared](http://www.google.com/reader/shared/07978161157711459001)
through [Google
Reader](http://en.wikipedia.org/wiki/Google_Reader "Google Reader"){rel="wikipedia"}.
Some of these are generated by feeds that I create using [Yahoo
Pipes](http://pipes.yahoo.com/ "Yahoo! Pipes"){rel="homepage"} to filter
my posts from a larger stream. As an aside, the one thing I haven't been
able to do in a sensible fashion is to aggregate all the comments I have
made on other people's blogs. Fundamentally this is because I can't do a
semantic search on 'Items that are comments, written by me' and then
convert that to an [RSS
feed](http://en.wikipedia.org/wiki/RSS "RSS"){rel="wikipedia"}. But
that's a separate issue. I can subscribe to other people's feeds and
they can subscribe to mine. Friendfeed is quite good on showing you
stuff from Friends of Friends as well which is a good discovery
mechanism. I have seen several [papers of
interest](http://friendfeed.com/e/25811e53-d556-cb6f-ee94-d2aa5733251c)
in [Neil Saunder's feed](http://friendfeed.com/neilfws) for instance.

Where Friendfeed isn't so good is on post processing. For instance, it
isn't straightforward to define the relationships between things. I have
been [taking notes on other people's
response](http://friendfeed.com/e/03332243-bc00-d5c9-49ca-5df95eaf6b62)
to my Blog posts, [sharing their posts through Google
Reader](http://friendfeed.com/e/0643287f-dfaa-7f9a-42b9-2c38b173b4d9)
and then annotating them in Friendfeed as a response or vice versa but
this is less than ideal. Also Friendfeed currently doesn't allow any
flexibility in what you want to see. I could probably manage without
seeing anyone else's
[Twitter](http://www.twitter.com "Twitter"){rel="homepage"} posts for
instance (not that they're not fun to read guys, just that they end up
taking up more of my time!). Maybe I don't want to see Dan Brickley's
shared videos but I do want to his shared items in Google Reader. I
could in principle do some of this in Yahoo Pipes -- I haven't tried as
yet but I should give it a go.

The next step beyond this would be much deeper integration and then
actually acting on items. Converting an incoming result from an
instrument into a meeting (or a booking on the next instrument). Acting
on the flow to notify a specific person of something important, and
again, starting to connect it all together into a web of interconnected
information that can then be subject to machine processing. This is much
more than just a lab book, or just a data feed, or even just laboratory
management environment. Its about controlling all the data streams in
your life and adding the metadata that will help you to make sense of
them.

In this context there is a lot of interest in
[Workstreamr](http://workstreamr.com/blog/) a new initiative that looks
to be part lifestream, part integration with existing management tools
like email and calendars, and ties this up with the kind of social tools
that enable communication, discovery, and sometimes just a conversation
about the weather. The idea seems to be, like FriendFeed that you are
involved with generating and viewing the feeds for yourself and others
but beyond this you can process and act on these feeds, generating
calendar items, send stuff on, embedding the whole thing into a planning
process. This will be very interesting to see in action. My concern is
that it will be too heavyweight for the kind of things I would like to
do and that it may end up focussed on business applications leading to a
poor fit with a research environment. However, I am happy to be proved
wrong and I have requested an invite to the closed beta.

Data portability remains fundamental to all of this and interoperation
between all these different tools and services is a must. Easy
conversion of an incoming data file into an annotated entry in a lab
book. The entry in a lab book kicking off a conversation, leading to a
calendar item for a meeting (with any associated paperwork). An awful
lot of this could be done today, by hand, using a mixture of RSS,
markup, and open and agreed data formats. The challenge lies in building
to tools that will allow us to manage and process this automatically.

p.s. Zemanta is doing much better now on suggested Links and Tags, much
less on volkswagen's coming through. Pictures not so good so far. Will
continue to update on progress.

[![](http://img.zemanta.com/pixie.png?x-id=55335ff1-5983-476b-9416-126224c56ff2){decoding="async"}](http://www.zemanta.com/ "Zemified by Zemanta")


As I mentioned a couple of weeks or so ago I’ve been playing around with Friendfeed. This is a ‘lifestreaming’ web service which allows you to aggregate ‘all’ of the content you are generating on the web into one place (see here for mine). This is interesting from my perspective because it maps well onto our ideas about generating multiple data streams from a research lab.

Friendfeed, lifestreaming, and workstreaming

admin

There were lots of helpful comments on my [previous
post](http://blog.openwetware.org/scienceintheopen/2007/12/17/the-problem-with-data/)
as well as some
[commiseration](http://feeds.feedburner.com/~r/PetermrsBlog/~3/201906826/)
from Peter Murray-Rust. Also Jean-Claude Bradley's group is starting to
face some similar issues with the combi-Ugi project ramping up. All in
the week that the Science Commons [Open Data protocol is
launched](http://sciencecommons.org/projects/publishing/open-access-data-protocol/).
I just wanted to bring out a few quick points:

The ease with which new data types can be incorporated into UsefulChem,
such as the [recent incorporation of a crystal
structure](http://usefulchem.wikispaces.com/Exp150) (see also JC's [Blog
Post](http://usefulchem.blogspot.com/2007/12/x-ray-crystallography-collaborator.html)),
shows the flexibility and ease provided by an open ended and free form
system in the context of the Wiki. The theory is that our slightly more
structured approach provides more implicit metadata, but I am conscious
that we have yet to demonstrate the extraction of the metadata back out
in a useful form.

Bill comments:

> ...I think perhaps the very first goal is just getting the data out
> there with metadata all over it saying â€œhere I am, come get meâ€?.

I agree that the first thing is to simply get the data up there but the
next question out of this comment must be how good is our metadata in
practise? So for instance, can anyone make any sense out of
[this](http://chemtools.chem.soton.ac.uk/projects/blog/blogs.php/bit_id/5416)
in isolation? Remember you will need to track back through links to the
post where this was 'made'. Nonetheless I think we need to see this
process through to its end. The comparison with UsefulChem is helpful
because we can decide whether the benefits of our system outweigh the
extra fiddling invovled, or conversely how much do we have to make the
fiddling less challenging to make it worthwhile. At the end of the day,
these are experiments in the best approaches to doing ONS.

Things that do make our life easier are an automatic catalogue of input
materials. This, and the ability to label things precisely for storage
is making a contribution to the way the lab is running. In principal
something similar can be achieved for data files. The main distinction
at the moment is that we generate a lot more data files than samples so
handling them is more logistically difficult.

Jean-Claude and Jeremiah have [commented further on Jeremiah's
Blog](http://blog-di-j.blogspot.com/2007/12/role-of-bioinformatics-in-open-notebook.html)
on some of the fault lines between computational and experimental
scientists. I just wanted to bring up a comment made by Jeremiah;

> It would be easier to understand however, if you used more common
> command-based plotting programs like gnuplot, R, and matlab.

This is quite a common perception. 'If you just used a command line
system you could simply export the text file'. The thing is that, and I
think I speak for a lot of wet biologists and indeed chemists, that we
simply can't be bothered. It is too much work to learn these packages
and fighting with command lines isn't generally something we are
interested in doing -- we'd rather be in the lab.

One of the very nice things about the data analysis package I use, [Igor
Pro](http://www.wavemetrics.com), is that it has a GUI built but it also
translates menu choices and mouse actions into a command line at the
bottom of the screen. What is more it has a quite powerful programming
language which uses exactly the same commands. You start using it by
playing with the mouse, you become more adept at repeating actions by
cutting and pasting stuff in the command line and then you can (almost)
write a procedure by pasting a bunch of lines into a procedure file. It
is, in my view, the outstanding example of a user interface that not
only provides functionality for the novice and expert user in a easily
accesible way, it also guides the novice into becoming a power user.

But for most applications we can't be bothered (or more charitably don't
have the time) to learn MatLab or Perl or R or GnuPlot (and certainly
not Tex!). Perhaps the fault line lies on the division between those who
*prefer* to use Word rather than Tex. One consequence of this is that we
use programs that have an irritating tendency to have proprietary file
systems. Usually we can export a text file or something a bit more open.
But sometimes this is not possible. It is almost always an extra step,
an extra file to upload, so even more work. Open document formats are
definitely a great step forward and XML file types are even better. But
we are a bit stuck in the middle of slowly changing process.

None of this is to say that I think we shouldn't put the effort in, but
more to say, that from the perspective of those of us who really don't
like to code, and particularly those of us generating data from 'beige
box' instruments the challenge of 'No insider information' is even
harder. As [Peter M-R
says](http://feeds.feedburner.com/~r/PetermrsBlog/~3/201906826/), the
glueware is both critical, and the hardest bit to get right. The problem
is, I can't write glueware, at least not without sticking my fingers to
each other.


There were lots of helpful comments on my previous post as well as some commiseration from Peter Murray-Rust. Also Jean-Claude Bradley’s group is starting to face some similar issues with the combi-Ugi project ramping up. All in the week that the Science Commons Open Data protocol is launched.

Messaggi di Rogue Scholar

Friendfeed, lifestreaming, and workstreaming

Following up on data storage issues