Cameron Neylon

We are in the slow process of gearing up within my group at RAL to
adopting the [Chemtools](http://chemtools.chem.soton.ac.uk/) LaBLog
system and in the process moving properly to an [Open
Notebook](http://en.wikipedia.org/wiki/Open_Notebook_Science) status.
This has taken much longer than I had hoped but there have been some
interesting lessons along the way. Here I want to think a bit about a
problem that has been troubling me for a while.

I haven't done a very good job of recording what I've been doing in the
times that I have been in a lab over the past couple of months. Anyone
who has been [following
along](http://biolab.isis.rl.ac.uk/projects/blog/blogs.php/blog_id/5)
will have seen small bursts of apparently unrelated activity where
nothing much ever seems to come to a conclusion. This has been divided
up mainly into a) a SANS experiment we did in early November which has
now moved into a data analysis phase, b) some preliminary, and thus far
fairly unconvincing experiments, attempting to use a very new laser
tweezers setup at the [Central Laser Facility](http://www.clf.rl.ac.uk/)
to measure protein-DNA interactions at the single molecule level and c)
other random odds and sods that have come by. None of these have been
very well recorded for a variety of reasons.

Data analysis, particularly when it uses a variety of specialist
software tools, is something I find very challenging to record. A common
approach is to take some relatively raw data, run it through some
software, and repeat, while fiddling with parameters to get a feel for
what is going on. Eventually the analysis is run "for real" and the
finalised (at least for the moment) structure/number/graph is generated.
The temptation is obviously just to formally record the last step but
while this might be ok as a minimum standard if only one person is
involved, when more people are working through data sets it makes sense
to try and keep track of exactly what has been done and which data has
been partially processes in which ways. This helps us both in terms of
being able to quickly track where we are with the process but also
reduces the risk of replicating effort.

The laser tweezers experiment involves a lot of optimising of buffer
conditions, bead loading levels, instrumental parameters and whatnot.
Essentially a lot of fiddling, rapid shifts from one thing to another
and not always being too sure exactly what is going on. We are still at
the stage of getting a feel for things rather than stepping through a
well ordered experiment. Again the recording tends to be haphazard as
you try on thingÂ  and then another. We're not even absolutely sure what
we should be recording for each "run" or indeed really what a "run" is
yet.

The common theme here is "fiddling" and the difficulty of recording it
efficiently, accurately, and usefully. What I would prefer to be doing
is somehow capturing the important aspects of what we're doing as we do
it. What is less clear is what the best way to do that is. In the case
of data analysis we have good model for how to do this well. Good use of
repositories and the use of versioned scripts for handling data
conversions, in the way the [Michael
Barton](http://www.michaelbarton.me.uk/) in particular has talked about
provide an example of good practice. Unfortunately it is good practice
that is almost totally alien to experimental biochemists and is also not
easily compatible with a lot of the software we use.

The ideal would be a work bench using a graphical representation of data
analysis tools and data repositories that would automatically generate
scripts and deposit these and versioned data files into an appropriate
repository. This would enable the "docking" of arbitrary web services,
software packages and whatever, as well as connection to shared data
stores. The purpose of the workbench would be to record what is done,
although it might also provide some automation tools. In many ways this
is what I think of when look at work flow engines like
[Taverna](http://en.wikipedia.org/wiki/Taverna_workbench) and platforms
for sharing workflows like [MyExperiment](http://myexperiment.org).

Its harder in the real world. Here the workbench is, well, the workbench
but the idea of recording everything along with contextual metadata is
pretty similar. The challenge lies in recording enough different aspects
of what is going on to capture the important stuff without generating a
huge quantity or data that can never be searched effectively. It is
possible to record multiple video streams, audio, screencast any control
computers , but it will be almost impossible to find anything in these
data streams.

A challenge that emerges over and over again in laboratory recording is
that you always seem to not be recording the thing that you really now
need to have. Yet if you record everything you still won't have it
because you won't be able to find it. Video, image, and audio search
will one day make a huge difference to this but in the meantime I think
we're just going to keep muddling on.


We are in the slow process of gearing up within my group at RAL to adopting the Chemtools LaBLog system and in the process moving properly to an Open Notebook status. This has taken much longer than I had hoped but there have been some interesting lessons along the way. Here I want to think a bit about a problem that has been troubling me for a while.

Recording the fiddly bits of experimental and data analysis work

*The second installment of the paper ([first part
here](http://cameronneylon.net/blog/a-personal-view-of-open-science-part-i/))
where I discuss building tools for Open (or indeed any) Science.*

**Tools for open science -- building around the needs of scientists**

It is the rapid expansion and development of tools that are loosely
categorised under the banner of 'Web2.0' or 'Read-write web' that makes
the sharing of research material available. Many of the generic tools,
particularly those that provide general document authoring capabilities,
have been adopted and used by a wide range of researchers. Online office
tools can enable collaborative development of papers and proposals
without the need for emailing documents to multiple recipients and the
resultant headaches associated with which version is which. Storing
spreadsheets, databases, or data online means that collaborators have
easy access to the most recent versions and can see how these are
changing. More generally the use of RSS feed readers and bookmarking
sites to share papers of interest and, to some extent, to distribute the
task of triaging the literature are catching in in some communities. The
use of microblogging platforms such as Twitter and aggregation and
conversational tools such as Friendfeed have recently been used very
effectively to provide coverage of conferences in progress, including
collaborative note-taking. In combination with streamed or recorded
video as well as screencasts and sharing of presentations online the
idea of a dstributed conference, while not an everyday reality, is
becoming feasible.

However it is often the case that,while useful, generic web based
services do not provide desired functionality or do not fit well into
the existing workflows of researchers. Here there is the opportunity,
and sometime necessity, to build specialised or adapated tools.
Collaborative preparation of papers is a good example of this.
Conventional web bookmarking services, such as del.icio.us provide a
great way of sharing the literature or resources that a paper builds on
with other authors but they do not automatically capture and recognise
the necessary metadata associated with published papers (journal, date,
author, volume, page numbers). Specialised services such as citeulke and
Connotea have been developed to enable one click bookmarking of
published literature and these have been used effectively by for example
using a specific tag for references associated with a specific paper in
progress. The problem with these services as they exist at the moment is
that they don't provide the crucial element in the workflow that
scientists want to aggregate the references for, the formatting of the
references in the finalised paper. Indeed the lack of formatting
functionality in GoogleDocs, the most widely used collaborative writing
tool, means that in practice the finalised document is usually cut and
pasted into Word and the references formatted using proprietary software
such as Endnote.The available tools do not provide the required
functionality.

A number of groups and organisations have investigated the use of Blogs
and Wikis as collaborative and shareable laboratory notebooks. However
few of these systems offer good functionality 'out of the box'. While
there are many electronic laboratory notebook systems sold by commercial
interests most are actually designed around securing data rather than
sharing it so are not of interesthere. While the group of Jean-Claude
Bradley has used the freely hosted WikiSpaces as a [laboratory
notebook](http://usefulchem.wikispaces.com) without further
modification, much of the data and analysis is hosted on other services,
including YouTube, FlickR, and GoogleDocs. The OpenWetWare group has
made extensive modifications to the MediaWiki system to [provide
laboratory notebook
functionality](http://openwetware.org/wiki/Lab_Notebook) whereas Garret
Lisi has adapted the TiddlyWiki framework as a way of [presenting his
notebook](http://deferentialgeometry.org). The [Chemtools
collaboration](http://chemtools.chem.soton.ac.uk/) at the University of
Southampton has developed a specialised Blog platform . Commercial
offerings in the area of web based lab notebooks are also starting to
appear. All of these different systems have developed because of the
specialised needs of recording the laboratory work of the scientists
they were designed for. The different systems make different assumptions
about where they fit in the workflow of the research scientist, and what
that workflow looks like. They are all, however, built around the idea
that they need to satisfy the needs of the user.

This creates a tension in tool building. General tools, that can be used
across a range of disciplines, are extremely challenging to design,
because workflows, and the perception of how they work, are different in
different disciplines. Specialist tools can be built for specific fields
but often struggle to translate into new areas. Because the market is
small in any field the natural desire for designers is to make tools as
general as possible. However in the process of trying to build for a
sufficiently general workflow it is often the case that applicability to
specific workflows is lost. There is a strong argument based on this for
building interoperable modules, rather than complete systems, that will
allow domain specialists to stich together specific solutions for
specific fields or even specific experiments. Interoperability of
systems and standards that enable it is a criteria that is sometimes
lost in the development process, but is absolutely essential to making
tools and processes shareable. The use of workflow management tools,
such as [Taverna](http://en.wikipedia.org/wiki/Taverna_workbench),
[Kepler](http://en.wikipedia.org/wiki/Kepler_scientific_workflow_system),
and [VisTrails](http://www.vistrails.org/index.php/Main_Page) have an
important role to play here.

While not yet at a stage where they are widely configurable by end users
the vision behind them has the potential both to make data analysis much
more straightforward for experimental scientist but also to solve many
of the problems involved in sharing process, as opposed to data. The
idea of visually wiring up online or local analysis tools to enable data
processing pipelines is compelling. The reason most experimental
scientists use spreadsheets for data analysis is that they do not wish
to learn programming languages. Providing visual programming tools along
with services with clearly defined inputs and outputs will make it
possible for a much wider range of scientists to use more sophisticated
and poweful analysis tools. What is more the ability to share, version,
and attribute, workflows will go some significant distance towards
solving the problem of sharing process. Services like
[MyExperiment](http://myexperiment.org) which provide an environment for
sharing and versioning Taverna workflows provide a natural way of
sharing the details of exactly how a specific analysis is carried out.
Along with an electronic notebook to record each specific use of a given
workflow or analysis procedure (which can be achieved automatically
though an API) the full details of the raw data, analysis procedure, and
any specific parameters used, can be recorded. This combination offers a
potential route out of the serious problem of sharing research processes
if the appropriate support infrastructure can be built up.

Also critical to successful sharing is a shared language or vocabulary.
The development of ontologies, controlled vocabularies, and design
standards are all important in sharing knowledge and crucial to
achieving the ulitmate goals of making this knowledge machine readable.
While there are divisions in the technical development and user
communities over the development and use of controlled vocabularies
there is little disagreement over the fact that good vocabularies
combined with good tools are useful. The disagreements tend to lie in
how they are best developed, when they should be applied, and whether
they are superior to or complementary to other approaches such as text
mining and social tagging. An integrated and mixed approach to the use
of controlled vocabularies and standards is the most likely to be
successful. In particular it is important to match the degree of
structure in the description to the natural degree of structure in the
object or objects being described. Highly structured and consistent data
types, such as crystal structures and DNA sequences, can benefit greatly
from highly structured descriptions which are relatively straightforward
to create, and in many cases are the standard outputs of an analysis
process. For large scale experimental efforts the scale of the data and
sample management problem makes an investment in detailed and structured
desriptions worth while. In a small laboratory doing unique work,
however, there may be a strong case for using local descriptions and
vocabularies that are less rigorous but easier to apply and able to grow
to fit the changing situation on the ground. Ideally designed in such a
way that mapping onto an external vocabulary is feasible if it is
required or useful in the future.

Making all of this work requires that researchers adopt these tools and
that a community develops that is big enough to provide the added value
that these tools might deliver. For a broad enough community to adopt
these approaches the tools must fit well in their existing workflow and
help to deliver the things that researchers are already motivated to
produce. For most researchers, published papers are the measure of their
career success and the basis of their reward structures. Therefore tools
that make it easier to write papers, or that help researchers to write
better papers, are likely to get traction. As the expectations of the
quality and completeness of supporting data increase for published
papers, tools that make it easier for the researcher to collate and
curate the record of their research will become important. It is the
process of linking the record of what happened in the laboratory, or
study, to the first pass intepretation and analysis of data, through
further rounds of analysis until a completed version is submitted for
review, that is currently poorly supported by available tools, and it is
this need that will drive the development of improved tools. These tools
will enable the disparate elements of the record of research, currently
scattered between paper notebooks, various data files on multiple hard
drives, and unconnected electronic documents, to be chained together.
Once this record is primarily electronic, and probably stored online in
a web based system, the choice to make the record public at any stage
from the moment the record is made to the point of publication, will be
available. The reason to link this to publication is to tie it into an
existing workflow in the first instance. Once the idea is embedded the
steps involved in making the record even more open are easily taken.

*[Part
III](http://cameronneylon.net/blog/a-personal-view-of-open-science-part-iii-social-issues/)
covers social issues around Open Science.*


<em>
 The second installment of the paper (first part here) where I discuss building tools for Open (or indeed any) Science.
</em>
<strong>
 Tools for open science – building around the needs of scientists
</strong>
It is the rapid expansion and development of tools that are loosely categorised under the banner of ‘Web2.0’ or ‘Read-write web’ that makes the sharing of research material available.

A personal view of Open Science – Part II – Tools

An update on the [Workshop that I announced
previously](http://blog.openwetware.org/scienceintheopen/2008/06/25/open-science-workshop-at-southampton-31-august-and-1-september/).
We have a number of people confirmed to come down and I need to start
firming up numbers. I will be emailing a few people over the weekend so
sorry if you get this via more than one route. The plan of attack
remains as follows:

Meet on evening of Sunday 31 August in Southampton, most likely at a
bar/restaurant near the University to coordinate/organise the details of
sessions.

Commence on Monday at \~9:30 and finish around 4:30pm (with the option
of discussion going into the evening) with three or four sessions over
the course of the day broadly divided into the areas of tools, social
issues, and policy. We have people interested and expert in all of these
areas coming so we should be able to to have a good discussion. The
object is to keep it very informal but to keep the discussion
productive. Numbers are likely to be around 15-20 people. For those not
lucky enough to be in the area we will aim to record and stream the
sessions, probably using a combination of [dimdim](http://dimdim.com),
[mogulus](http://mogulus.com), and [slideshare](http://slideshare.com).
Some of these may require you to be signed into our session so if you
are interested drop me a line at the account below.

To register for the meeting please send me an email to my gmail account
(cameronneylon). To avoid any potential confusion, even if you have
emailed me in the past week or so about this please email again so that
I have a comprehensive list in one place. I will get back to you with a
request via PayPal for Â£15 to cover coffees and lunch for the day (so
if you have a PayPal account you want to use please send the email from
that address). If there is a problem with the cost please state so in
your email and we will see what we can do. We can suggest options for
accomodation but will ask you to sort it out for yourself.

I have set up a [wiki](http://sotonopensciworkshop.wikispaces.com/) to
discuss the workshop which is currently completely open access. If I see
spam or hacking problems I will close it down to members only (so it
would be helpful if you could create an account) but hopefully it might
last a few weeks in the open form. Please add your name and any relevant
details you are happy to give out to the [Attendees
page](http://sotonopensciworkshop.wikispaces.com/Attendees) and add any
presentations or demos you would be interested in giving, or would be
interested in hearing about, on the [Programme
suggestion](http://sotonopensciworkshop.wikispaces.com/Programme+suggestions)
page.


An update on the Workshop that I announced previously. We have a number of people confirmed to come down and I need to start firming up numbers. I will be emailing a few people over the weekend so sorry if you get this via more than one route. The plan of attack remains as follows:  Meet on evening of Sunday 31 August in Southampton, most likely at a bar/restaurant near the University to coordinate/organise the details of sessions.

Southampton Open Science Workshop 31 August and 1 September

During the [workshop in late
February](http://blog.openwetware.org/scienceintheopen/2008/02/29/placeholder-post-for-workshop-comments/)
we had discussions about possible implementations of
[Taverna](http://taverna.sourceforge.net/) work flows to automate
specific processes to make our life easier. One specific example we
discussed was the reduction and initial analysis of Small Angle Neutrons
Scattering data. Here I want to describe a bit of the background to what
this is and what we might do to kick of the discussion.

Small angle scattering (SAS) is a technique that can be used to
determine large scale structural features (1 -- 100s nm) of molecules in
solution. This makes it appealing for the study of complexes, partially
and completely disordered systems, as well as confirming that the
overall solution structure is similar to that determined from high
resolution techniques. For biomolecular structure it is essentially
possible to get shapes at some level of detail. Most SAS is carried out
using X-rays but there are specific advantages to using neutrons for
some systems, particularly biological complexes (I've written a recent
[review on SAXS/SANS](http://www.ncbi.nlm.nih.gov/pubmed/18214466) in
biology if you want more details).

SAS does what is says on the can. You put a sample in a collimated beam
and observe the scattering of xrays or neutrons at small angles, usually
less than 5 deg. The general setup involve a sample position in front of
a long tank with a planar detector that can be moved up and down the
tank. For SANS there are a range of background and normalisation
measurements that need to be carried out to characterise absorption and
other issues. This means that converting the area detector data
(basically an image file, usually of around 1k x 1k pixels) to a simple
plot of scattered intensity (I) vs angle (usually given in units of
momentum transfer, Q\[angstroms\^-1\]) is not trivial. However it can
usually be reduced to a set of measurements that need to be done to
characterise the state of the instrument (which generally doesn't change
for a given set of measurments) and the measurements on the sample
itself. In addition to this there are (nearly) as many different tools
for converting area detector data to I vs Q as there are people looking
after these instruments.

What we would like to do is to be able to load up all of our raw data
(or at least pointers to it) into the LaBLog and then execute a
[Taverna](http://taverna.sourceforge.net/) workflow (which we could
share in [MyExperiment](http://www.myexperiment.org/)) to process, or
help us process, the data in a semi-automated fashion. This is assisted
by the fact that one of the data reduction tools,
[GRASP](http://www.ill.eu/sites/grasp/grasp_main.html) developed by
Charles Dewhurst at the [ILL](http://www.ill.fr), is written in MatLab.
Since the data we have to hand is ILL data from the
[D22](http://www.ill.eu/d22/home-d22/) instrument I will focus on this.

The idea would be, for a specific set of data, to define those data
files that relate to the instrument configuration and those that relate
to samples. [GRASP](http://www.ill.eu/sites/grasp/grasp_main.html)
allows a set of data files to be loaded in that define the
configuration. At this point there is manual intevention required to
define beam centres and to identify any dodgy pixels or detector
elements. There are two main types of files, transmission data involves
putting an attenuator in the beam and letting the direct beam hit the
detector, scattering data is taken with the attenuator out, but a beam
stop in to protect the detector from the direct beam.

The data files required for characterising a given instrument
configuration are; transmission and scattering from the empty beam (no
sample), transmission and scattering from an empty sample cell,
transmission and scattering from an H2O sample (sometimes cadmium gives
a 'flat background' for normalising between experiments), and
transmission and scattering from a boronated carbon sample (B4C,
sometimes vanadium, essentially blocks the beam and gives the
'electronic background', i.e. the detector background). Once this is set
up the processing of data files for that specific instrument
configuration is relatively straightforward

For each sample scattering and transmission data are recorded. These are
then used, with the instrument configuration setting to generate a
straightforward set of one dimensional data (I vs Q, error in I) which
is the data which is generally used for further analysis. This reduction
of the sample data can be fairly readily automated. We also have a
recently agreed [XML format for 1-D reduced small angle scattering
data](http://www.smallangles.net/wgwiki/index.php/cansas1d_documentation),
so we have a nice format to output.

The idea is therefore as follows:

1.  A [Taverna](http://taverna.sourceforge.net/) workflow is invoked
    that requires pointers to the instrument configuration files
    required. These could be found either by user instruction, or via
    the appropriate pointers in the LaBLog
2.  The workflow obtains the files and calls the MatLab code, loading up
    the files into the correct slots.
3.  It then calls the user to check over details that need manual
    intervention, such as beam centres, and any dodgy pixels. This is
    done via the GUI of the MatLab code in
    [GRASP](http://www.ill.eu/sites/grasp/grasp_main.html). The
    instrument configuration setup is then finished.
4.  The workflow then grabs the appropriate data files and automatically
    processes them, generating the appropriate reduced data.
5.  [Taverna](http://taverna.sourceforge.net/) then writes to the LaBLog
    a record of what it has done including pointers to input and output
    files, processing parameters etc.

This potentially automates a lot of the grunt work without removing the
checks that are required so as not to generated badly reduced data. In
fact if the workflow asks questions along the way it can asist the user
with doing the processing correctly. Finally the workflow can be readily
shared using [MyExperiment](http://www.myexperiment.org/), although
there is still likely to be the requirement to download
[GRASP](http://www.ill.eu/sites/grasp/grasp_main.html).


During the workshop in late February we had discussions about possible implementations of Taverna work flows to automate specific processes to make our life easier. One specific example we discussed was the reduction and initial analysis of Small Angle Neutrons Scattering data. Here I want to describe a bit of the background to what this is and what we might do to kick of the discussion.

Messaggi di Rogue Scholar

Recording the fiddly bits of experimental and data analysis work

A personal view of Open Science – Part II – Tools

Southampton Open Science Workshop 31 August and 1 September

Incorporating My Experiment and Taverna into the LaBLog – A possible example