Imperial College London

Henry Rzepa

::: {.kcite-section kcite-section-id="7290"}
I blogged about this [two years
ago](http://www.ch.imperial.ac.uk/rzepa/blog/?p=1967){target="_blank"} and
thought a brief update might be in order now. To support the discussions
here, I often perform calculations, and most of these are then deposited
into a [DSpace digital
repository](https://spectradspace.lib.imperial.ac.uk:8443/dspace/){target="_blank"},
along with metadata. Anyone wishing to have the full details of any
calculation can retrieve these from the repository. Now in 2012, such
repositories are more important than ever. 

![](http://www.ch.imperial.ac.uk/rzepa/blog/wp-content/uploads/2012/07/figshare.jpg "figshare"){.aligncenter
.wp-image-7291 decoding="async" width="400"}

In the UK, the main funding organisations are increasingly requiring
researchers to deposit their primary data in such open archives, and
some disciplines are better than others at this (chemistry does not rank
very highly in general however in terms of deposition of data). Our
DSpace server is a local one running at Imperial College, but a few
months back I became aware of Figshare, which aspires to operate on a
much wider and more general scale.  So I have injected one of the
calculations reported in another post (the IRC for the sodium tolyl
thiolate reaction with dichlorobutenone) into Figshare, making use of
the API which has recently been developed for this purpose and
implemented by  Matt Harvey. As with DSspace, it issues a DOI, which can
be then quoted wherever appropriate (and particularly in scientific
articles). This particular deposition is
[10.6084/m9.figshare.93096](http://dx.doi.org/10.6084/m9.figshare.93096){target="_blank"}

This repository is still undergoing a lot of development, but already
one can see many interesting features, such as export to Endnote or
Mendeley, and a QR barcode for devices with cameras. I would encourage
anyone who regularly generates e.g. computational chemistry data, or
knows a group that does, to encourage them to make use of such
facilities.

**Postscript:** If you have a look at [this deposition in
Figshare](http://figshare.com/articles/Gaussian_Job_Archive_for_C2H6/93114){target="_blank"} you
may already notice some of the developments I note above.  Matt Harvey
(who, with Mark Hahnel of Figshare, developed our publish script) has
added to the entry:

> \* A data descriptor document URL

> \* Wikipedia and pubchem links (automatically resolved from Inchi/Key
> searches)

> \* Links to chemspider searches

> \* Links to all other objects in the  [Spectra DSpace
> repository](https://spectradspace.lib.imperial.ac.uk:8443/dspace/handle/10042/13){target="_blank"}
> with a common Inchi/Key

::: {.wp_orcid_field}
[https://orcid.org/0000-0002-8635-8390](https://orcid.org/0000-0002-8635-8390){target="_blank"
rel="author"}
:::
:::


I blogged about this two years ago and thought a brief update might be in order now. To support the discussions here, I often perform calculations, and most of these are then deposited into a DSpace digital repository, along with metadata. Anyone wishing to have the full details of any calculation can retrieve these from the repository. Now in 2012, such repositories are more important than ever.

Digital repositories. An update.

Cameron Neylon

ChemSpidey lives! Even in the face of [Karen James' heavy
irony](http://twitter.com/kejames/status/3474150785) I am still amazed
that someone like me with very little programming experience was able to
pull together something that actually worked effectively in a live demo.
As long as you're not actively scared of trying to put things together
it is becoming relatively straightforward to build tools that do useful
things. Building ChemSpidey relied heavily on existing services and
other people's code but pulling that together was a relatively
straightforward process. The biggest problems were fixing the strange
and in most cases undocumented behaviour of some of the pieces I used.
So what is ChemSpidey?

ChemSpidey is a [Wave
robot](http://code.google.com/apis/wave/extensions/robots/index.html)
that can be found at chemspidey@appspot.com. The [code
repository](http://github.com/cameronneylon/ChemSpidey) is available at
[Github](http://github.com) and you should feel free to re-use it in
anyway you see fit, although I wouldn't really recommend it at the
moment, it isn't exactly the highest quality. One of the first
applications I see for Wave is to make it easy to author (semi-)semantic
documents which link objects within the document to records on the web.
In chemistry it would be helpful to link the names of compounds through
to records about those compounds on the relevant databases.

If ChemSpidey is added to a wave it watches for text of the form
"chem\[ChemicalName{;weight {m}g}\]" where the curly bracketed parts are
optional. When a blip is submitted by hitting the "done" button
ChemSpidey searches through the blip looking for this text and if it
finds it, strips out the name and sends it) to the [ChemSpider
SimpleSearch
service](http://www.chemspider.com/Search.asmx?op=SimpleSearch).
ChemSpider returns a list of database ids and the robot currently just
pulls the top one off the list and adds the text ChemicalName
(csid:####) to the wave, where the id is linked back to ChemSpider. If
there is a weight present it asks the [ChemSpider MassSpec
API](http://www.chemspider.com/MassSpecAPI.asmx?op=GetExtendedCompoundInfo)
for the nominal molecular weight calculates the number of moles and
inserts that. You can see [video of it working
here](http://www.viddler.com/explore/CameronNeylon/videos/8/) (look
along the timeline for the ChemSpidey tag).

![ChemSpidey](http://farm4.static.flickr.com/3453/3860977361_88220c2555_o.jpg){decoding="async"
loading="lazy" width="570" height="180"}

What have I learned? Well some stuff that is probably obvious to anyone
who is a proper developer. Use the current version of the API. Google
AppEngine pushes strings around as unicode which broke my tests because
I had developed things using standard Python strings. But I think it
might be useful to start drawing some more general lessons about how
best to design robots for research, so to kick of the discussion here
are my thoughts, many of which came out of discussions with Ian Mulvany
as we prepared for last weeks demo.

1.  **Always add a Welcome Blip when the Robot is added to a wave**.
    This makes the user confident that something has happened. Lets you
    notify users if a new version has been released, which might change
    the way the robot works, and lets you provide some short
    instructions.It's good to include a version number here as well.
2.  **Have some help available.** Ian's [Janey
    robot](http://github.com/IanMulvany/janey-robot) responds to the
    request (janey help) in a blip with an extended help blip explaining
    context. Blips are easily deleted later if the user wants to get rid
    of them and putting these in separate blips keeps them out of the
    main document.
3.  **Where you modify text leave an annotation.** I've only just
    started to play with annotations but it seems immensely useful to at
    least attempt to leave a trace here of what you've done that makes
    it easy for either your own Robot or others, or just human users to
    see who did what. I would suggest leaving annotations that identfy
    the robot, include any text that was parsed, and ideally provide
    some domain information. We need to discuss how to setup some name
    spaces for this.
4.  **Try to isolate the "science handling" from the "wave handling".**
    ChemSpidey mixes up a lot of things into one Python script. Looking
    back at it now it makes much more sense to isolate the interaction
    with the wave from the routines that parse text, or do mole
    calculations. This means both that the different levels of code
    should become easier for others to re-use and also if Wave doesn't
    turn out to be the one system to rule them all that we can re-use
    the code. I am no architecture expert and it would be good to get
    some clues from some good ones about how best to separate things
    out.

These are just some initial thoughts from a very novice Python
programmer. My code satisfies essentially none of these suggestions but
I will make a concerted attempt to improve on that. What I really want
to do is kick the conversation on from where we are at the moment, which
is basically playing around, into how we design an architecture that
allows rapid development of useful and powerful functionality.


ChemSpidey lives! Even in the face of Karen James’ heavy irony I am still amazed that someone like me with very little programming experience was able to pull together something that actually worked effectively in a live demo. As long as you’re not actively scared of trying to put things together it is becoming relatively straightforward to build tools that do useful things.

Writing a Wave Robot &#8211; Some thoughts on good practice for Research Robots

It's been an interesting week or so in the Chemistry online world.
Following on from my [musings about data
services](http://blog.openwetware.org/scienceintheopen/2009/01/27/the-integrated-lab-record-or-the-web-native-lab-notebook/)
and the preparation I was doing for a talk the week before last I asked
[Tony Williams](http://www.chemspider.com/blog/) whether it was possible
to embed spectra from [ChemSpider](http://chemspider.com) on a generic
web page in the same way that you would embed a
[YouTube](http://youtube.com) video, [Flickr](http://flickr.com)
picture, or [Slideshare](http://slideshare.net) presentation. The idea
is that if there are services out on the cloud that make it *easier* to
put some rich material in your own online presence by hosting it
somewhere that understands about your data type, then we have a chance
of pulling all of these disparate online presences together.

Tony went on to release two features, one that enables you to [embed a
molecule](http://www.chemspider.com/blog/why-are-chemical-structures-like-youtube-videos.html),
which [Jean-Claude has
demonstrated](http://onschallenge.wikispaces.com/Exp056) over on the
[ONS Challenge Wiki](http://onschallenge.wikispaces.com/). Essentially
by cutting and pasting a little bit of text from ChemSpider into
Wikispaces you get a nicely drawn image of the molecule, and the
machinery is in place to enable good machine readability of the
displayed page (by embedding chemical identifiers within the code) as
well as enabling the aggregation of web based information about the
molecule back at Chemspider.

The second feature was the one I had asked about, [the embedding of
spectra](http://www.chemspider.com/blog/why-are-spectra-like-youtube-videos.html).
Again this is really useful because it means that as an experimentalist
you can host spectra on a service that gets what they are, but you can
*also* incorporate them in a nice way back into your lab book, online
report, or whatever it is you are doing. This has already enabled Andy
Lang and Jean-Claude to build a very cool game, [initially in Second
Life](http://usefulchem.blogspot.com/2009/02/nmr-game-on-second-life.html)
but now also [on the web](http://lxsrv7.oru.edu/~alang/about.php). Using
the spectral and chemical information from Chemspider the player is
presented with the spectrum and three molecules; if they select the
correct molecule they get some points, if they get it wrong they lose
some. As Tony has [pointed
out](http://www.chemspider.com/blog/a-plea-to-the-community-to-help-teach-students-spectroscopy.html),
this is also a way of crowdsourcing the curation process -- if the
majority of people disagree with the "correct" assignment then maybe the
spectrum needs a second look. Chemistry Captchas anyone?

The other even this week has been the [efforts by Mitch over at the
Chemistry
Blog](http://www.chemistry-blog.com/2009/02/10/want-to-author-a-book-with-me/)
to set up an online resource for named reactions by crowdsourcing
contributions and ultimately turning it into a book. Mitch deserves
plaudits for this because he's gone on and *done* something rather than
just talked about it and we need more people like that. Some of us have
[criticised the
details](http://friendfeed.com/e/bf6a718c-08ff-4ad6-ba53-fa9f081a8078/Chemistry-Blog-Blog-Archive-Want-to-author-a-book/)
(also see [comments at the original
post](http://www.chemistry-blog.com/2009/02/10/want-to-author-a-book-with-me/))
of how he is going about it but from my perspective this is definitely
criticism motivated by the hope that he will succeed and that by making
some changes early on, there is the chance to get much more out of the
contributions that he gets.

In particular [Egon](http://chem-bla-ics.blogspot.com/) asked whether it
would be better to use
[Wikipedia](http://en.wikipedia.org/wiki/Main_Page) as the platform for
aggregating the named reaction; a point which I agree with. The problem
that people see with Wikipedia is largely that of image. People are
concerned about inaccurate editing, about the sometimes combative
approach of senior editors that are not necessarily expert in the are.
Part of the answer is to just get in there and do it -- particularly in
chemistry there are a range of people working hard to try and get stuff
cleaned up. Lots of work has gone into the Chemical boxes and named
reactions would be an obvious thing to move on to. Nonetheless it may
not work for some people and to a certain extent as long as the material
that is generated can be aggregated back to Wikipedia I'm not really
fussed.

The bigger concern for us "chemoinformatics jocks" (I can't help but
feel that categorising me as a foo-informatics anything is a little off
beam but never mind (-;) was the [example
pages](http://name.howtochem.com/) Mitch put up where there was very
little linking back of data to other resources. So there was no way, for
instance, to know that this page was even about a specific class of
chemicals. The schemes were shown as plain images, making it very hard
for any information aggregation service to do anything useful.
Essentially the pages didn't make full use of the power of the web to
*connect* information.

Mitch in turn has taken the criticism offered in a positive fashion and
has [thrown down the
gauntlet](http://www.chemistry-blog.com/2009/02/12/open-thread-chemistry-collaborations-round-2/);
effectively asking the question, "well if you want this marked up, where
are the tools to make it easy, and the instructions in plain English to
show how to do it?". He also asks, if named reactions aren't the best
place to start, then what would be a good collaborative project. Fair
questions, and I would hope the Chemspider services start to point in
the right direction. Instead of drawing an image of a molecule and
pasting it on a web page, use the service and connect to the molecule
itself, this connects the data up, and gives you a nice picture for
free. It's not perfect. The ideal situation would be a little chemical
drawing palette. You draw the molecule you want, it goes to the data
service of choice, finds the correct molecule (meaning the user doesn't
need to know what SMILES, InChis or whatever are), and then brings back
whatever you want; image, data, vendors, price. This would be a really
powerful demonstration of the power of linked data and it probably could
be pulled together from existing services.

But what about the second question? What would be a good project? Well
this is where that second Chemspider functionality comes in. What about
flooding Chemspider with high quality, electronic copies, of NMR
spectra. Not the molecules that your supervisor will kill you for
releasing, but all those molecules you've published, all those hiding in
undergraduate chemistry practicals. Grab the electronic files, lets find
a way of converting them all to JCamp online, and get 'em up on
Chemspider as Open Data.


It’s been an interesting week or so in the Chemistry online world. Following on from my musings about data services and the preparation I was doing for a talk the week before last I asked Tony Williams whether it was possible to embed spectra from ChemSpider on a generic web page in the same way that you would embed a YouTube video, Flickr picture, or Slideshare presentation.

The growth of linked up data in chemistry &#8211; and good community projects

[![Sign at The Googleplex. Google limited access to Xenu.net, in March
2002.](http://upload.wikimedia.org/wikipedia/en/thumb/e/ee/Googleplex_Welcome_Sign.jpg/202px-Googleplex_Welcome_Sign.jpg){decoding="async"}](http://en.wikipedia.org/wiki/Image:Googleplex_Welcome_Sign.jpg)I
realised the other day that I havenâ€™t written an exciteable blog post
about getting an invitation to
[SciFoo](http://en.wikipedia.org/wiki/Science_Foo_Camp "Science Foo Camp"){rel="wikipedia"}!
The reason for this is that [I got overexcited over on
FriendFeed](http://friendfeed.com/e/0b4e74bd-9513-4c20-7330-3d51868290fb)
instead and havenâ€™t really had time to get my head together to write
something here. But in this post I want to propose a session and think
through what the focus and aspects of that might be.

I am a passionate advocate of two things that I think are intimately
related. I believe strongly in the need and benefits that will arise
from building, using, and enabling the effective search and processing
of a scientific data commons. I
\[[1](http://blog.openwetware.org/scienceintheopen/2008/04/16/the-science-exchange/),[2](http://blog.openwetware.org/scienceintheopen/2008/05/11/more-on-the-science-exchance-or-building-and-capitalising-a-data-commons/)\]
and others (including
[John](http://network.nature.com/blogs/user/wilbanks/2008/05/10/on-the-erosion-of-the-public-domain)
[Wilbanks](http://network.nature.com/blogs/user/wilbanks/2008/05/11/on-a-data-commons),
[Deepak Singh](http://feeds.feedburner.com/~r/mndoci/~3/288151920/), and
[Plausible
Accuracy](http://www.plausibleaccuracy.com/2008/05/12/data-should-be-public-domain-and-more-esoteric-blog-based-rasslin/))
have written on this quite a lot recently. The second aspect is that I
believe strongly in the need for effective useable and generic tools to
record science as it happens and to process that record so that others
can use it effectively. To me these two things are intimately related.
By providing the tools that enable the record to be created and
integrating them with the systems that will store and process the data
commons we can enable scientists to record their work better,
communicate it better, and make it available *as a matter of course* to
other scientists (not necessarily immediately I should add, but when
they are comfortable with it).

At last yearâ€™s SciFoo Chris DiBona ran a session called â€˜Give us the
Dataâ€™ \[Ed. As noted below Jon Trowbridge led the session with Chris
DiBona acting as 'session chair'\] Now I wasnâ€™t there but Deepak Singh
[wrote the session up in a blog
post](http://mndoci.com/blog/2007/08/09/scifoo-google-and-large-scientific-datasets/)
which is my main inspiration. The idea here seemed to be to send in big
data sets and that
[Google](http://en.wikipedia.org/wiki/Google "Google"){rel="wikipedia"}
could slurp these up for processing. Now the world has moved on from
last year and I believe we can propose something much more ambitious
*with the tools we have available*. We can build the systems, or at
least the prototypes of systems, that will be so useful for recording
and sharing data that people will want to use them. Google AppEngine,
Amazon Web Services, Freebase,
[CouchDB](http://www.couchdb.com/ "CouchDB"){rel="homepage"}, Twine, and
the growing efforts in actually making controlled vocabularies useful
and useable, as well as a growing web of repositories and services make
this look like an achievable target in a way that it wasnâ€™t twelve
months ago. Once we start building these repositories of data then there
will be a huge incentive both for scientists and big players such as
Google, Microsoft, Yahoo, IBM, and Amazon to look at how to effectively
process, re-process, and present this data. That will be exciting, but
first we need to build the tools and systems that will enable us to put
the data there.

What does this â€˜systemâ€™ look like. Iâ€™m going to suggest a possible
model, based on my non-technical understanding of how these things work.
I will probably get the details wrong (and please comment where I do).
Letâ€™s start at the storage end. We are only interested in open
repositories, where it is explicitly stated that data is in the [public
domain](http://en.wikipedia.org/wiki/Public_domain "Public domain"){rel="wikipedia"}
and therefore freely re-useable. Some of these repositories exist for
specific domains.
[ChemSpider](http://www.chemspider.com/ "ChemSpider"){rel="homepage"} ,
NMRShiftDB and crystallography databases in the chemistry domain,
GenBank,
[PDB](http://en.wikipedia.org/wiki/Protein_Data_Bank "Protein Data Bank"){rel="wikipedia"},
and many others in the biosciences domain. To be honest, for most large
datasets repositories already exist. The problem is not â€˜big
scienceâ€™, the problem is â€˜small scienceâ€™ or â€˜long tailâ€™
science. The vast quantity of data and description that is generated by
small numbers o people in isolated labs. Huge amounts of data
languishing on millions of laptops in non-standard formats. All that
stuff that no-one can really be bothered dealing with.

The storage itself is not really a problem. It can go on [Amazon
S3](http://en.wikipedia.org/wiki/Amazon_S3 "Amazon S3"){rel="wikipedia"},
Google servers, or into the various [Institutional
Repositories](http://en.wikipedia.org/wiki/Institutional_repository "Institutional repository"){rel="wikipedia"}
that Universities are rapidly deploying internationally. In fact, it
should go into all three and a few other places besides. Storage is
cheap, redoing the experiment is not. The digital curation people are
also working on the format problem with standards like
[ORE](http://www.ukoln.ac.uk/repositories/digirep/index/OAI-ORE)
providing a means of wrapping collections of data up and describing the
format internally. Combine this with an open repository of data formats
and the problem of storing and understanding data goes away, or rather
it is still a large technical problem but one that is reasonably soluble
with sufficient effort. One thing that is needed is standards for
describing data formats; both standards in terms of the description and
quality standards for what are the minimal requirements for describing
data (MiSDaFD anyone?). A lot of work is required in the area of
controlled vocabularies and ontologies, particularly in science domains
where they havenâ€™t yet got much traction. These donâ€™t need to be
perfect, they just need to be descriptive. But once again, how to go
about doing this is reasonably clear (devil is in the details I know).

The fundamental problem is how to capture the data, associated metadata,
and do the wrapping in a way that is an easy and convenient enough that
scientists will actually do it. It will come as no suprise to the
regularn readers of this blog that I would advocate the development of
tools to enable scientists to do this effectively at the stage at which
they already record their experiments; when they are done. Not only is
this the point where the scientist already has the task of planning,
executing, and recording their experiment. It is also the point at which
embedding good metadata and recording practice will be the most benefit
to them.

So we need to build a lab book system, or a virtual research environment
if you prefer, because this will encompass much more than the
traditional paper notebook. It will also encompass much more than the
current crop of electronic lab notebooks. I donâ€™t want to go into the
details of what I think this should look like here; thatâ€™s the subject
for a post in its own right. But to give an outline this system will
consist of a feed aggregator (not unlike
[Friendfeed](http://friendfeed.com)) where the user can subscribe to
various data streams, some from people, some from instruments or
sensors, and some that they have generated themselves, pictures at
[Flickr](http://flickr.com), comments on [Twitter](http://twitter.com).
The lab book component itself will look more or less like a word
processor, or a blog, or a wiki, depending on what the user wants. In
this system the user will generate documents (or pages); some will
describe samples, some procedures, and some ideas. Critically this
authoring system will have an open plugin architecture supported by an
open source community that enables the user to grab extra functionality
automatically.

A simple plugin might recognise a DNA sequence, or an INCHI code, and
provide a functional graphical display, complete with automatically
generated links back to the appropriate databases. Another plugin might
provide access to a range of ontologies or controlled vocabularies which
enable autocompletion of typed text with the appropriate name or easy or
automatically generate the PDB/SBML/MiBBI compliant datafiles required
for deposition. Journal submission templates could also be plugins or
gadgets that assist in pulling all the relevant information together for
submission and wrapping it up in an appropriate format. Semantic
authoring at the journal submission stage is never going to work.
Building it in early on and sucking it through when required just might.
Adding semantic information could be as easy (but probably should be no
easier than) formatting text. Select, right click, add semantics from
drop down menu.

This environment will have a proper versioning system, web integration
to push documents into the wider world, and a WYSIWYG interface. It will
expose human readable and machine readable versions of the files, and
will almost certainly use an underlying XML document format. It will
probably expose both microformats, rdf, and free text tags.

I think there are essentially two routes towards this kind of system.
One is to build into existing word processor and document handling
ecosystems. Essentially building a plugin architecture for Word or
OpenOffice. This would have the advantage of a system that is already
widely used and familiar to scientists. It has the disadvantage (at
least in the Microsoft case) that it would be built on top of a closed
source product and be reliant on the continued existence and support of
a large company. The alternative approach is to bring a completely open
source product through. This will require much more work, particularly
on the user interface side, but may be easier to handle in terms of open
source plugin architecture. It could be entirely web based. Google
AppEngine provides a very interesting environment for building such a
thing. CouchDB may be a good way of maintaining the underlying data.
Google Gadgets may well provide a lot of the desired plugin architecture
and widgets of various, types, javascript, bookmarklets, and all the
various tools on the web may provide a lot of the rest.

The purpose of this system is ultimately to encourage people to place
research results in the public domain in a way that makes them useful.
For it to get traction it has to be a no-brainer for people to adopt as
a central part of the way they run their laboratory. If it doesnâ€™t
make peopleâ€™s life easier; a lot easier; its not going to get any
traction. The User interface will be crucial, reliability will be
crucial, and the availability of added functionality which is currently
simply unavailable will be crucial.

Agreeing an architecture is likely to be a challenge but an open source
project is probably a more effective way of leveraging effort from a
wide community. There are many open standards available that can be used
here and the key is getting them all working together. Ideally the
choice of authoring environment wonâ€™t matter because it can still push
documents to the web in the same format(s), still use the same plugins,
still interface with the databases. Building this will cost money, quite
a lot of money, but the flip side is enabling the availability of the
data, the procedures into the public domain. As I heard Christoph
Steinbeck say yesterday, we can either get on and do this, or wait for
the singularity. I think weâ€™ve got a while to wait for that, so I
think its time to get on with the build. Who wants to be involved?

Related articles

- [The Open Data licensing
  issue](http://mndoci.com/blog/2008/05/11/the-open-data-licensing-issue/ "Open in new window")
  \[via Zemanta\]
- [Google App Engine Launches: Goes Directly After
  Amazon](http://www.centernetworks.com/google-app-engine-launches "Open in new window")
  \[via Zemanta\]
- [Reaching for the Sky Through The Compute
  Clouds](http://www.readwriteweb.com/archives/reaching_for_the_sky_through_compute_clouds.php "Open in new window")
  \[via Zemanta\]

[![](http://img.zemanta.com/pixie.png?x-id=7668403a-d9c4-4f53-b255-2fa59d7e22ec){decoding="async"}](http://www.zemanta.com/ "Zemified by Zemanta")[![](http://img.zemanta.com/pixie.png?x-id=){decoding="async"}](http://www.zemanta.com/ "Zemified by Zemanta")[![](http://img.zemanta.com/pixie.png?x-id=){decoding="async"}](http://www.zemanta.com/ "Zemified by Zemanta")

[![](http://img.zemanta.com/pixie.png?x-id=){decoding="async"}](http://www.zemanta.com/ "Zemified by Zemanta")


I realised the other day that I havenâ€™t written an exciteable blog post about getting an invitation to SciFoo! The reason for this is that I got overexcited over on FriendFeed instead and havenâ€™t really had time to get my head together to write something here.

How do we build the science data commons? A proposal for a SciFoo session

Image from
[Wikipedia](http://commons.wikipedia.org/wiki/Image:Banknotes.jpg) via
[Zemanta](http://www.zemanta.com "Zemanta ltd."){rel="homepage"}[![Banknotes
from all around the World donated by visitors to the British Museum,
London](http://upload.wikimedia.org/wikipedia/commons/thumb/0/09/Banknotes.jpg/202px-Banknotes.jpg){decoding="async"}](http://commons.wikipedia.org/wiki/Image:Banknotes.jpg)[](http://commons.wikipedia.org/wiki/Image:Banknotes.jpg)

Following on from the discussion a few weeks back [kicked off by
Shirley](http://onebiglab.blogspot.com/2008/04/envisioning-scientific-community-as-one.html)
at [One Big Lab](http://onebiglab.blogspot.com) and [continued
here](http://blog.openwetware.org/scienceintheopen/2008/04/16/the-science-exchange/)
I've been thinking about how to actually turn what was a throwaway
comment into reality:

> What is being generated here is new science, and science isnâ€™t paid
> for *per se*. The resources that generate science are supported by
> governments, charities, and industry but the actual production of
> science is not supported. The *truly* radical approach to this would
> be to turn the system on its head. Donâ€™t fund the universities to
> *do* science, fund the journals to *buy* science; then the system
> would reward increased efficiency.

There is a problem at the core of this. For someone to pay for access to
the results, there has to be a monetary benefit to them. This may be
through increased efficiency of their research funding but that's a
rather vague benefit. For a serious charitable or commercial funder
there has to be the potential to either make money, or at least see that
the enterprise could become self sufficient. But surely this means
monetizing the data somehow? Which would require restrictive licences,
which is not at the end what we're about.

The other story of the week has been the, in the end very useful,
kerfuffle caused by
[ChemSpider](http://www.chemspider.com/ "ChemSpider"){rel="homepage"}
moving [to a CC-BY-SA
licence](http://www.chemspider.com/blog/chemspider-adopts-creative-commons-licenses.html),
and the confusion that has been revealed regarding data, licencing, and
the [public
domain](http://en.wikipedia.org/wiki/Public_domain "Public domain"){rel="wikipedia"}.
John Wilbanks, whose [comments on the ChemSpider
licence](http://feeds.feedburner.com/~r/PetermrsBlog/~3/286632057/),
sparked the discussion has written two posts
\[[1](http://network.nature.com/blogs/user/wilbanks/2008/05/10/chemspider-good-intentions-and-the-fog-of-licensing),
[2](http://network.nature.com/blogs/user/wilbanks/2008/05/10/on-the-erosion-of-the-public-domain)\]
which I found illuminating and have made things much clearer for me. His
point is that data naturally belongs in the public domain and that the
public domain and the freedom of the data itself needs to be protected
from erosion, both legal, and conceptual that could be caused by our
obsession with licences. What does this mean for making an effective
data commons, and the [Science
Exchange](http://blog.openwetware.org/scienceintheopen/2008/04/16/the-science-exchange/)
that could arise from it, financially viable?

I start with a couple of premises:

1.  Data belongs in the public domain
2.  An effective and useful data commons requires well structured data
3.  Preparing high quality data costs money, and the tools do not really
    exist to support this in general
4.  Academic career progression depends at core on one thing. How much
    money you bring in.

**Getting the data**

So I started with the notion of paying researchers to make data
available, originally phrased as 'pay the journals to buy papers'. What
I really meant was paying people to put research results somewhere
useful. So let us imagine we can pay people to deposit data (we'll
figure out how later). We don't want to be swamped with rubbish so the
data has to be well structured, tagged up and machine readable. If we're
paying for it, we set the standards. We also want to encourage re-use of
data, perhaps by paying a premium for the deposition of data that
re-uses other data. And in turn, perhaps pay a premium to those whose
data is re-used. The key point is that the presence of capital and cash
flow will drive patterns of behaviour. By being a market maker you
create the market, and to some extent control the behaviour of the
market, whether that be in commodities like coffee, or scientific data.

The data has to be dedicated to the public domain. The value of having a
central organisation, lets call it the Foundation, directing this is
that this dedication can be centrally managed and effectively handled.
In fact, such an organisation would probably outsource the work to an
expert organisation such as [Science
Commons](http://sciencecommons.org/) or [Open Knowledge
Foundation](http://www.openknowledgefoundation.org/). The system could
allow embargoes or delayed release, but this would be a timed release,
not a 'release on publication' clause. The release cannot be
indefinitely delayed. The fundamental principle must remain that
researchers are being paid to support committing their data to the
public domain.

**Providing the systems**

How do we get the big boys interested in this? What will drive Google,
Microsoft, Amazon, IBM to put real money into supporting such a
foundation, activity, or company. I think the answer to this lies in the
commercial opportunities that are created by having an insider knowledge
of the system that is indexing, collating, and crawling the data. The
data is in the public domain, and probably not in a repository as we
understand it, but on the cloud, or federated amongst multiple
repositories. But insider knowledge of the form, structure, location,
and tagging of the data is a commercially priviledged position.

We are only accepting well structured data, and this means providing
systems that make structuring research data easy. I'm not talking here
about a few webforms, or a slightly improved LIMS system. I'm talking
about fully functional laboratory notebook systems that talk to
instruments, that already have the API to most equipment built in, that
will provide helpful hints on how to describe your data using the
appropriate controlled vocabularies and above all something that looks
and behaves like a combination of Word and Google home page. Something
that is so easy to use that people actually do use it; that they use it
because it is useful to them and the like using it. Something that is
going to cost on order of \$100M to build.

But once it is built, you've got the best research recording tool
around. Something you can sell to commercial research organisation to
run in their own systems. Running these systems in house will enable
them to control their competitive research but will also make sharing of
pre-competitive research more effective for them. ELN implementations in
big pharma are budgeted at around the \$10M mark. There is a market to
recoup investments at the \$100M level. Particularly if this can be used
as the starting point for the next generation of generic enterprise data
management tools.

**Making it pay**

The Foundation at the centre of this has priviledged access to the data
through knowing and controlling the system that runs it. It will also
have the most timely access. In addition through providing the tools
(free at point of use for the basic service) that will enable data
deposition it is providing a service to the community. That service may
come with some conditions. First right of refusal to commercialise
opportunities that the foundation identifies; the right to sell access
to the products of processing the data; the right to process embargoed
data to identify commercial opportunities.

Research funders may also be interested in using the Foundation to drive
the availability of data. It is possible to imagine schemes whereby part
of a research grant is funnelled through the organisation to pay for the
deposition of data. Perhaps a 20% premium over what would be available
through a conventional grant could be paid to the researcher on
deposition of the data with perhaps 5% being paid to the Foundation.
Funders are putting hundreds of millions into data centres that no-one
is too sure what to do with. Maybe that money could be used more
effectively to drive data deposition quality. Some funders may also see
this as a good model for direct funding. Putting money in to drive the
generation of specific data set. Channel funding through the foundation
to pay groups to deposit the results rather than pay them to do the
research. For small foundations or charitable concerns this may be a
much more effective means of driving the outcomes they want.

Then there is the service model. This commons is not just for academic
research. Commercial bodies will be interested as well. And they will
want to exploit the insider knowledge the Foundation has of these
systems. Premium services to search, not just for the right data, or the
right connection, but the right person for an internal position, the
right group to collaborate with on a specific problem. The Foundation
will be able to provide premium services on top of the data and it will
be able to charge for them. Scientific journals could buy access for the
lodgement of supplementary data. Why hold it on their own servers,
costing money, and not necessarily being much use when it could be
provided in a more useful form for re-use and further exploitation? Or
perhaps turn it on its head, could the foundation pay journals for the
right to host supplementary data (including all the raw data associated
with a paper). Could this be another route to funding Open Access
journals?

Targeted advertising should not be rejected either. It works rather well
for Google as a money raising venture so there is no reason why it
shouldn't here. Care is obviously required not to allow advertising to
drive the process but as a complementary revenue stream it could be
useful. Overall there are funding streams here that could be exploited
to generate serious quantities of cash flow, running towards the \$M100s
per year, in markets that are much, much bigger, so incremental
increases in efficiency are worth these sums.

**Does it scale?**

This can work on a small scale with small amounts of money. The amounts
of data that are currently available in a useful form are vanishingly
small restricting the potential for a flood of submissions. Structuring
data is hard work and costs money so if the rewards are small and the
tools are not readily available a flood is unlikely. As it grows there
is a problem though. This is essentially a problem of liquidity, a large
body of capital is required to support both the flow of information in,
and the exploitation of that information. It is unlikely that the
foundation will be able to fund the full costs of the research that
generates data. It will be able to pay a market driven rate to support
the care, structuring, and processing of that data. Essentially the
Foundation is buying futures in the forward value of the data. The
complication is that it is also the market maker, and driver for those
futures. In some markets this is called insider trading. Here I don't
think anyone is necessarily losing out.

On a large scale as the dominant (or one of a set of dominant) index I
believe the model is sustainable as long as there is sufficient
liquidity. The problem lies in moving from the small scale to the large
scale, while undertaking the tools development project. Defining the
financial and commercial viability of that trajectory is beyond my
expertise, but I'd be interested to hear what people think. The
foundation can control its capital reserves by not accepting data, but
this is likely to cause the equivalent of a run on a bank. The details
of the business model need be looked at by people with proper business
heads. But it seems to me that this is plausible, if properly planned
and executed.

This will take large sums of money. And it may well not make money. But
it will enable more effective research. And that is worth a lot of
money, even if indirectly.

[![](http://img.zemanta.com/pixie.png?x-id=58b79e9e-4f4c-4a84-ac36-927bd5592805){decoding="async"}](http://www.zemanta.com/ "Zemified by Zemanta")

[![](http://img.zemanta.com/pixie.png?x-id=a5735185-7096-4242-a413-519eca69c874){decoding="async"}](http://www.zemanta.com/ "Zemified by Zemanta")


Image from Wikipedia via Zemanta  Following on from the discussion a few weeks back kicked off by Shirley at One Big Lab and continued here I’ve been thinking about how to actually turn what was a throwaway comment into reality:  There is a problem at the core of this. For someone to pay for access to the results, there has to be a monetary benefit to them.

Rogue Scholar Posts

Digital repositories. An update.

Writing a Wave Robot – Some thoughts on good practice for Research Robots

The growth of linked up data in chemistry – and good community projects

How do we build the science data commons? A proposal for a SciFoo session

More on the science exchance – or building and capitalising a data commons