Cameron Neylon

[![Enzyme](http://upload.wikimedia.org/wikipedia/commons/thumb/9/97/Phenylalanine_hydroxylase_brighter.jpg/202px-Phenylalanine_hydroxylase_brighter.jpg){decoding="async"}](http://commons.wikipedia.org/wiki/Image:Phenylalanine_hydroxylase_brighter.jpg)[](http://commons.wikipedia.org/wiki/Image:Phenylalanine_hydroxylase_brighter.jpg)

A story of two major retractions from a well known research group has
been getting a lot of play over the last few days with a [News
Feature](http://www.nature.com/news/2008/080514/full/453275a.html) (1)
and
[Editorial](http://www.nature.com/nature/journal/v453/n7193/full/453258b.html)
(2) in the 15 May edition of
[Nature](http://www.nature.com/nature/index.html "Nature (journal)"){rel="homepage"}.
The story turns on claim that Homme Hellingaâ€™s group was able to
convert the [E.
coli](http://en.wikipedia.org/wiki/Escherichia_coli "Escherichia coli"){rel="wikipedia"}
ribose binding protein into a [Triose phosphate
isomerase](http://en.wikipedia.org/wiki/Triosephosphate_isomerase "Triosephosphate isomerase"){rel="wikipedia"}
(TIM) using a computational design strategy. Two papers on the work
appeared, one in Science
([3](http://www.ncbi.nlm.nih.gov/pubmed/15218149)) and one in J Mol Biol
([4](http://www.ncbi.nlm.nih.gov/pubmed/17196220)). However another
group, having obtained plasmids for the designed enzymes, could not
reproduce the claimed activity. After many months of work the group
established that the supposed activity appeared to that of the
bacteriaâ€™s native TIM and not that of the designed enzyme. The
paperâ€™s were retracted and Hellinga went on to accuse the graduate
student who did the work of fabricating the results, a charge of which
she was completely cleared.

Much of the heat the story is generating is about the characters
involved and possible misconduct of various players, but thatâ€™s not
what I want to cover here. My concern is about how much time, effort,
and tears could have been saved if all the relevant raw data was made
available in the first place. Demonstrating a new enzymatic activity is
very difficult work. It is absolutely critical to rigorously exclude the
possibility of any contaminating activity and in practice this is
virtually impossible to guarantee. Therefore a negative control
experiment is very important. It appears that this control experiment
was carried out, but possibly only once, against a background of
significant variability in the results. All of this lead to another
group wasting on the order of twelve months trying to replicate these
results. Well, not wasting, but correcting the record, arguably a very
important activity, but one for which they will get little credit in any
meaningful sense (an issue for another post and mentioned by Noam Harel
in a comment at the [News
Feature](http://www.nature.com/news/2008/080514/full/453275a.html)
online).

So what might have happened if the original raw data were available?
Would it have prevented the publication of the papers in the first
place? Itâ€™s very hard to tell. The referees were apparently convinced
by the quality of the data. But if this was â€˜typical dataâ€™ (using
the special scientific meaning of typical *vis* â€˜the best weâ€™ve
gotâ€™) and the referees had seen the raw data with greater variability
then maybe they would have wanted to see more or better controls;
perhaps not. Certainly if the raw data were available the second group
would have realised much sooner that something was wrong.

And this is a story we see over and over again. The selective
publication of results without reference to the full set of data; a
slight shortcut taken or potential issues with the data somewhere that
is *not revealed* to referees or to the readers of the paper; other
groups spending months or years attempting to replicate results or
simply use a method described by another group. And in the meantime
graduate students and postdocs get burnt on the pyre of scientific
'progress' discovering that something isn't reproducible.

The Nature editorial is subtitled â€˜Retracted papers require a thorough
explanation of what went wrong in the experimentsâ€™. In my view this
goes nowhere near far enough. There is no longer any excuse for not
providing *all* the raw and processed data as part of the supplementary
information for published papers. Even in the form of scanned lab book
pages this could have made a big difference in this case, immediately
indicating the degree of variability and the purity of the proteins.
Many may say that this is too much effort, that the data cannot be
found. But if this is the case then serious questions need to be asked
about the publication of the work. Publishers also need to play a role
by providing more flexible and better indexed facilities for
supplementary information, and making sure they are indexed by search
engines.

Some of us go much further than this, and believe that [making the raw
data
immediately](http://drexel-coas-elearning.blogspot.com/2006/09/open-notebook-science.html)
available is a better way to do science. Certainly in this case it might
have reduced the pressure to rush to publish, might have forced a more
open and more thorough scrutiny of the underlying data. This kind of
radical openness is not for everyone perhaps but it should be less prone
to gaffes of the sort described here. I know I can have more faith in
the work of my group where I can put my fingers on the raw data and
check through the detail. We are still going through the process of
implementing this move to complete (or as complete as we can be)
openness and its not easy. But it helps.

Science has moved on from the days where the paper could only contain
what would fit on the printed pages. It has moved on from the days when
an informal circle of contacts would tell you which groupâ€™s work was
repeatable and which was not. The pressures are high and potential for
career disaster probably higher. In this world the reliability and
*completeness* of the scientific record is crucial. Yes there are
technical difficulties in making it all available. Yes it takes effort,
and yes it will involve more work, and possibly less papers. But the
only thing that ultimately can really be relied on is the raw data
(putting aside deliberate fraud). If the raw data doesnâ€™t form a
central part of the scientific record then we perhaps need to start
asking whether the usefulness of that record in its current form is
starting to run out.

1.  Editorial Nature 453, 258 (2008)
2.  Wenner M. Nature 453, 271-275 (2008)
3.  Dwyer, M. A. , Looger, L. L. & Hellinga, H. W. Science 304,
    1967â€"1971 (2004).
4.  Allert, M. , Dwyer, M. A. & Hellinga, H. W. J. Mol. Biol. 366,
    945â€"953 (2007).

[![](http://img.zemanta.com/pixie.png?x-id=64627f7e-70a4-4208-8802-2b5de61f7b84){decoding="async"}](http://www.zemanta.com/ "Zemified by Zemanta")


A story of two major retractions from a well known research group has been getting a lot of play over the last few days with a News Feature (1) and Editorial (2) in the 15 May edition of Nature. The story turns on claim that Homme Hellingaâ€™s group was able to convert the E. coli ribose binding protein into a Triose phosphate isomerase (TIM) using a computational design strategy.

Messaggi di Rogue Scholar

Avoid the pain and embarassment – make all the raw data available