Cameron Neylon

I think it is fair to say that even those of us most enamored of
post-publication peer review would agree that its effectiveness remains
to be demonstrated in a convincing fashion. Broadly speaking there are
two reasons for this; the first is the problem of social norms for
commenting. As in there aren't any. I think it was [Michael
Nielsen](http://michaelnielsen.org) who referred to the
"[Kabuki](http://en.wikipedia.org/wiki/Kabuki) Dance of scientific
discourse". It is entirely allowed to stab another member of the
research community in the back, or indeed the front, but there are
specific ways and forums in which it is acceptable to do. No-one quite
knows what the appropriate rules are for commenting on online fora, as
best [described most
recently](http://stevekochscience.blogspot.com/2009/06/my-first-rating-and-commenting-of-plos.html)
by [Steve Koch](http://stevekochscience.blogspot.com).

My feeling is that this is a problem that will gradually go away as we
evolve norms of behaviour in specific research communities. The current
"rules" took decades to build up. It should not be surprising if it
takes a few years or more to sort out an adapted set for online
interactions. The bigger problem is the one that is usually surfaced as
"I don't have any time for this kind of thing". This in turn can be
translated as, "I don't get any reward for this". Whether that reward is
a token for putting on your CV, actual cash, useful information coming
back to you, or just the warm feeling that someone else found your
comments useful, rewards are important for motivating people (and
researchers).

One of the things that links these two together is a sense of loss of
control over the comment. Commenting on journal web-sites is just that,
commenting on *the* *journal's website*. The comment author has "given
up" their piece of value, which is often not even citeable, but also
lost control over what happens to their piece of content. If you change
your mind, even if the site allows you to delete it, you have no way of
checking whether it is still in the system somewhere.

In a sense, when the Web 2.0 world was built it was got nearly precisely
wrong for personal content. For me [Jon Udell](http://blog.jonudell.net)
has written most clearly about this when he talks about the
publish-subscribe pattern for successful frameworks. In essence *I*
publish my content and you choose to subscribe to it. This works well
for me, the blogger, at this site, but it is not so great for the
commenter who has to leave their comment to my tender mercies on my
site. It would be better if the commenter could publish their comment
and I could *syndicate* it back to my blog. This creates all sorts of
problems; it is challenging for you to aggregate your own comments
together and you have to rely on the functionality of specific sites to
help you follow responses to your comments. Jon wrote about this better
than I can in [his blog
post](http://blog.jonudell.net/2009/03/06/hosted-lifebits-meets-infobus/).

So a big part of the problem could be solved if people streamed their
own content. This isn't going to happen quickly in the general sense of
everyone having a web server of their own -- it still remains too
difficult for even moderately skilled people to be bothered doing this.
Services will no doubt appear in the future but current broadcast
services like twitter offer a partial solution (its "my" twitter
account, I can at least pretend to myself that I can delete all of it).
The idea of using something like the twitter service at
[microrevie.ws](http://microrevie.ws/how-to) as
[suggested](http://friendfeed.com/science-2-0/5c1d0de0/sketch-of-rating-scheme-for-science-online-gave)
by [Daniel Mietchen](http://ways.org/en/blog/daniel) this week can go a
long way towards solving the problem. This takes a structured tweet of
the form @hreview {Object};{your review} followed optionally by a number
of asterisks for a star rating. This doesn't work brilliantly for papers
because of problems with the length of references for the paper, even
with shortened dois, the need for sometimes lengthy reviews and the
shortness of tweets. Additionally the twitter account is not
automatically associated with a unique research contributor ID. However
the principle of the author of the review controlling their own content,
while at the same time making links between themselves and that content
in a linked open data kind of way is extremely powerful.

Imagine a world in which your email outbox or local document store is
also webserver (via any one of an emerging set of tools like
[Wave](http://wave.google.com), [DropBox](http://getdropbox.com), or
[Opera Unite](http://unite.opera.com)). You can choose who to share your
review with and change that over the time. If you choose to make it
public the journal, or the authors can give you some form of credit. It
is interesting to think that author-side charges could perhaps be
reduced for valuable reviews. This wouldn't work in a naive way, with
\$10 per review, because people would churn outÂ  large amounts of
rubbish reviews, but if those reviews are out on the linked data web
then their *impact* can be measured by their page rank and the authors
rewarded accordingly.

Rewards and control linked together might provide a way of solving the
problem -- or at least of solving it faster than we are at the moment.


I think it is fair to say that even those of us most enamored of post-publication peer review would agree that its effectiveness remains to be demonstrated in a convincing fashion. Broadly speaking there are two reasons for this; the first is the problem of social norms for commenting. As in there aren’t any.

Rogue Scholar Beiträge

Pub-sub/syndication patterns and post publication peer review