You’ll remember that in the last installment (before Matt got distracted and wrote about archosaur urine), I proposed a general schema for aggregating scores in several metrics, terming the result an LWM or Less Wrong Metric.

I said last time that my new paper on Better ways to evaluate research and researchers proposes a family of Less Wrong Metrics, or LWMs for short, which I think would at least be an improvement on the present ubiquitous use of impact factors and H-indexes. What is an LWM?

Like Stephen Curry, we at SV-POW! are sick of impact factors. That’s not news. Everyone now knows what a total disaster they are: how they are signficantly correlated with retraction rate but not with citation count; how they are higher for journals whose studies are less statistically powerful; how they incentivise bad behaviour including p-hacking and over-hyping.

My paper "Surfacing the deep data of taxonomy" (based on a presentation I gave in 2011) has appeared in print as part to a special issue of Zookeys : The manuscript was written shortly after the talk, but as is the nature of edited volumes it's taken a while to appear. My tweet about the paper sparked some interesting comments from David Shorthouse.

I attended the recent ALM Workshop 2013 and data challenge hosted by Public Library of Science (PLOS) in San Francisco. The workshop covered various issues having to do with altmetrics, or article-level metrics (ALM). The same workshop last year definitely had a feeling of we don’t know x, y, and z , while the workshop this year felt like we know a lot more. There were many great talks - you can see the list of speakers here.

We recently had a paper come out in a special issue on article-level metrics in the journal Information Standards Quarterly. Our paper basically compared article-level metrics provided by different aggregators. The other papers covered various article-level metrics topics from folks at PLOS, Mendeley, and more. Get our paper.

As has now been widely reported, NISO have a $200K grant from the Alfred P Sloan Foundation to develop standards for AltMetrics. Why? If there’s one consistent lesson from standardisation processes, it’s that standards which codify existing practice do well, while those that try to invent new practice in the form of a standard do badly.

One consequence of having a database of literature with external identifiers such as DOIs is that we can plug into a bunch of external services to get additional information about a reference. For example, altmetric can take a DOI and display some article level metrics. As an experiment I've added code for altmetric badges to the web page in BioNames that displays publications.

The holy grail of research assessment is a means of automatically tracking the way research changes the way practitioners act in the real world. How does new research influence policy? Where has research been applied by start-ups? And have new findings changed the way medical practitioners treat patients?

This is an edited version of the text that I spoke from at the Altmetrics Workshop in Koblenz in June. Impact as re-use and the way it enables us to reframe the argument around the impact and dissemination of curiosity driven research.