Interesting Replicable "Badge" for journal articles

I am writing to point out an interesting experiment that’s started up in the ACM Transactions on Mathematical Software (TOMS) computational journal.  There’s an editorial by Mike Heroux that explains the process.

In brief, TOMS has created a “Replicated Computational Results (RCR)” review process, which is really a designation saying that the computational results published in an article are replicable.  As you might expect, this adds an extra burden on the authors to submit their work in a way that enables this RCR review, an extra burden on the editor to work with this part of the submission, and a completely new burden on the RCR reviewer.  If this process is successfully completed, the paper gets an RCR designation on its cover page, the journal editors hopefully get higher quality work, and the reviewer gets satisfaction in a job well done and a credited contribution to the field.

This seems a really interesting experiment; I hope it succeeds.

p.s.  I’m also personally happy that the first paper that received this designation is an output of an NSF-funded SI2 project: Field G. Van Zee and Robert A. van de Geijn. 2015. BLIS: A framework for rapidly instantiating BLAS functionality. ACM Trans. Math. Softw. 41, 3 (May 2015), with James Willenbring writing the first RCR review.

Note

Since the Reproducible Scientific Computing blog created and curated by Doug Thain has become inactive, this is a repost from there.

Disclaimer

Some work by the author was supported by the National Science Foundation (NSF) while working at the Foundation; any opinion, finding, and conclusions or recommendations expressed in this material are those of the author and do not necessarily reflect the views of the NSF.