rOpenSci

Stefanie Butland

Nick Golding

Chris Grieves

Hugo Gruson

Thomas White

Hao Ye

 

## ![Stefanie Butland](stefanie-butland.jpg){style="margin: 0px 10px; width: 75px;" align="left"} Stefanie Butland, *rOpenSci Community Manager* {#img-srcstefanie-butlandjpg-altstefanie-butland-stylemargin-0px-10px-width-75px-alignleft-stefanie-butland-_ropensci-community-manager_}

Some things are just irresistible to a community manager -- PhD student
Hugo Gruson's recent tweets definitely fall into that category.

<figure>
<img src="sRKKrJ0.png" alt="Pavo tweets" />
</figure>

I was surprised and intrigued to see an example of our software peer
review guidelines being used in a manuscript review, independent of our
[formal collaboration](/blog/2017/11/29/review-collaboration-mee/) with
the journal [Methods in Ecology and Evolution
(MEE)](https://besjournals.onlinelibrary.wiley.com/journal/2041210x).
This is exactly the kind of thing rOpenSci is working to enable by
developing a good set of practices that broadly apply to research
software.

But who was this reviewer and what was their motivation? What role did
the editors handling the manuscript play? I contacted the authors and
then the journal and, in less than a week we had everyone on board to
talk about their perspectives on the process.

 

## ![Nick Golding](nick-golding.jpg){style="margin: 0px 10px; width: 75px;" align="left"} Nick Golding, *Associate Editor, Methods in Ecology and Evolution* {#img-srcnick-goldingjpg-altnick-golding-stylemargin-0px-10px-width-75px-alignleft-nick-golding-_associate-editor-methods-in-ecology-and-evolution_}

To me, MEE's role is to help increase the quality of the methods used in
ecology and evolution, and this includes research software. It would be
great to reach a point where all the research software used in ecology
is at the same high standard as the packages that have been through
rOpenSci software peer review.

Not all R packages that we receive at MEE fit in with the rOpenSci
package scope, but I'd love to see them go through a similar process.
This is where the [rOpenSci review
checklist](https://devguide.ropensci.org/reviewtemplate.html) comes in.
In my view, it's the gold standard 🏆 for reviewing R packages and I was
thrilled to see that Hao (manuscript reviewer) had used it with this
paper.

The idea of doing code review as part of reviewing a manuscript is new
to a lot of people. Often, invited reviewers decline because they don't
think they have the right experience. If you have experience with
creating packages though, reviewing code isn't something to be worried
about. [rOpenSci's
guidelines](https://devguide.ropensci.org/reviewerguide.html) are a
great way for people new to reviewing code to become comfortable with
the process.

 

## ![Hao Ye](hao-ye.jpg){style="margin: 0px 10px; width: 75px;" align="left"} Hao Ye, *Manuscript Reviewer* {#img-srchao-yejpg-althao-ye-stylemargin-0px-10px-width-75px-alignleft-hao-ye-_manuscript-reviewer_}

When I was asked to review the code for the pavo 2.0
manuscript^[1](#fn:1){.footnote-ref role="doc-noteref"}^, I had an
initial moment of panic -- I had no experience doing formal code review.
Luckily, I knew that rOpenSci had a set of reviewing guidelines, and
that a few MEE Applications papers had used them. The same guidelines
are also used by the [Journal of Open Source
Software](https://joss.theoj.org/) (JOSS). Although this submission
wasn't flagged for rOpenSci review, I didn't see a conflict with using
their guidelines for my task.

The checklist helped me to organise my review. I started with the basic
package review template, and then focused on a detailed look at the
primary vignette (which is where I expect most users start). The
rOpenSci guidelines encourage the use of some automated tools, like
`goodpractice` to facilitate reviewing. The hardest part was providing
suggestions to address what the `goodpractice::gp()` function flagged as
complex or redundant code. The remainder of the review went pretty
smoothly. I'm a fan of task checklists, so I'm glad that the authors
found my comments useful. Hopefully the changes will help with the
future maintenance of the package.

 

## ![Thomas White](thomas-white.jpg){style="margin: 0px 10px; width: 75px;" align="left"} ![Hugo Gruson](hugo-gruson.jpg){style="margin: 0px 10px; width: 75px;" align="left"} Thomas White and Hugo Gruson, *Manuscript Authors* {#img-srcthomas-whitejpg-altthomas-white-stylemargin-0px-10px-width-75px-alignleft-img-srchugo-grusonjpg-althugo-gruson-stylemargin-0px-10px-width-75px-alignleft-thomas-white-and-hugo-gruson-_manuscript-authors_}

We were immediately struck by the rigor and thoughtfulness of the
reviews and pleasantly surprised to see reference to rOpenSci in Hao's
\[anonymous\] review. It was clear that Hao and two other reviewers had
invested significant time in examining not only the manuscript and
documentation, but the codebase itself. An uncommon, but welcome
experience.

Our package was singularly improved as a result, both for end-users and
ourselves. Many of the suggestions that we implemented -- such as
comprehensive test coverage, explicit styling, greater code safety,
executable examples, and contributor guidelines -- will persist and
guide the development of this (and related) packages into the future.

We know that software is challenging to review since the overlap of
field-specific expertise between developers and biologists is relatively
limited. This is where the value of rOpenSci's work in developing
tractable standards for reviewers and developers really comes into
focus, as well as the willingness of journals such as MEE to encourage
their use. We're just grateful for the experience and would be thrilled
to see the practice expand in scope and reach where possible.

 

## ![Chris Grieves](chris-grieves.jpg){style="margin: 0px 10px; width: 75px;" align="left"} Chris Grieves, *Assistant Editor, Methods in Ecology and Evolution* {#img-srcchris-grievesjpg-altchris-grieves-stylemargin-0px-10px-width-75px-alignleft-chris-grieves-_assistant-editor-methods-in-ecology-and-evolution_}

Since the early days of the journal, code and software papers (or
Applications articles as we call them) have been really important to
MEE. In our [Policy on Publishing
Code](https://besjournals.onlinelibrary.wiley.com/hub/journal/2041210x/policyonpublishingcode.html?)
we highlight our commitment to ensuring the quality of code through the
peer review process.

We've got a team of dedicated Applications Editors who handle code
manuscripts and they do a great job of balancing their comments on the
manuscript and the code that goes along with it. Resources like the
rOpenSci package review guidelines can really help to take the pressure
off these Editors, and they give reviewers confidence to comment on the
code. It's great to have the chance to promote them here and we hope
that this post will encourage more people to check them out.

[We also partner directly with rOpenSci for software peer
review](https://methodsblog.com/2017/11/29/software-review/). If you
have an R package that meets the aims and scope of both MEE and
rOpenSci, you can opt for a joint review in which the R package is
reviewed by rOpenSci, followed by fast-tracked review of the manuscript
by MEE. Manuscripts published through this process are recognized via a
mark on both HTML and PDF versions of their paper. We've had two
articles published to date as a result of this
partnership^[2](#fn:2){.footnote-ref role="doc-noteref"}^
^[3](#fn:3){.footnote-ref role="doc-noteref"}^.

## Reflections

Having a manuscript reviewed can often feel like a quite mysterious
process. Your work disappears into a black box and comes out with a load
of anonymous suggestions for how to improve it. At rOpenSci and *Methods
in Ecology and Evolution*, we want to help open up that black box.
Thanks to Hugo's tweet of gratitude, and the goodwill of the editors,
reviewers and authors of the pavo 2.0 paper, this post provides a
glimpse of what is possible. Will you give it a try next time?

::: {.footnotes role="doc-endnotes"}

------------------------------------------------------------------------

1.  ::: {#fn:1}
    Maia, R., Gruson, H., Endler, J. A., & White, T. E. (2019). pavo 2:
    new tools for the spectral and spatial analysis of colour in R.
    Methods in Ecology and Evolution.
    <https://doi.org/10.1111/2041-210X.13174> [↩︎](#fnref:1){.footnote-backref
    role="doc-backlink"}
    :::

2.  ::: {#fn:2}
    Sciaini, M., Fritsch, M., Scherer, C., & Simpkins, C. E. (2018).
    NLMR and landscapetools: An integrated environment for simulating
    and modifying neutral landscape models in R. Methods in Ecology and
    Evolution, 9(11), 2240-2248.
    <https://doi.org/10.1111/2041-210X.13076> [↩︎](#fnref:2){.footnote-backref
    role="doc-backlink"}
    :::

3.  ::: {#fn:3}
    Zizka, A., Silvestro, D., Andermann, T., Azevedo, J., Duarte Ritter,
    C., Edler, D., ... & Svantesson, S. CoordinateCleaner: Standardized
    cleaning of occurrence records from biological collection databases.
    Methods in Ecology and Evolution.
    <https://doi.org/10.1111/2041-210X.13152> [↩︎](#fnref:3){.footnote-backref
    role="doc-backlink"}
    :::
:::


Stefanie Butland,
<em>
 rOpenSci Community Manager
</em>
Some things are just irresistible to a community manager – PhD student Hugo Gruson’s recent tweets definitely fall into that category.

When Standards Go Wild - Software Review for a Manuscript

Maëlle Salmon

Noam Ross

Scott Chamberlain

Karthik Ram

rOpenSci is pleased to announce a new collaboration with the [Methods in
Ecology and Evolution
(MEE)](http://besjournals.onlinelibrary.wiley.com/hub/journal/10.1111/(ISSN)2041-210X/),
a journal of the [British Ecological
Society](https://www.britishecologicalsociety.org/), published by Wiley
press ^[1](#fn:1){.footnote-ref role="doc-noteref"}^. Publications
destined for MEE that include the development of a scientific R package
will now have the option of a joint review process whereby the R package
is reviewed by rOpenSci, followed by fast-tracked review of the
manuscript by MEE. Authors opting for this process will be recognized
via a mark on both web and print versions of their paper.

We are very excited for this partnership to improve the rigor of both
scientific software and software publications and to provide greater
recognition to developers in the fields of ecology and evolution. It is
a natural outgrowth of our interest in supporting scientists in
developing and maintaining software, and of MEE's mission of vetting and
disseminating tools and methods for the research community. The
collaboration formalizes and eases a path already pursued by
researchers: The
[**rotl**](https://onlinelibrary.wiley.com/doi/10.1111/2041-210X.12593/abstract),
[**RNexML**](https://onlinelibrary.wiley.com/doi/10.1111/2041-210X.12469/abstract),
and
[**treebase**](https://onlinelibrary.wiley.com/doi/10.1111/j.2041-210X.2012.00247.x/abstract)
packages were all developed or reviewed by rOpenSci and subsequently had
associated manuscripts published in MEE.

## About rOpenSci software review

[rOpenSci](/) is a diverse community of researchers from academia,
non-profit, government, and industry who collaborate to develop and
maintain tools and practices around open data and reproducible research.
The rOpenSci suite of tools is made of core infrastructure software
developed and maintained by the [project staff](/about#team). The suite
also contains numerous packages that are contributed by members of the
broader R community. The volume of community submissions has grown
considerably over the years necessitating a formal system of review
quite analogous to that of a peer reviewed academic journal.

rOpenSci welcomes full software submissions that fit within our [aims
and scope](https://devguide.ropensci.org/policies.html#aims-and-scope),
with the option of a pre-submission inquiry in cases when the scope of a
submission is not immediately obvious. This software peer review
framework, known as the rOpenSci Onboarding process, operates with three
editors and one editor in chief who carefully vet all incoming
submissions. After an editorial review, editors solicit detailed, public
and signed reviews from two reviewers, and the path to acceptance from
then on is similar to a standard journal review process. Details about
the system are described in [various](/blog/2016/03/28/software-review/)
[blog](/blog/2017/09/01/nf-softwarereview/)
[posts](/blog/2017/09/11/software-review-update/) by the editorial team.

## Collaboration with journals

This is our second collaboration with a journal. Since late 2015,
rOpenSci has partnered with the [Journal of Open Source software
(JOSS)](https://joss.theoj.org/), an open access journal that publishes
brief articles on research software. Packages accepted to rOpenSci can
be submitted for fast-track publication at JOSS, in which JOSS editors
may evaluate based on rOpenSci's reviews alone. As rOpenSci's review
criteria is significantly more stringent and designed to be compatible
with JOSS, these packages are generally accepted without additional
review. We have had great success with this partnership providing
rOpenSci authors with an additional venue to publicize and archive their
work. Given this success, we are keen on expanding to other journals and
fields where there is potential for software reviewed and created by
rOpenSci to play a significant role in supporting scientific findings.

## The details

Our new partnership with MEE broadly resembles that with JOSS, with the
major difference that MEE, rather than rOpenSci, leads review of the
manuscript component. Authors with R packages and associated manuscripts
that fit the Aims and Scope for both
[rOpenSci](https://devguide.ropensci.org/policies.html#aims-and-scope)
and
[MEE](http://www.methodsinecologyandevolution.org/view/0/aimsAndScope.html)
are encouraged to first submit to rOpenSci. The
[**rotl**](https://onlinelibrary.wiley.com/doi/10.1111/2041-210X.12593/abstract),
[**RNexML**](https://onlinelibrary.wiley.com/doi/10.1111/2041-210X.12469/abstract),
and
[**treebase**](https://onlinelibrary.wiley.com/doi/10.1111/j.2041-210X.2012.00247.x/abstract)
packages are all great examples of such packages. MEE editors may also
refer authors to this option if authors submit an appropriate manuscript
to MEE first.

On submission to rOpenSci, authors can use our updated [submission
template](https://github.com/ropensci/software-review/issues/new) to
choose MEE as a publication venue. Following acceptance by rOpenSci, the
associated manuscript will be reviewed by an expedited process at MEE,
with reviewers and editors having the knowledge that the software has
already been reviewed and the public reviews available to them.

Should the manuscript be accepted, a footnote will appear in the web
version and the first page of the print version of the MEE article
indicating that the software as well as the manuscript has been
peer-reviewed, with a link to the rOpenSci open reviews.

As with any collaboration, there may be a few hiccups early on and we
welcome ideas to make the process more streamlined and efficient. We
look forward to the community's submissions and to your participation in
this process.

Many thanks to MEE's Assistant Editor Chris Grieves and Senior Editor
Bob O'Hara for working with us on this collaboration.

::: {.footnotes role="doc-endnotes"}

------------------------------------------------------------------------

1.  ::: {#fn:1}
    See also MEE's post from today at
    <https://methodsblog.wordpress.com/2017/11/29/software-review/> [↩︎](#fnref:1){.footnote-backref
    role="doc-backlink"}
    :::
:::


rOpenSci is pleased to announce a new collaboration with the Methods in Ecology and Evolution (MEE), a journal of the British Ecological Society, published by Wiley press
<sup>
 1
</sup>
. Publications destined for MEE that include the development of a scientific R package will now have the option of a joint review process whereby the R package is reviewed by rOpenSci, followed by fast-tracked review of the manuscript by MEE.

Messages de Rogue Scholar

When Standards Go Wild - Software Review for a Manuscript

Announcing a New rOpenSci Software Review Collaboration