Benefits of sharing code with articles

I'm writing this blog post in preparation for a contribution to the upcoming RDA Plenary 19 session "BoF: Sharing code that supports research articles: is it time to align goals and publisher policies?" at 0030 UTC on 23 June. I want to talk about how and why software (code) is shared.

Publishers

When we think about different stakeholders who are trying to influence the way software is developed and shared, we can consider publishers, who are beginning to require software citation, and in some cases, some types of code sharing as well, as an element that supports published papers. This can enable a variety of positive rewards, including:

• Validation: code sharing enables others to be able to understand (to read and analyze) the computational parts of the work done in the article. This requires the source code to be available (archived) as long as the article is available.
• Reproducibility: code sharing enables others to be able to reproduce (run) the computational parts of the work done in the article. This requires the code to be available (runnable) as long as the article is available, which in turn requires correct versions of dependencies to be available and the build system to work, or the code be frozen virtually, such as in a container.
• Reusability: code sharing enables others to be able to build on code used in the computational parts of the work done in the article, for their own work. This has the same requirements as reproducibility, and also requires solid documentation and examples/training, and ideally, a modular software design.
• Expandability/contributability: code sharing enables others to be able to add to the code used in the computational parts of the work done in the article, to improve it. This requires a working version of code and dependencies to be available, and the code authors to be maintainers who are willing to collaborate with their communities on code.
• Credit: code sharing enables others to be able to cite the software and credit its authors when they use it. This requires clear metadata for the software to exist, stored in an accessible and interpretable (by humans and machines) way.

Funders

In addition, we can also think about funding agencies, which have a mixed record in terms of requirements for software sharing.

For example, NIH has recently created a new policy for data management and sharing (starting Jan 2023), replacing the existing policy that’s from 2003. This new policy "requires researchers to prospectively plan for how scientific data will be preserved and shared through submission of a Data Management and Sharing Plan." However, NIH does not have any clear policy for software. There is a Research Tools Policy, but it seems to have been written to make it hard to understand what it applies to. For example, it doesn't mention software or code explicitly. It does, however, point to a Federal Register item from 1999 that, in a footnote, says "Databases and materials subject to copyright, such as software, are also research tools in many contexts. Although the information provided here may be applicable to such resources, the NIH recognizes that databases and software present unique questions which cannot be fully explored in this document."

While NSF does not have specific detailed requirements about sharing software, high-level requirements in the NSF Proposal & Award Policies & Procedures Guide (PAPPG), 2021, Chapter XI.D.4 include:

• c. Investigators and grantees are encouraged to share software and inventions created under the grant or otherwise make them or their products widely available and usable.
• d. NSF normally allows grantees to retain principal legal rights to intellectual property developed under NSF grants to provide incentives for development and dissemination of inventions, software, and publications that can enhance their usefulness, accessibility and upkeep. Such incentives do not, however, reduce the responsibility that investigators and organizations have as members of the scientific and engineering community, to make results, data and collections available to other researchers.

An example of a forward-looking policy is the current work towards national guidelines on Software Management Plans in the Netherlands. This group has released a draft set of guidelines for community consultation (through 20 June 2022), and I expect this to lead to a requirement for Software Management Plans in the near future.

Institutions

In addition to funders and publishers, we can also think about Institutions, specifically the institutions that employ researchers, such as universities, laboratories, and governments. Some of these institutions have a mission of benefitting society, and it seems that they should care about sharing software to serve this mission. However, in practice, there are few institutions with policies that support this, such as evaluation of such software work in hiring, promotion, and tenure. In part, this is because such a goal of sharing knowledge may be seen as conflicting with other goals, such as profiting from institutionally developed intellectual property or other financial incentives. An example of this is university technology transfer offices that seek to maximize profit rather than maximizing the transfer of technology.

Professional societies

While many professional societies are publishers (as discussed above), they are also the disciplinary communities in which accepted practices can be documented and disseminated. For example, ACM's policies on artifact review and badging can be viewed as a policy for ACM publications, but they also are a set of practices that this part of the computer science community is converging towards, applying to more than just ACM publications. And in Earth science, AGU's guidelines on Software and Data for Authors were partially derived from ESIP's guidelines, which are intended to be discipline-wide and not specific to any one publisher.

Conclusions

Many different stakeholders have been making progress on guidelines for sharing data associated with publications, and progress by one is often discussed in forums such as Research Data Alliance plenaries and working/interest groups. The state of guidelines for software is less advanced and less coordinated, though there are good steps being made by individual stakeholders and some coordination, such as in this RDA plenary session and in other venues such as the Research Software Alliance (ReSA). I look forward to continued progress by individual organizations and increased collaboration, leading to community-accepted policies and practices for software sharing.

This post has also been published as https://doi.org/10.59350/g0rht-eh102 on the Rouge Scholar.