Open Tools and R Packages for Open Science

Rich FitzJohn

Matt Pennell

Amy Zanne

Will Cornwell

Science is reportedly in the middle of a reproducibility crisis. Reproducibility seems laudable and is frequently called for (e.g., nature and science).  In general the argument is that research that can be independently reproduced is more reliable than research that cannot be independently reproduced.

Reproducible research is still a challenge

Steve Moss

A guest blog post by Steve Moss

Why Python? A little background! I started using Python in the summer of 2010. I had applied for the Master of Research postgraduate degree in Computational Biology at the University of York. They teach the programming portion of their course using Python. I thought it might be useful to learn it, before starting, to give me a bit of a head start.

Facilitating Open Science with Python

Scott Chamberlain

Scholarly metadata - the meta-information surrounding articles - can be super useful. Although metadata does not contain the full content of articles, it contains a lot of useful information, including title, authors, abstract, URL to the article, etc. One of the largest sources of metadata is provided via the Open Archives Initiative Protocol for Metadata Harvesting or OAI-PMH.

Scholarly metadata in R

Mike Mahoney

Maëlle Salmon

The package waywiser maintained by Mike Mahoney provides ergonomic methods for assessing spatial models.Assessing predictive models of spatial data can be challenging,both because these models are typically built for extrapolating outside the original region represented by training data and due to potential spatially structured errors,with “hot spots” of higher than expected error clustered geographically due to spatial structure in the

Help make waywiser better! User requests wanted

Noam Ross

We are excited to announce that R-Universe has been awarded a Google Season of Docs grant. R-Universe is rOpenSci’s platform for testing, building, distributing, and discovering R packages, led by Jeroen Ooms.

R-Universe Documentation Gets a Boost from Google Season of Docs

Jeroen Ooms

Abstract / TLDR   R-universe now builds WASM binaries of all R packages for use in WebR applications (such as shinylive). For example to test the dev version of dplyr, you can open the WebR demo page and run: install.packages('dplyr',  repos = c('https://tidyverse.r-universe.dev', 'https://repo.r-wasm.org'))  As explained below, not all R packages are supported yet;

R-universe now builds WASM binaries for all R packages

Alejandra Bellini

Yanina Bellini Saibene

The R-Universe is used by professionals from different fields, although they all pursue the same objective:

 to offer their R packages in a simple and accessible way

. At rOpenSci we work to provide tools that facilitate access to data and research software generated with good software development practices and in a friendly community.

The rOpenSci Multiverse

Introducing cran.dev shortlinks! On r-universe you can find package repositories from many different organizations and maintainers. But sometimes you just want to lookup a particular CRAN package, without knowing the developer.The new cran.dev shortlink service lets you navigate or link directly to the r-universe homepage and docs of any established CRAN package.

New cran.dev shortlinks to package information and documentation

Lucio Casalla

Steffi LaZerte

A new post of our interview series “Meeting the stars of the R-universe”. We aim to introduce the teams and people behind the development of software and packages many of us use and which are available through the R-Universe. We want to highlight and explore different teams and projects around the world, the work they do, their processes and users. Our third stop is the

 United States

to talk with members of the PEcAn project.

Meeting the Stars of the R-Universe: PEcAn, an Open Source Project to Take Care of the Planet

A new post of our interview series “Meeting the stars of the R-universe”. We aim to introduce the teams and people behind the development of software and packages many of us use and which are available through the R-Universe. We want to highlight and explore different teams and projects around the world, the work they do, their processes and users.

Meeting the Stars of the R-Universe: Researching Our Brain with the Magic of the R-Universe

The Champions Program got off to a great start in 2023! We’re happy to report on the first couple of months in our first run of the rOpenSci Champions Program. In September 2022, we launched the program, advertising for both mentors and mentees to apply. We received a total of

 102

applications from

 31

different countries. Seventy-four applications were for champions and twenty eight for mentors.

rOpenSci Champions Program Kick off

The rOpenSci Team

Dear rOpenSci friends, it’s time for our monthly news roundup! You can read this post on our blog.Now let’s dive into the activity at and around rOpenSci!rOpenSci HQ Multilingual Publishing  Open Source and Open Science are global movements, but most of their material and resources are published in English, meaning non-English speakers face a significant barrier to being part of these movements.

rOpenSci News Digest, November 2022

Jeffrey Stevens

In the last post, we used the qualtRics

 1

package to import survey data directly from Qualtrics accounts.I often use Qualtrics for survey data collected via Amazon’s Mechanical Turk, which is an automated system that connects online respondents to paid surveys.However, sometimes those data are not super useful because the quality of the data is poor.In the anonymous, large-scale market of online survey respondents, the financial

Working with Qualtrics Data - Part 2: Excluding Data

Mark Padgham

We recently introduced a new paragraph to the development version of our dev guide  This complements the statistical software review requirement for Bayesian software. This tech note’s aim is to make the new requirement louder, demonstrate some approaches to verbosity control, and to gather community feedback!

Please Shut Up! Verbosity Control in Packages

Karthik Ram

Our team has been cranking out a large number of tools over the past several months. As regular readers are aware, our software packages provide programmatic access to a diverse and extensive trove of scientific data. More recently we’ve expanded our efforts to build more general purpose and cross-domain tools.

rOpenSci developer meeting in March

Dear rOpenSci friends, it’s time for our monthly news roundup! You can read this post on our blog.Now let’s dive into the activity at and around rOpenSci!rOpenSci HQ Recordings of useR! 2021  Recordings of useR! 2021 are now available on the conference website.

rOpenSci News Digest, August 2021

How packages appear in r-universe  Last month we explained how r-universe makes it easy to search and browse through the countless R packages, articles, and datasets to let you discover and learn new things. We are continuously growing this database by adding more R projects, to guide you through everything the R ecosystem has to offer. Currently r-universe is tracking and indexing of over 18.000 R packages.

How r-universe searches for packages on CRAN / Bioconductor

At rOpenSci, much of our code, content and infrastructure is hosted on GitHub over several organizations – described on our resources page.This post summarizes some steps we’ve taken to safeguard our GitHub organizations.Paying attention to access rights &amp;

Safeguards and Backups for GitHub Organizations

Remember our recent post showing that one can wrangle Markdown filesprogrammatically without regex? Thattech note showed how to convert Markdown bodies to XML in order toextract information from them.

tinkr: editing Markdown documents using XML tools

Are you passionate about statistical methods and software? If so we would love for you to join our team to dig deep into the world of statistical software packages. You’ll develop standards for evaluating and reviewing statistical tools, publish, and work closely with an international team of experts to set up a new software review system.

rOpenSci Hiring for New Position in Statistical Software Testing and Peer Review

Adam Sparks

NASA generates and provides heaps of data to the scientific community. Not allof it is looking out at the stars. Some of it is looking back at us here onEarth. NASA’s Earth science program observes, understands and models theEarth system

 1

. We can use these data to discover how our Earth is changing,to better predict change, and to understand the consequences for life on Earth.

POWER to the People

rOpenSci

Stefanie Butland

Ildi Czeller

Bob Rudis

“Security” can be a daunting, scary, and (frankly) quite often a very boring topic. BUT!, we promise that this Community Call on May 7th will be informative, engaging, and enlightening (or, at least not boring)! Applying security best practices is essential not only for developers or sensitive data storage but also for the everyday R user installing R packages, contributing to open source, working with APIs or remote servers.

Community Call - Security for R

Thanks to the second post of the series where we obtained data fromeBird we knowwhat birds were observed in the county of Constance. Now, not allspecies’ names mean a lot to me, and even if they did, there are a lotof them.

What are these birds? Complement occurrence data with taxonomy and traits information

rgbif was seven years old yesterday!What is rgbif? rgbif gives you access to data from the Global Biodiversity Information Facility (GBIF) via their API.

rgbif: seven years of GBIF in R

Sam Albers

Leonardo Collado-Torres

Mauro Lepore

Joyce Robbins

Omayma Said

R packages are widely used in science, yet the code behind them often does not come under scrutiny. To address this lack, rOpenSci has been a pioneer in developing a peer review process for R packages. The goal of pkginspector is to help that process by providing a means to better understand the internal structure of R packages.

What's inside? pkginspector provides helpful tools for inspecting package contents

For the fourth and last day of project recaps from this year’s unconf, here is an overview of the next five projects. (Full set of project recaps: recap 1, recap 2, recap 3, recap 4.) In the spirit of exploration and experimentation at rOpenSci unconferences, these projects are not necessarily finished products or in scope for rOpenSci packages.

Unconf18 projects 4: umapr, greta, roomba, proxy-bias-vignette, http caching

Charles T. Gray

To give you an idea of where I am in my R developer germination, I’d just started reading about testing when I received an email from @rOpenSci inviting me to review the weathercan package. Many of us in the R community feel like imposters when it comes to software development. In fact, as a statistician, it was a surprise to me when I was recently called a developer.

An Ode to Testing, my first review

The problem  Text-mining - the art of answering questions by extracting patterns, data, etc. out of the published literature - is not easy. It’s made incredibly difficult because of publishers. It is a fact that the vast majority of publicly funded research across the globe is published in paywall journals.

fulltext v1: text-mining scholarly works

Tim Trice

What is rrricanes Why Write rrricanes? There is a tremendous amount of weather data available on the internet. Much of it is in raw format and not very easy to obtain. Hurricane data is no different. When one thinks of this data they may be inclined to think it is a bunch of map coordinates with some wind values and not much else. A deeper look will reveal structural and forecast data.

rrricanes to Access Tropical Cyclone Data

Tony Fischetti

Version 2.0 of my data set validation package assertr hit CRAN just this weekend. It has some pretty great improvements over version 1. For those new to the package, what follows is a short and new introduction. For those who are already using assertr, the text below will point out the improvements. I can (and have) go on and on about the treachery of messy/bad datasets.

Data validation with the assertr package

The new magick package is an ambitious effort to modernize and simplify high-quality image processing in R. It wraps the ImageMagick STL which is perhaps the most comprehensive open-source image processing library available today. The ImageMagick library has an overwhelming amount of functionality. The current version of Magick exposes a decent chunk of it, but being a first release, documentation is still sparse.

Advanced Image-Processing in R with Magick, Part I

I’ve recently heard the idea of “offline first” via especially Hood.ie. We of course don’t do web development, but primarily build R interfaces to data on the web. Internet availablility is increasinghly ubiqutous, but there still are times and places where you don’t have internet, but need to get work done.

Caching API calls offline

Previously on this blog and on my own personal blog, I have discussed how easy it is to create interactive maps on Github using a combination of R, git and Github. This is done using a file format called

 geojson

, a file format based on JSON (JavaScript Object Notation) in which you can specify geographic data along with any other metadata.

Style GeoJSON

Today we are pleased to announce that rOpenSci has been awarded a generous 180K grant from the Alfred P. Sloan foundation. This funding will allow us to develop a whole new suite of tools and provide scientists with general purpose toolkits to access various kinds of scientific data. We will also be traveling a whole bunch this year and running workshops at several conferences and universities.

rOpenSci awarded 180K from The Sloan Foundation

Our onboarding processensures that packages contributed by the community undergo atransparent, constructive, non adversarial and open review process.Before even submitting my first R package to rOpenSci onboarding systemin December 2015, I spent a fair amount of time reading through previousissue threads in order to assess whether onboarding was a friendly placefor me: a newbie, very motivated to learn more but a newbie nonetheless.I soon got

The social weather of rOpenSci onboarding system

rOpenSci specializes in creating R libraries for accessing data resources on the web from R. Most times you request data from the web in R with our packages, you should have no problem. However, you evenutally will run into problems. In addition, there are advanced things you can do modifying requests to web resources that fall in the

 advanced stuff

category.

Curling - exploring web request options

Apoorv Anand

Bruna Wundervald

Eunseop Kim

Emi Tanaka

François Michonneau

Francisco Cardozo

Erle Holgersen

Marie Vendettuoli

Jonathan Keane

One important aspect of rOpenSci’s mission is to build capacity and promote passionate community members who help the open science and open source software community grow and improve. The rOpenSci Champions Program is a unique opportunity for us to identify, recognize, and reward such individuals and support them in their work to improve their communities and the wider ecosystem.

Introducing rOpenSci Mentors - Cohort 2023-2024

Thomas J. Leeper

There is no problem in science quite as frustrating as

 other peoples’ data

. Whether it’s malformed spreadsheets, disorganized documents, proprietary file formats, data without metadata, or any other data scenario created by someone else, scientists have taken to Twitter to complain about it. As a political scientist who regularly encounters so-called “open data” in PDFs, this problem is particularly irritating.

Release 'open' data from their PDF prisons using tabulizer

In a recent blog post we discussed caching calls to the web offline, on your own computer. Just like you can cache data on your own computer, a data provider can do the same thing. Most of the data providers we work with do not provide caching. However, at least one does: EOL, or Encyclopedia of Life.

Caching Encyclopedia of Life API calls

In order to facilitate a transformation towards open and reproducible research, rOpenSci is building and improving not only the technical infrastructure, but the social infrastructure as well. To support this, occasionally a Community Call will focus on a topic that reflects the values of rOpenSci.

Community Call v12 - How do I create a code of conduct for my event/lab/codebase?

The R package ecosystem now contains more than 10K packages, and several flagship packages belong under the rOpenSci suite. Some of these are: magick for image manipulation, plotly for interactive plots, and git2r for interacting with git. rOpenSci is a community of people making software to facilitate open and reproducible science/research.

rOpenSci Software Review: Always Improving

Natural history museums have long been valuable repositories of data on species diversity. These data have been critical for fostering and shaping the development of fields such as  biogeography and systematics.

Introducing the ecoengine package

We just released v0.2 of rnoaa. For details on the update, see the release notes.

Changes in rnoaa v0.2.0

Lincoln Mullen

The R package ecosystem for natural language processing has been flourishing in recent days. R packages for text analysis have usually been based on the classes provided by the NLP or tm packages. Many of them depend on Java. But recently there have been a number of new packages for text analysis in R, most notably text2vec, quanteda, and tidytext.

New package tokenizers joins rOpenSci

Juan Pablo Ruiz Nicolini

It was 2015, a Political Science master’s student had to process data to hand in his thesis and decided to take the opportunity to learn how to use R. To the long and winding road of the academic requirement was added an extra degree of difficulty: incorporating programming software from scratch, with a somewhat steep learning curve.

Divide and Conquer: From polar To the Polarverse

Miles McBain

If you read my reflection #1 on rOpenSci Onboarding, then you know I see value in the Onboarding process. A LOT of value even. This post is about where that value lies.

Where is the value in package peer review?

David Winter

A new version of rentrez, our package for the NCBI’s EUtils API, is makingit’s way around the CRAN mirrors. This release represents a substantialimprovement to rentrez, including a new vignettethat documents the whole package.

Rentrez 1_0 released

I’ve raved about the value of extending a personalized welcome to new community members and I recently shared six tips for running a successful hackathon-flavoured unconference. Building on these, I’d like to share the specific approach and (free!) tools I used to help prepare new rOpenSci community members to be productive at our unconference.

The Value of Welcome, part 2: How to prepare 40 new community members for an unconference

At rOpenSci, we create and curate software to help scientists with the data life cycle. These tools access, download, manage, and archive scientific data in open, reproducible ways. Early on, we realized this could only be a community effort. The variety of scientific data and workflows could only be tackled by drawing on contributions of scientists with field-specific expertise. With the community approach came challenges.

How rOpenSci uses Code Review to Promote Reproducible Science

Haydee Svab

Beatriz Milz

We designed the rOpenSci Champions Program with a mentorship aspect. Mentoring plays a significant role in the growth and development of both mentors and mentees alike.

rOpenSci Champions Program Teams: Meet Haydee Svab and Beatriz Milz

Thanks to a quite overdue update of Hugo on our build system

 1

, our website can now harness the full power of Hugo code highlighting for Markdown-based content.What’s code highlighting apart from the reason behind a tongue-twister in this post title?In this post we shall explain how Hugo’s code highlighter, Chroma, helps you prettify your code (i.e.

 syntax highlighting

), and accentuate parts of your code (i.e.

 line

Highlights of Hugo Code Highlighting

This week magick 1.6 appeared on CRAN. This release is a big all-round maintenance update with lots of tweaks and improvements across the package. The NEWS file gives an overview of changes in this version.

Magick 1.6: clipping, geometries, fonts, fuzz, and a bit of history

Pao Corrales

rOpenSci Champions Program Teams: Meet Pao Corrales and Adam Sparks

rOpenSci is seeking mentors to support our inaugural cohort of rOpenSci Champions! The rOpenSci Champions Program is for

 people from historically and systematically excluded groups

who are interested in contributing to rOpenSci and the broader ecosystem of open source and open science communities.

Become a Mentor for rOpenSci Champions!

The R-universe build system  The R-universe system is a complex effort, consisting of numerous frontend and backend pieces that operate across various platforms. A key challenge in developing such a system is managing overall complexity by finding ways to reduce the problem into smaller, loosely coupled components, which can be thought of, and developed, somewhat independently.

A first look at the R-universe build infrastructure

Badges in r-universe  After creating your personal universe, the dashboard on https://{yourname}.r-universe.dev shows the version and other details for each package in your repository. We have also added a new tab that lists the available badges for the repository.

Generating and customizing badges in r-universe

Browsing articles on R-universe In the past weeks we have extended the R-universe build system and front-ends with some new features for publishing

 articles

. Articles can be used for documentation of R packages, but also for other purposes, such as reports, tutorials, papers, or research compendia.Publishing of articles in R-universe is based on the R vignette system, and we may add support for other types of articles later on. The

Publishing and browsing articles on R-universe

Two new packages! We released two new packages that we are using in r-universe to render package documentation: postdoc and prismjs. The goal of postdoc is very simple: generate beautiful single-page package manuals in HTML format. Postdoc uses our prismjs and katex packages for server-side highlighting and math.

postdoc 1.0: minimal and uncluttered HTML package manuals

Michael Quinn

Elin Waring

Theme song:

 PSA

by Jay-Z We announced the testing version of skimr v2 onJune 19, 2018. After more than ayear of (admittedly intermittent) work, we’re thrilled to be able to say thatthe package is ready to go to CRAN. So, what happened over the last year? Andwhy are we so excited for v2?Wait, what is a “skimr”? skimr is an R package for summarizing your data.

(Re)introducing skimr v2 - A year in the life of an open source R project

Marcelo S. Perlin

Package yfR recently passed peer review at rOpenSci and is all about downloading stock price data from Yahoo Finance (YF). I wrote this package to solve a particular problem I had as a teacher: I needed a large volume of clean stock price data to use in my classes, either for explaining how financial markets work or for class exercises. While there are several R packages to import raw data from YF, none solved my problem.

New Package yfR

A new version of the ropensci hunspell package has been released to CRAN. Hunspell is the spell checker library used by LibreOffice, OpenOffice, Mozilla Firefox, Google Chrome, Mac OS-X, InDesign, Opera, RStudio and many others. It provides a system for tokenizing, stemming and spelling in almost any language or alphabet.

Hunspell 2.0: High-Performance Stemmer, Tokenizer, and Spell Checker for R

Sina Rüeger

Julia Gustavsen

TL;DR  rsnps is a package that enables the retrieval of single nucleotide polymorphism (SNP) data from the  NCBI’s dbSNP database and openSNP by providing wrappers for the APIs. Single nucleotide polymorphisms represent differences at one specific position in a detected biological sequence compared to the reference. ncbi_snp_query() now returns all reported variant allele frequencies in dbSNP in column maf_population in form of a tibble.

rsnps 0.5.0: New ncbi_snp_query() Features

The rOpenSci project is a poster child for the fluid collaboration that has become increasingly common these days thanks to platforms like Twitter and GitHub. It has been really inspring to see open discussions take shape as rough ideas, which rapidly turn into prototype research software, all of which are now happening in the order of few days to weeks rather than months to years.

rOpenHack report

Dear rOpenSci friends, it’s time for our monthly news roundup! You can read this post on our blog.Now let’s dive into the activity at and around rOpenSci!rOpenSci HQ R-universe gets advanced search! The landing page for rOpenSci’s R-universe platform has gotten a makeover to facilitate easy searching through thousands of available R packages and articles.

rOpenSci News Digest, March 2022

Dear rOpenSci friends, it’s time for our monthly news roundup! You can read this post on our blog.Now let’s dive into the activity at and around rOpenSci!rOpenSci HQ rOpenSci is hiring its next Community Manager! Come be part of making our great community even better; Help start a new program promoting diverse leadership in open source; Join a remote, flexible, global, friendly community and team!

rOpenSci News Digest, February 2022

Did you know that GitHub lets you refer to the default branch of any repository by substituting the branch name with HEAD in the url? This is a very useful trick to write robust code that works regardless of whether the default branch is called main or master, and will keep working when the default branch gets renamed at some point.

Don't Lose your HEAD over Default Branches

Dear rOpenSci friends, it’s time for our monthly news roundup!

rOpenSci News Digest, April 2022

Abstract / TLDR   R-universe now provides MacOS arm64 binaries for all R packages.

R-universe now builds MacOS ARM64 binaries for use on Apple Silicon (aka M1/M2/M3) systems

Dear rOpenSci community,  This is a bit of a bittersweet announcement… After nearly 13 years, it’s time for me to step down as the Executive Director of rOpenSci. In the summer of 2011 I co-founded rOpenSci alongside a group of dedicated colleagues. What began as a casual collaboration among open science enthusiasts quickly evolved into something far more meaningful.

From the Founding Director: My Farewell to rOpenSci

Following our recent post on “Safeguards and Backups for GitHub Organizations”, nearly one month ago we went one step further and made two-factor authentication (2FA) required for all members and outside collaborators of our main organization, ropensci.It was a timely decision as GitHub since then announced it

 will require all users who contribute code on GitHub.com to enable one or more forms of two-factor authentication (2FA) by the end of

Requiring 2FA for Our Main GitHub Organization

I am pleased, excited, and humbled to announce that I am stepping into the role of Executive Director of rOpenSci starting April 1.  First, let me give my gratitude to our outgoing Executive Director and friend Karthik Ram for his leadership and mentorship running rOpenSci the past decade. He’s been a steady hand and visionary that helped our community accomplish so much together in this time.

Hello from our New Executive Director!

Dear rOpenSci friends, it’s time for our monthly news roundup! You can read this post on our blog.Now let’s dive into the activity at and around rOpenSci!rOpenSci HQ Community call: @ropensci-review-bot help!

rOpenSci News Digest, November 2021

Last week, version 1.0 of the magick package appeared on CRAN: an ambitious effort to modernize and simplify high quality image processing in R. This R package builds upon the Magick++ STL which exposes a powerful C++ API to the famous ImageMagick library.

Magick 1.0: 🎩 ✨🐇 Advanced Graphics and Image Processing in R

Mitchell O'Hara-Wild

Icons in R                Icons can be added to your R Markdown documents using short prefixes which identify the font’s library.fa: Font Awesomeai: Academiconsii: ionicons For example, `r  icon::fa("rocket")` can be used to add the rocket icon from Font Awesome. This interface is convenient if you are familiar with the icon you want, or if you are dynamically selecting your icon.

rOpenSci - open tools for open science

R-universe now builds MacOS ARM64 binaries for use on Apple Silicon (aka M1/M2/M3) systems