Jeffrey Stevens

In the last post, we used the qualtRics
<sup>
 1
</sup>
package to import survey data directly from Qualtrics accounts.I often use Qualtrics for survey data collected via Amazon’s Mechanical Turk, which is an automated system that connects online respondents to paid surveys.However, sometimes those data are not super useful because the quality of the data is poor.In the anonymous, large-scale market of online survey respondents, the financial

Working with Qualtrics Data - Part 2: Excluding Data

Janani Ravi

Steffi LaZerte

In March we held a [Community Call](/commcalls/2020-03-18/) discussing
the maintenance of R packages.This call included a [starting
presentation](https://speakerdeck.com/juliasilge/maintaining-an-r-package)
by [Julia Silge](/author/julia-silge/) followed by a discussion
featuring panelists with a wide [variety of
backgrounds](/blog/2020/03/04/commcall-mar2020/#speakers): [Elin
Waring](/author/elin-waring), [Erin Grand](/author/erin-grand),
[Leonardo Collado-Torres](/author/leonardo-collado-torres) and [Scott
Chamberlain](/author/scott-chamberlain).

<figure class="center">
<img src="speakers.png" width="800"
alt="Headshots of the moderator and four panelists" />
</figure>

The rOpenSci Package Maintenance Community Call was my (Janani's) first
Community Call and I loved it.For R/software-dev newbies, learning the
right terminology/lingo/vocabulary is everything.It can take a few dozen
blog posts and many hours of reading before beginners get to the 'aha'
moment of 'oh, these are the terms I need to use to search for what I'm
thinking!'.As there are many online resources out there, the default
expectation is that one can search for and learn almost everything
provided one knows the right keywords.There's nothing like hearing a
lively technical banter of experts to pick up the vernacular that one
can easily build upon.The first-hand tips and tricks, do's and don'ts,
personal anecdotes of what worked beautifully and what crashed terribly,
offered by years of experience are yet unmatched in bringing newbies
into speaking the community's language.That is the precise gap filled in
by the timely and helpful rOpenSci Package Maintenance Community Call!

During rOpenSci Community Calls, there is a [shared
document](https://docs.google.com/document/d/1KvuVNU4ICE_FHvJMAfKns_pZnOt_1cgJ4wokxGGuDaE/)
that allows anyone to add notes or questions during the discussion.I
always take notes for my own use but quickly realized the benefit for
all by taking collaborative notes.This live shared document helps
everyone, including newbies, think through and formulate what they would
like to say.It also gives people the option to participate without
having to interrupt and speak up on the call (thus reducing the barrier
for people, especially newcomers, to ask questions).The document also
gives an opportunity for anyone in the community to share their
expertise and add their perspective.

We felt that the rich content in the video and collaborative notes
document from this call warranted a post that points readers to the
material to ensure more people benefit from it. Here we've collated the
questions and links to the various answers to facilitate look up.

## Summarizing questions and answers

### Moderator questions

**1. What does it mean to "maintain an R
package"?**([video](https://vimeo.com/399048757#t=17m01s) \|
[document](https://docs.google.com/document/d/1KvuVNU4ICE_FHvJMAfKns_pZnOt_1cgJ4wokxGGuDaE/edit#heading=h.w8lncwza4rft))

**2. How do you manage issues / feature requests?**
([document](https://docs.google.com/document/d/1KvuVNU4ICE_FHvJMAfKns_pZnOt_1cgJ4wokxGGuDaE/edit#heading=h.89gqj51qdulg))  
**What workflows do you use to do this?**
([document](https://docs.google.com/document/d/1KvuVNU4ICE_FHvJMAfKns_pZnOt_1cgJ4wokxGGuDaE/edit#heading=h.ccqqertq2gox))  
([video](https://vimeo.com/399048757#t=22m04s))

**3. What is a path for new contributors to R packages? How can healthy
norms be passed on?**([video](https://vimeo.com/399048757#t=30m25s) \|
[document](https://docs.google.com/document/d/1KvuVNU4ICE_FHvJMAfKns_pZnOt_1cgJ4wokxGGuDaE/edit#heading=h.iasdrvr3cnko))

(Related: What should someone do if they want to start helping maintain
one of your packages? First step?)

**4. What considerations go into decisions around dependency
management?** ([video](https://vimeo.com/399048757#t=33m37s))  
**APIs that change?** ([video](https://vimeo.com/399048757#t=35m45s))  
([document](https://docs.google.com/document/d/1KvuVNU4ICE_FHvJMAfKns_pZnOt_1cgJ4wokxGGuDaE/edit#heading=h.g0dhr5i6flm3))

**5. What does the process of changing maintainers look like? What sets
up this transition for
success?**([video](https://vimeo.com/399048757#t=38m27s) \|
[document](https://docs.google.com/document/d/1KvuVNU4ICE_FHvJMAfKns_pZnOt_1cgJ4wokxGGuDaE/edit#heading=h.6624x8niikrz))

**6. We've talked about a lot of different facets of package management.
Which are the same vs. different for internal
packages?**([video](https://vimeo.com/399048757#t=41m09s))

**7. How do you decide to submit to a centralized repository like CRAN
or Bioconductor? Peer review like JOSS or rOpenSci? Stay only on
GitHub?**([video](https://vimeo.com/399048757#t=42m43s) \|
[document](https://docs.google.com/document/d/1KvuVNU4ICE_FHvJMAfKns_pZnOt_1cgJ4wokxGGuDaE/edit#heading=h.ebytpo161oow))

**8. What does someone need to know or skills they need to have to start
maintaining a package?**([video](https://vimeo.com/399048757#t=48m01s)
\|
[document](https://docs.google.com/document/d/1KvuVNU4ICE_FHvJMAfKns_pZnOt_1cgJ4wokxGGuDaE/edit#heading=h.uao03hoinrd0))

### Audience questions

**Daniel Sjoberg**  
What is the best practice for ensuring continued support for older R
versions when dependencies of dependencies of dependencies keep upping
the minimum version of R
required?([document](https://docs.google.com/document/d/1KvuVNU4ICE_FHvJMAfKns_pZnOt_1cgJ4wokxGGuDaE/edit#heading=h.u6de4rghsnnj))

**Athanasia Monika Mowinckel**  
Do any of you have a package that depends on software that is not from
R, for instance another command line tool.I.e. your R package wraps
system calls to the command line software and uses output from that in
R.How do you improve user experience when the program needs environment
variables R does not pick up with
`Sys.getenv()`?([document](https://docs.google.com/document/d/1KvuVNU4ICE_FHvJMAfKns_pZnOt_1cgJ4wokxGGuDaE/edit#heading=h.5mz42hvitrpf))

**Janani Ravi**  
I would also like to know how to incorporate a significant part of non-R
scripts within the R package workflow -- functions that use
system/system2 calls, for instance!Are there any systematic ways and
good examples that could point to
this?([document](https://docs.google.com/document/d/1KvuVNU4ICE_FHvJMAfKns_pZnOt_1cgJ4wokxGGuDaE/edit#heading=h.zgb6oefaddub))

**Lennert Schepers**  
If I want to start with fixing an issue/contributing to a
well-documented package, what are essential parts that I should know
before fixing?Should I learn package documentation and
testing/awcontinuous integration and all other aspects of R packages...
or are there parts that are necessary and others that are less
urgent?Are there things that should be absolutely avoided? (things that
can break the whole
package)?([document](https://docs.google.com/document/d/1KvuVNU4ICE_FHvJMAfKns_pZnOt_1cgJ4wokxGGuDaE/edit#heading=h.t2hv36r5c6dz))

**Eric Scott**  
What are some best practices for making big changes to a package?For
example, changing the output of a function from a list to a
tibble.Should this be a new function?Should there be an argument to get
the old behavior back?  
Warnings to users to update their
code?([document](https://docs.google.com/document/d/1KvuVNU4ICE_FHvJMAfKns_pZnOt_1cgJ4wokxGGuDaE/edit#heading=h.8mnxkd5s2m4e))

## Resources

You can also check out the [list of
resources](https://docs.google.com/document/d/1KvuVNU4ICE_FHvJMAfKns_pZnOt_1cgJ4wokxGGuDaE/edit#heading=h.d6sc6cmn6n4)
related to this community call.

## Excited for more?

We hope this has inspired you to join in with future community
calls!What would YOU like to hear about on a future rOpenSci Community
Call?Add your votes and ideas for topics and speakers by 1) browsing the
[issues in this public
repository](https://github.com/ropensci-org/community-calls); 2) giving
a
![](https://emoji.slack-edge.com/T026GCWK4/thumbsup_all/50096a1020.gif){style="max-width:20px"}
or commenting on an issue; 3) opening a new issue if your idea doesn't
fit in any others.


In March we held a Community Call discussing the maintenance of R packages.This call included a starting presentation by Julia Silge followed by a discussion featuring panelists with a wide variety of backgrounds: Elin Waring, Erin Grand, Leonardo Collado-Torres and Scott Chamberlain.

Maintaining an R Package - Community Call Summary

Julia Silge

rOpenSci is one of the first organizations in the R community I ever
interacted with, when I participated in the [2016 rOpenSci
unconf](https://juliasilge.com/blog/i-went-to-ropensci/). I have since
[reviewed several rOpenSci packages](/software-review/) and been so
happy to be connected to this community, but I have never submitted or
maintained a package myself. All that changed when I heard the call for
a new maintainer for the
[qualtRics](https://github.com/ropensci/qualtRics) package. "IT'S GO
TIME," I thought. 😎

[Qualtrics](https://www.qualtrics.com/) is an online survey and data
collection software platform. Qualtrics is used across many domains in
both academia and industry for online surveys and research, including by
me at my day job as a data scientist at [Stack
Overflow](https://stackoverflow.com/). While users can manually download
survey responses from Qualtrics through a browser, importing this data
into R is then cumbersome. The qualtRics R package implements the
retrieval of survey data using the Qualtrics API and aims to reduce the
pre-processing steps needed in analyzing such surveys. This package has
been a huge help to me in my real day-to-day work, and I have been so
grateful for the excellent work of the package's original authors,
including the previous maintainer [Jasper Ginn](https://jasperginn.io/).
This package is currently the only package on CRAN that offers
functionality like this for Qualtrics' API, and is included in the
official Qualtrics API documentation.

The previous maintainer of this package was no longer using Qualtrics in
his work life and needed to step away from maintaining qualtRics. Also,
early in 2018, the package had been archived on CRAN for a fairly minor
issue: using [testthat](https://testthat.r-lib.org/) for tests but [not
having it included in `Suggests`](http://r-pkgs.had.co.nz/tests.html).
So for me, that meant this project would involve several steps: making
improvements I wanted to the package for usability (especially with
regards to authentication credentials), bringing the package in line
with CRAN policies, and getting the package back on CRAN but as a new
maintainer of an archived package.

### Qualtrics and the Stack Overflow Developer Survey

The timing of when I did the usability work for this package was either
wonderful or terrible, depending on your perspective. I did that work
concurrently with the analysis for the [2019 Stack Overflow Developer
Survey](https://insights.stackoverflow.com/survey/2019). This survey is
the largest and most comprehensive survey of people who code around the
world each year, and this year we had almost 90,000 respondents who
answered questions covering everything from their favorite technologies
to what music they listen to while coding.

The survey is fielded using Qualtrics and I've used the qualtRics
package for the past several years as part of my workflow. This year, I
made changes to the package while I was working with the Developer
Survey data. I'll be honest; it was... A LOT to juggle at one time. 😳
However, it did allow me to test out improvements and changes
iteratively with the real world data I had at hand, and I think I ended
up with a better result than I would have otherwise. We use Qualtrics
for other surveys at Stack Overflow throughout the year, so I am not
just a package maintainer but also a year-round, real-life user of this
package.

After the bulk of my analysis work was done for the survey, I turned to
[getting qualtRics back on
CRAN](https://cran.r-project.org/web/packages/policies.html). The
previous maintainer, who has been unfailingly helpful throughout this
process, sent an email to CRAN with the details involved (his name, my
name, the fact of the package's archiving). Then I submitted the package
to CRAN, basically as if it were a new package. I got some feedback back
from CRAN initially, to use more single quotes in `Description` around
the word "Qualtrics" and also some remarks about how my examples were
surrounded by `\dontrun{}` as well as a reminder not to have my package
write to the user's home filespace. I was somewhat puzzled by the last
items, because nothing in this package does write to the user's home
filespace and the examples all are calling the API and thus need
authentication (which is clearly explained in the
documentation/examples). I was somewhat worried 😩 but I ended up
responding via email and submitting again with the same text in CRAN
comments, explaining those points. Fortunately, the package was accepted
to CRAN that time!

### GOOD IDEA/BAD IDEA

Changing the names of all the functions in a package already being used
in the real world is, generally, a bad idea. **I did it anyway,
everybody.** Let me try to convince you that this was the right call.

- When I took over as package maintainer, the qualtRics package had been
  off CRAN for about nine months. If ever there is an appropriate time
  for a revamp in user-facing function names, it's a time like this,
  when there is a long absence from a centralized repository.
- The most important user-facing functions in this package previously
  had names like `getSurvey()` and `getSurveys()`, which users
  (including me) found difficult to distinguish between. These functions
  called the Qualtrics API, but **not** the part of the API that
  Qualtrics has named `getSurvey`. It was a confusing situation.
- When this package was [first reviewed by
  rOpenSci](https://github.com/ropensci/software-review/issues/192), the
  original maintainer noted that the function names did not conform to
  rOpenSci standards but he preferred to keep them as is since the
  package was already on CRAN. Opening up the possibility of this kind
  of more drastic change now allowed me to bring the function and
  argument names within [rOpenSci
  standards](https://devguide.ropensci.org/building.html#function-and-argument-naming).

The old versions of the functions currently still work but give a
warning, suggesting that the user try out the new versions. If this
change causes more than a passing inconvenience to you as a user, I AM
SO SORRY. I believe this decision will set up the qualtRics package for
improved usability in the long term.

### Explore the new qualtRics!

Another change with this release is a [pkgdown site for
qualtRics](https://ropensci.github.io/qualtRics/). I used the custom
pkgdown template [rotemplate](https://docs.ropensci.org/rotemplate/)
created for rOpenSci packages to style the site, and I am so pleased
with the results! If you are a new user for the qualtRics package, or
someone who wants to poke around and see what's changed and how to get
up to speed with the update, I recommend checking out this [spiffy new
site](https://ropensci.github.io/qualtRics/) for all the details. 🚀


rOpenSci is one of the first organizations in the R community I ever interacted with, when I participated in the 2016 rOpenSci unconf. I have since reviewed several rOpenSci packages and been so happy to be connected to this community, but I have never submitted or maintained a package myself. All that changed when I heard the call for a new maintainer for the qualtRics package. “IT’S GO TIME,” I thought.

Messaggi di Rogue Scholar

Working with Qualtrics Data - Part 2: Excluding Data

Maintaining an R Package - Community Call Summary

Relaunching the qualtRics package