Scott Edmunds

<figure class="wp-block-image size-large">
<img
src="http://gigasciencejournal.com/blog/wp-content/uploads/2022/10/katya-ross-I4YsI1zWq_w-unsplash-1.jpg"
class="wp-image-4600" loading="lazy" decoding="async"
srcset="http://gigasciencejournal.com/blog/wp-content/uploads/2022/10/katya-ross-I4YsI1zWq_w-unsplash-1.jpg 710w, http://gigasciencejournal.com/blog/wp-content/uploads/2022/10/katya-ross-I4YsI1zWq_w-unsplash-1-300x244.jpg 300w"
sizes="(max-width: 710px) 100vw, 710px" width="710" height="578"
alt="interactive figures" />
</figure>

*Frictionless Data improves not just machine readability of scientific
articles, but also enables humans to directly interact with the data
within the article itself. A new article in GigaByte demonstrates a new
workflow where frictionless data can help bring papers to life with
interactive figures.*

The need for information from research outputs to be more findable,
accessible, interoperable, and reusable (FAIR) has spurred researchers,
database managers, and publishers to continually look for new and better
ways to make information machine-readable. Another equally important
area is creating articles that readers can actively engage with, rather
than passively taking in information from reading a published article.
One tool that easily improves machine readability of data is a data
standard called [Frictionless Data](https://frictionlessdata.io/),
developed by the [Open Knowledge Foundation](https://okfn.org/).
Published [today in *GigaByte*](https://doi.org/10.46471/gigabyte.70) a
new Technical Release paper demonstrates that not only does Frictionless
Data drastically improve machine readability, it can also turn normally
static figures within the article into dynamic entities that allow
readers to directly interact with the data within the article.
Demonstrating that the use of Frictionless Data can tackle two important
activities: allowing both man and machine to use and directly engage
with scientific outputs in a dynamic fashion.

Integration of Frictionless Data was carried out on an article by a team
of researchers from the University of Melbourne led by Professor Anthony
Papenfuss, whose lab have been long time advocates of open and
reproducible research (being the [first authors with
*GigaScience*](http://gigasciencejournal.com/blog/parasites-protocols-part-2-author-qa-tony-papenfuss/)
to integrate protocols.io protocols into [their
papers](https://doi.org/10.1186/s13742-016-0129-2) back in 2016). Making
sure the data, source code, and every other sharable component of their
research is openly available to the community. This makes their work
especially amenable to utilising new tools on top of their articles to
make the published work dynamic and actively usable. The article
[here](https://doi.org/10.46471/gigabyte.70) presents two new open
source tools, [svaRetro](https://doi.org/doi:10.18129/B9.bioc.svaRetro)
and [svaNUMT](https://doi.org/doi:10.18129/B9.bioc.svaNUMT), for
interpreting difficult to structural variation in genome analysis. These
help annotate novel genomic events that are missed in most genome
assembly pipelines: such as retrotransposition events and insertion of
DNA fragments from the mitochondria to the nuclear DNA, which contribute
to the complexity of genome sequences and the understanding of gene
function and genome evolution.

Commenting on why they follow these open practices the authors say:
"Reproducibility is a cornerstone of science and a particular focus for
us and our research.  We want to make our data and analyses reproducible
and open so that they are accessible by others, and easy to use and
build upon. We hope that others find our methods useful, and that our
analyses and data are reused in future developments such as benchmarking
studies."

​​**Frictionless Data smoothening the flow of Open Science**  
The openness and [availability](https://doi.org/10.5281/zenodo.7053649)
of all of the research components behind these tools and analyses
created a perfect opportunity to implement Frictionless Data to make the
article far more machine readable. During the process of adding this to
the article, Raniere Silva from City University of Hong Kong, as part of
a FAIR data internship, made the fortuitous discovery that Frictionless
Data could also play a role in improving human interaction with the
article. You can get a behind-the-scenes and more detailed look of his
work this summer in a [guest
blog](http://gigasciencejournal.com/blog/from-frictionless-data-to-interactive-visualisation/).
The figures, for the first time, were regenerated in an interactive
manner. In the example here, readers can not only view the summary
information presented in the figure, they can hover over data points to
see the exact numbers and information behind these, and also manipulate
the figure itself to view specific components that are of interest.

<figure class="wp-block-image size-full">
<img
src="http://gigasciencejournal.com/blog/wp-content/uploads/2022/10/ezgif-2-12ba66f686.gif"
class="wp-image-4601" loading="lazy" decoding="async" width="600"
height="641" alt="interactive figure example" />
<figcaption>Interactive figures on display, demonstrating the
interactivity of Figure 4 in the paper.</figcaption>
</figure>

Ranier says: "My biggest surprise was that the Frictionless Data Package
specifications in conjunction with the popular Plotly tool has functions
to convert a static visualisation into a dynamic one. This massively
reduces the barrier for many researchers to produce dynamic data
visualisation as they only need to add a line or two to their code.
*GigaByte* made a huge leap by publishing the dynamic data visualisation
and I hope it inspires other journals to publish dynamic data
visualisation."

When asked what they found most useful from this process, the authors
stated: "The interactive figures are a great addition to the paper. We
found the interactive functions made reading labels easier, especially
for label-rich figures, and liked that the figures were accessible in
SVG format, allowing viewing and editing without losing information from
the figures."

To promote the use of Frictionless Data in more published articles,
Raniere wrote a [detailed handbook](http://dx.doi.org/10.5524/102316)
that includes an introduction to the use of Frictionless Data, an
introduction to the specifications, short working examples for creating
an author's own data package, and long examples, based on published
articles in *GigaScience* and *GigaByte* journals, illustrating the
creation and use of Frictionless Data. The goal is for [the
handbook](http://dx.doi.org/10.5524/102316) to serve as the start of a
conversation within the scientific community of how to embrace
Frictionless Data. This handbook also provides a resource and guidance
to make things easier and for data producers to submit articles with
these packages to data publishers, such as GigaScience Press.

Of added interest, in addition to the inclusion of Frictionless Data,
paper is that for the first time as the figures were regenerated in an
interactive manner this process combined a [CODECHECK
certificate](https://doi.org/10.5281/zenodo.7084333) of reproducible
computation. We've previously [written about
CODECHECK](http://gigasciencejournal.com/blog/codecheck-certificate/)
and showcased a few
[examples](https://doi.org/10.1093/gigascience/giaa026) of these
certificates in *GigaScience*, but this is the first time the
CODECHECKing process has been worked into the generation of interactive
figures. And also the first example used in the review and publication
process of a *GigaByte* paper.

<figure class="wp-block-image size-large is-resized">
<img
src="http://gigasciencejournal.com/blog/wp-content/uploads/2022/10/Screenshot-2022-10-06-at-12.37.19-PM-1024x683.png"
class="wp-image-4605" loading="lazy" decoding="async"
srcset="http://gigasciencejournal.com/blog/wp-content/uploads/2022/10/Screenshot-2022-10-06-at-12.37.19-PM-1024x683.png 1024w, http://gigasciencejournal.com/blog/wp-content/uploads/2022/10/Screenshot-2022-10-06-at-12.37.19-PM-300x200.png 300w, http://gigasciencejournal.com/blog/wp-content/uploads/2022/10/Screenshot-2022-10-06-at-12.37.19-PM-768x512.png 768w, http://gigasciencejournal.com/blog/wp-content/uploads/2022/10/Screenshot-2022-10-06-at-12.37.19-PM-1536x1025.png 1536w, http://gigasciencejournal.com/blog/wp-content/uploads/2022/10/Screenshot-2022-10-06-at-12.37.19-PM.png 1592w"
sizes="(max-width: 768px) 100vw, 768px" width="768" height="512"
alt="interactive figutes" />
<figcaption>CODECHECK certificate of reproducible computation for the
paper.</figcaption>
</figure>

***And I, for one, welcome our future machine readers***  
The use of Frictionless Data and all the downstream elements it enables,
serves as transformative steps in scientific publishing, as they improve
machine readability and reproducibility, and turn scientific articles
from their old-fashioned static format into a 21^st^ century living
document. Increasing trust and interaction for our human readers through
interactive figures has long been a goal of GigaScience Press (see our
2014 attempts at ["push button"
papers](http://gigasciencejournal.com/blog/carmen-reproducible-research-and-push-button-papers/)
using Knitr and our [showcasing last
year](http://gigasciencejournal.com/blog/gigabyte-executable-research-articles/)
of Stencila's Executable Research Article format). This new approach
demonstrates an integrated way of achieving this in our *GigaByte*
papers using the novel XML-first publishing workflow from River Valley
Technologies. With the Frictionless Data package file for our machine
readers to better interact with and understand the data and software
[available in our GigaDB repository](http://dx.doi.org/10.5524/102318).
These types of novel, data-literate additions to the publication process
are part of the reason *GigaByte* was the winner of the [2022 ALSPS
Innovation in Publishing
Award](http://gigasciencejournal.com/blog/gigabyte-alpsp-innovation-award/)
last month.

<figure
class="wp-block-embed-youtube wp-block-embed is-type-video is-provider-youtube wp-embed-aspect-16-9 wp-has-aspect-ratio">
<div class="wp-block-embed__wrapper">
<div class="iframe">
<div id="player">

</div>
<div class="player-unavailable">
<h1 id="ein-fehler-ist-aufgetreten." class="message">Ein Fehler ist
aufgetreten.</h1>
<div class="submessage">
<a href="https://www.youtube.com/watch?v=ltlZ4HdJ1qY"
target="_blank">Sieh dir dieses Video auf www.youtube.com an</a> oder
aktiviere JavaScript, falls es in deinem Browser deaktiviert sein
sollte.
</div>
</div>
</div>
</div>
</figure>

To encourage more authors to use Frictionless Data in their articles,
all manuscripts submitted to *GigaByte* before the end of 2022 that
include Frictionless Data examples will be given a free APC (currently
\$350). Authors interested in finding out how to do this should contact
the editors at *GigaByte* at editorial@gigabytejournal.com.

**Further Reading:**  
Dong R, Cameron D, Bedo J, Papenfuss AT. (2022). svaRetro and svaNUMT:
modular packages for annotating retrotransposed transcripts and nuclear
integration of mitochondrial DNA in genome sequencing data. GigaByte.
2022. <https://doi.org/10.46471/gigabyte.70>  
  
Dong R; Cameron D; Papenfuss AT (2022): Supporting data for "svaRetro
and svaNUMT: modular packages for annotating retrotransposed transcripts
and nuclear integration of mitochondrial DNA in genome sequencing data"
GigaScience Database. <http://dx.doi.org/10.5524/102318>

Raniere Gaia Costa da Silva. (2022). CODECHECK Certificate 2022-018.
Zenodo. [](https://doi.org/10.5281/zenodo.7030414)
<https://doi.org/10.5281/zenodo.7084333>

The post [Improvements for Man and Machine in Scientific
Publishing](http://gigasciencejournal.com/blog/frictionless-data-interactive-figures/){rel="nofollow"}
appeared first on
[GigaBlog](http://gigasciencejournal.com/blog){rel="nofollow"}.


<em>
 Frictionless Data improves not just machine readability of scientific articles, but also enables humans to directly interact with the data within the article itself.
</em>

Improvements for Man and Machine in Scientific Publishing

*A guest post from our summer data science intern Raniere Silva. Raniere
is a PhD Candidate at City University of Hong Kong and interested in
reproducible research.*

::: {.wp-block-image .is-style-rounded}
<figure class="alignright size-large">
<img
src="http://gigasciencejournal.com/blog/wp-content/uploads/2022/09/H0rIr3cL_150x150.jpg"
class="wp-image-4563" loading="lazy" decoding="async" width="150"
height="150" />
<figcaption><em>Raniere Silva</em></figcaption>
</figure>
:::

During July and August 2022, I was a summer intern at GigaScience Press
investigating how *GigaScience* and *GigaByte* journals could use
Frictionless Data to help researchers make data driven discovery faster.
Collection of data is the most time and financially expensive part of
any research and repetition should be avoided. To avoid repetition of
experiments, many fields have organised infrastructure to share data,
for example [The European Nucleotide Archive
(ENA)](http://www.ebi.ac.uk/ena) from the [European Bioinformatics
Institute](https://www.ebi.ac.uk/) and [Gene Expression Omnibus
(GEO)](https://www.ncbi.nlm.nih.gov/geo/) from the [United State's
National Institutes of Health (NIH)](https://www.nih.gov/). Many of the
infrastructure projects have developed their own data format standards
and combining data can be time consuming. The [Open Knowledge
Foundation](https://okfn.org/) and [Datopian](https://www.datopian.com/)
have proposed a data container format, called [Frictionless
Data](https://frictionlessdata.io/), and are overseeing the development
of tools with the goal of reducing the steps needed to download and
combine data. With their goal of publishing research data and software
in a more reproducible manner, GigaScience Press has already carried out
a number of [reproducibility case
studies](https://doi.org/10.1371/journal.pone.0127612), and experiments
with platforms such as
[CodeOcean](http://gigasciencejournal.com/blog/data-intensive-software-publishing-sailing-the-code-ocean-qa-with-ruibang-luo/)
and
[Stenci.la](http://gigasciencejournal.com/blog/gigabyte-executable-research-articles/).
With that in mind they helped organise this internship to explore the
utility of Frictionless Data in data publishing.

<figure class="wp-block-image">
<img
src="https://lh3.googleusercontent.com/_lnKGbQH2_rhWi0f_HbdmfJ0wssHZedNlAIO2KiWlMrDyNLfw_L5--YrtteB-cGaMB01E3uOG7g_vij0r4wyI8NAzmoO1YSH6FvQktYOTkb5n1J89qNNjdxtPCJTNixnm4R42dDhFngcWVFIRy3n3JTEIoObIAHWlfFLZNqLUaZbXaI8UY7Wq2_nYg"
decoding="async" alt="Frictionless Data image" />
</figure>

A [Frictionless Data package](https://frictionlessdata.io/) is a JSON
file enclosing a list of local or remote resources (the data) and the
meta-information of the package and each resource (for example, author
and licence). The start point of my investigation was how to wrap the
supplement material of an article from *GigaScience* or *GigaByte*
journals stored in their in-house [GigaDB
repository](http://gigadb.org/) as a Frictionless Data package. I'm sure
that some GigaDB users would appreciate a structured version of the list
of resources in a dataset they could use to download many files using
Python or R instead of clicking on the links at the dataset web page but
this was a weak user case because datasets in GigaDB already have a
semi-structured README file.

Searching for a stronger user case to adopt the Frictionless data
package in GigaDB, I started to look at recreating the data
visualisations in a paper from the data provided and add some
interactives to the data visualisation. Because of the short duration of
the internship, I limited my investigation to tabular data that GigaDB
stores as CSV files. Theoretically, the Frictionless data package could
store, in addition to the CSV file, the data visualisation description.
For example, it could store a R snippet like:

``` wp-block-code
ggplot(aes(x=Petal.Length,y=Petal.Width)) +
  geom_point() +
  labs(
    x = "Length",
    y = "Width",
    title ="Petal size"
  )
```

or a Vega snippet like

``` wp-block-code
{
  "$schema": "https://vega.github.io/schema/vega-lite/v5.json",
  "data": {
    "url": "iris.csv"
  },
  "encoding": {
    "x": {
      "field": "Petal\\.Length",
      "type": "quantitative"
    },
    "y": {
      "field": "Petal\\.Width",
      "type": "quantitative"
    }
  },
  "height": 400,
  "mark": "circle",
  "params": [
    {
      "bind": "scales",
      "name": "grid",
      "select": "interval"
    }
  ],
  "width": 400
}
```

Unfortunately, the Frictionless Data ecosystem is still small and no
website currently exists that will receive from the user the URL of a
Frictionless data package and render a data visualisation from it.
Additionally, the cross-origin resource sharing (CORS) policy imposes an
extra bump in the road to produce such a website.

What I discovered during my internship is that [Plotly Open Source
Graphing Libraries](https://plotly.com/graphing-libraries/) provide
wraps to convert matplotlib or ggplot2 data visualisation to HTML based
Plotly data visualisation, for example:

``` wp-block-code
plotly.tools.mpl_to_plotly(data_vis_with_matplotlib)
```

and

``` wp-block-code
plotly::ggplotly(data_vis_with_ggplot2)
```

Quoting [Dr Kirstie
Whitaker](https://www.turing.ac.uk/people/researchers/kirstie-whitaker),
this was "too easy not to do"! I started a [small collection of
examples](https://raniere-phd.gitlab.io/interactive-data-visualisation/)
showing how to use Plotly's wraps and other libraries to create a self
contained HTML based data visualisation that *GigaByte* can embed as
part of the published paper.

Back to Frictionless Data, the main output of my internship is a
[handbook for researchers interested to know more about Frictionless
Data](https://raniere-phd.gitlab.io/frictionless-data-handbook/). The
handbook covers an introduction to use Frictionless Data (see the
screenshot image), an introduction to the specification, minimal working
examples of creating your own data package, and long examples, based on
published articles in *GigaScience* and *GigaByte* journals,
illustrating the creation and use of Frictionless Data. 

<figure class="wp-block-image">
<img
src="https://lh6.googleusercontent.com/oGM5Ea3RBA8Z5SPIZHnev6C-0RTvtbjQQ36yWbmyT2JmVwUf7-WeMvGHrByHFmrnAaIFao4_iuEWWyEFGNT5wFEAFT6k6hMcaNPVpmZE_2y_f58zbzBP6sQui0aNiv1l_SjAfFel_Hew8LA40xiTUlhrfBsd-NpSMplDf6mEeCs7j2vGeGwcbsf3Dw"
decoding="async" />
</figure>

The goal is for the handbook to be the start of a conversation with the
scientific community of how to embrace Frictionless Data. And provide a
resource and guidance to make things easier for data producers to
produce and submit these packages to data publishers such as GigaScience
Press.

Another output of my internship was the simultaneous
[CODECHECK](https://codecheck.org.uk/)ing (and issuing of a
[certificate](https://doi.org/10.5281/zenodo.7030414)) for a yet to be
published paper in *GigaByte*. CODECHECK is a project led by Dr. Stephen
Eglen and Dr. Daniel Nüst that promotes the independent execution of
computations underlying scholarly research articles (see a [previous
post on this in
GigaBlog](http://gigasciencejournal.com/blog/codecheck-certificate/)).
The CODECHECK process was a starting point for joyful philosophical and
technical conversations around the CODECHECK community workflow. 

You should see some examples of these frictionless data packages
published in our journals soon (with some interesting additional
features that have come out of this curation process), so watch this
space for the end products.  
  
**Further Reading:**  
Frictionless Data Handbook for Researchers
[https://raniere-phd.gitlab.io/frictionless-data-handbook/](https://raniere-phd.gitlab.io/frictionless-data-handbook/%20)

The post [From Frictionless Data to Interactive
Visualisation](http://gigasciencejournal.com/blog/from-frictionless-data-to-interactive-visualisation/){rel="nofollow"}
appeared first on
[GigaBlog](http://gigasciencejournal.com/blog){rel="nofollow"}.


<em>
 A guest post from our summer data science intern Raniere Silva. Raniere is a PhD Candidate at City University of Hong Kong and interested in reproducible research.
</em>
During July and August 2022, I was a summer intern at GigaScience Press investigating how
<em>
 GigaScience
</em>
and
<em>
 GigaByte
</em>
journals could use Frictionless Data to help researchers make data driven discovery faster.

Messaggi di Rogue Scholar

Improvements for Man and Machine in Scientific Publishing

From Frictionless Data to Interactive Visualisation