University of Glasgow

Roderic Page

::: {.separator style="clear: both;"}
[![](https://blogger.googleusercontent.com/img/a/AVvXsEh7WW8TlfN2u3xe_E-sH5IK6AWYnAoWaKrP2b32UawUeqMpPlq6ZFk5BtJVZsMmCNh5j3QRsTj5H0Ee55RRGntnc9yjj_mNB8KHmH2dzocCgyLS2VFOxsBji6u4Ey6qxlDAnT-zrsBpnDcTchbhgt1x0Sf7RkmIMkS1y4-_3KCQian-SeIF-g=s200){border="0"
width="128" original-height="1000"
original-width="1000"}](https://blogger.googleusercontent.com/img/a/AVvXsEh7WW8TlfN2u3xe_E-sH5IK6AWYnAoWaKrP2b32UawUeqMpPlq6ZFk5BtJVZsMmCNh5j3QRsTj5H0Ee55RRGntnc9yjj_mNB8KHmH2dzocCgyLS2VFOxsBji6u4Ey6qxlDAnT-zrsBpnDcTchbhgt1x0Sf7RkmIMkS1y4-_3KCQian-SeIF-g=s1000){style="display: block; padding: 1em 0; text-align: center; clear: right; float: right;"}
:::

I\'ve released a very crude GraphQL endpoint for WikiData. More
precisely, the endpoint is for a subset of the entities that are of
interest to WikiCite, such as scholarly articles, people, and journals.
There is a crude demo at <https://wikicite-graphql.herokuapp.com>. The
endpoint itself is at <https://wikicite-graphql.herokuapp.com/gql.php>.
There are various ways to interact with the endpoint, personally I like
the [Altair GraphQL Client](https://altair.sirmuel.design) by [Samuel
Imolorhe](https://github.com/imolorhe).

As I\'ve [mentioned
earlier](https://iphylo.blogspot.com/2021/04/it-been-while.html) it\'s
taken me a while to see the point of GraphQL. But it is clear it is
gaining traction in the biodiversity world (see for example the [GBIF
Hosted Portals](https://dev.gbif.org/hosted-portals.html)) so it\'s
worth exploring. My take on GraphQL is that it is a way to create a
self-describing API that someone developing a web site can use without
them having to bury themselves in the gory details of how data is
internally modelled. For example, WikiData\'s query interface uses
SPARQL, a powerful language that has a steep learning curve (in part
because of the administrative overhead brought by RDF namespaces, etc.).
In my previous SPARQL-based projects such as
[Ozymandias](https://ozymandias-demo.herokuapp.com) and
[ALEC](http://alec-demo.herokuapp.com) I have either returned SPARQL
results directly (Ozymandias) or formatted SPARQL results as [schema.org
DataFeeds](https://schema.org/DataFeed) (equivalent to RSS feeds)
(ALEC). Both approaches work, but they are project-specific and if
anyone else tried to build based on these projects they might struggle
for figure out what was going on. I certainly struggle, and I wrote
them!

So it seems worthwhile to explore this approach a little further and see
if I can develop a GraphQL interface that can be used to build the sort
of rich apps that I want to see. The demo I\'ve created uses SPARQL
under the hood to provide responses to the GraphQL queries. So in this
sense it\'s not replacing SPARQL, it\'s simply providing a (hopefully)
simpler overlay on top of SPARQL so that we can retrieve the data we
want without having to learn the intricacies of SPARQL, nor how Wikidata
models publications and people.


I've released a very crude GraphQL endpoint for WikiData. More precisely, the endpoint is for a subset of the entities that are of interest to WikiCite, such as scholarly articles, people, and journals. There is a crude demo at https://wikicite-graphql.herokuapp.com. The endpoint itself is at https://wikicite-graphql.herokuapp.com/gql.php.

GraphQL for WikiData (WikiCite)

Is it\'s been a while since I\'ve blogged here. The last few months have
been, um, interesting for so many reasons. Meanwhile in my little corner
of the world there\'s been the constant challenge of rethinking how to
teach online-only, whilst also messing about with a bunch of things in a
rather unfocused way (and spending way too much time populating
[Wikidata](https://www.wikidata.org)). So here I\'ll touch on a few
rather random topics that have come up in the last few months, and
elsewhere on this blog I\'ll try and focus on some of the things that
I\'m working on. In many ways this blog post is really to serve as a
series of bookmarks for things I\'d like to think about a bit more.

## Taxonomic precision and the \"one tree\"

One thing that had been bugging me for a while was my inability to find
the source of a quote about taxonomic precision that I remembered as a
grad student. I was pretty sure that David Penny and Mike Handy had said
it, but where? Found it at last:

> Biologists seem to seek "The One Tree" and appear not to be satisfied
> by a range of options. However, there is no logical difficulty with
> having a range of trees. There are 34,459,425 possible trees for 11
> taxa ([Penny et al. 1982](https://doi.org/10.1038/297197a0)), and to
> reduce this to the order of 10-50 trees is analogous to an accuracy of
> measurement of approximately one part in 10^6^.
>
> Many measurements in biology are only accurate to one or two
> significant figures and pale when compared to physical measurements
> that may be accurate to 10 significant figures. To be able to estimate
> an accuracy of one tree in 10^6^ reflects the increasing
> sophistication of tree reconstruction methods. (Note that, on this
> argument, to identify an organism to a species is also analogous to a
> measurement with an accuracy of approximately one in 10^6^.). ---
> \"Estimating the reliability of evolutionary trees\" p.414
> [doi:10.1093/oxfordjournals.molbev.a040407](https://doi.org/10.1093/oxfordjournals.molbev.a040407)

I think this quote helps put taxonomy and phylogeny in the broader
context of quantitative biology. Building trees that accurately place
taxa is a computationally challenging task that yields some of the most
precise measurements in biology.

## Barcodes for everyone

> DNA for 95 specimens in 40 min at negligible cost (not included:
> beverage consumption during 20 min break): this is the 1st video
> demonstrating the methods used in "MinION barcodes: biodiversity
> discovery and identification by everyone, for everyone".
> <https://t.co/W2LuaJ6SYk>
> [pic.twitter.com/eNDk6eU2xa](https://t.co/eNDk6eU2xa)
>
> --- Rudolf Meier (@RudolfMeier15) [March 22,
> 2021](https://twitter.com/RudolfMeier15/status/1373991744388300800?ref_src=twsrc%5Etfw)

This is yet another exciting paper from Rudolf Meier\'s lab (see earlier
blog post [Signals from Singapore: NGS barcoding, generous interfaces,
the return of faunas, and taxonomic
burden](https://iphylo.blogspot.com/2018/07/signals-from-singapore-ngs-barcoding.html)).
The preprint
[doi:10.1101/2021.03.09.434692](https://doi.org/10.1101/2021.03.09.434692)
is on bioRxiv. It feels like we are getting ever-closer to the
biodiversity tricorder.

## Barcodes for Australia

> [\#ArabaBioscan](https://twitter.com/hashtag/ArabaBioscan?src=hash&ref_src=twsrc%5Etfw)
> week 23 <https://t.co/n9zvE2lTlF> Some of the local
> [\#Ichneumonoidea](https://twitter.com/hashtag/Ichneumonoidea?src=hash&ref_src=twsrc%5Etfw)
> diversity from this week\'s Malaise sample
> [\#BIOSCAN](https://twitter.com/hashtag/BIOSCAN?src=hash&ref_src=twsrc%5Etfw)
> [\#entomology](https://twitter.com/hashtag/entomology?src=hash&ref_src=twsrc%5Etfw)
> [\#DNABarcoding](https://twitter.com/hashtag/DNABarcoding?src=hash&ref_src=twsrc%5Etfw)
> [@iBOLConsortium](https://twitter.com/iBOLConsortium?ref_src=twsrc%5Etfw)
> [pic.twitter.com/yR7yg0B7Ys](https://t.co/yR7yg0B7Ys)
>
> --- Donald Hobern (@dhobern) [April 5,
> 2021](https://twitter.com/dhobern/status/1379207567285518336?ref_src=twsrc%5Etfw)

Donald Hobern ([@dhobern](https://twitter.com/dhobern)) has been
[blogging](https://stangeia.hobern.net/category/biodiversity-informatics/bioscan/araba-bioscan/)
about insects collected in malaise traps in Aranda, Australian Capital
Territories (ACT). The insects are being photographed (see stream on
[Flickr](https://flickr.com/photos/tags/arababioscan)) and will be
barcoded.

## No barcodes please we\'re taxonomists!

A paper with a title like \"Minimalist revision and description of 403
new species in 11 subfamilies of Costa Rican braconid parasitoid wasps,
including host records for 219 species\" (Harvey et al.
[doi:10.3897/zookeys.1013.55600](https://doi.org/10.3897/zookeys.1013.55600)
was always likely to cause problems, and sure enough some [taxonomists
had a meltdown](https://taxacom.markmail.org/thread/t4swze2vv6k3mssb). A
lot of the arguments centered around whether DNA sequences counted as
words, which seems surreal. DNA sequences are strings of characters,
just like natural language. Unlike English, not all languages have word
breaks. Consider Chinese for example, where search engines can\'t break
text up into words for indexing, but instead use
[*n*-grams](https://en.wikipedia.org/wiki/N-gram). I mention this simply
because *n*-grams are a useful way to index DNA sequences and to compute
sequence similarly without performing a costly sequence alignment. I
used this technique in my [DNA barcode
browser](https://iphylo.blogspot.com/2020/10/gbif-challenge-success.html).
If we move beyond arguments about whether a picture and a DNA sequence
is enough to describe a species (if all species every discovered were
described this way we\'d arguably be much better off than we are now) I
think there is a core issue here, namely the relative size of the
intersection between taxa that have been described classically (i.e.,
with words) and those described almost entirely by DNA (e.g., barcodes)
will likely drop as more and more barcoding is done, and this has
implications for how we do biology (see [Dark taxa: GenBank in a
post-taxonomic
world](https://iphylo.blogspot.com/2011/04/dark-taxa-genbank-in-post-taxonomic.html)).

## Bioschema

The dream of linked data rumbles on. [Schema.org](https://schema.org) is
having a big impact on standardising basic metadata encoded in web
sites, so much so that anyone building a web site now needs to be
familiar with schema.org if you want your site to do well in search
engine rankings. I made extensive use of schema.org to model
bibliographic data on Australian animals for my [Ozymandias
project](https://iphylo.blogspot.com/2019/04/ozymandias-biodiversity-knowledge-graph.html).

[Bioschemas](https://bioschemas.org) aims to provide a biology-specific
extension to schema.org, and is starting to take off. For example, GBIF
pages for species now have schema.org embedded as JSON-LD, e.g. the page
for [Chrysochloris visagiei Broom,
1950](https://www.gbif.org/species/2432181) has this JSON-LD embedded in
a

In part inspired by Roger Hyam\'s work on WFO I released a [Life Science
Identifier (LSID) Resolver](https://lsid.herokuapp.com) to make LSIDs
resolvable. I\'ll spare you the gory details, but you can think of LSIDs
as DOIs for taxonomic names. They came complete with a decentralised
resolution mechanism (based on Internet domain names) and standards for
what information they return (RDF as XML), and millions were minted for
animal, fungi, and plant names. For various reasons they didn\'t really
take off (they were technically tricky to use and didn\'t return
information in a form people could read, so what were the chances?).
Still, they contain a lot of valuable information for those of us
interested in having lists of names linked to the primary literature.
Over the years I have been collecting them and wanted a way to make them
available. I\'ve chosen a super-simple approach based on [storing them
in compressed form in GitHub](https://github.com/rdmpage/lsid-cache) and
wrapping that repo in simple web site. Lots of limitations, but I like
the idea that LSIDs actually, you know, persist.

## DOIs for Biodiversity Heritage Library

> Six months ago we started BHL\'s Persistent Identifier Working Group,
> so it\'s time for a huge shout out to the amazing efforts of
> [@mlichtenberg](https://twitter.com/mlichtenberg?ref_src=twsrc%5Etfw)
> [@SusanWLynch](https://twitter.com/SusanWLynch?ref_src=twsrc%5Etfw)
> [@fauxbrarian](https://twitter.com/fauxbrarian?ref_src=twsrc%5Etfw)
> [@missellb](https://twitter.com/missellb?ref_src=twsrc%5Etfw)
> [@BHLProgramMgr](https://twitter.com/BHLProgramMgr?ref_src=twsrc%5Etfw)
> & [@rdmpage](https://twitter.com/rdmpage?ref_src=twsrc%5Etfw) to mint
> DOIs for the historic literature on
> [@BioDivLibrary](https://twitter.com/BioDivLibrary?ref_src=twsrc%5Etfw).
> [\#RetroPIDs](https://twitter.com/hashtag/RetroPIDs?src=hash&ref_src=twsrc%5Etfw)
> [pic.twitter.com/s4jJBUEQg9](https://t.co/s4jJBUEQg9)
>
> --- Nicole Kearney (@nicolekearney) [April 1,
> 2021](https://twitter.com/nicolekearney/status/1377416824287977474?ref_src=twsrc%5Etfw)

In between everything else I\'ve been working with BHL to add DOIs to
the literature that they have scanned. Some of this literature is old
and of limited scientific value (but sure looks pretty - Nicole Kearney
is going to take me to task for saying that), but a lot of it is recent,
rich, and scientifically valuable. I\'m hoping that the coming months
will see a lot of this literature emerge from relative obscurity and
become a first class citizen of the taxonomic and biodiversity
literature.

## Summary

I guess something meaningful and deep should go here\... nope, I\'m
done.


Is it's been a while since I've blogged here. The last few months have been, um, interesting for so many reasons.

It's been a while...

Dennis Irorere

Scott Chamberlain

### Introduction

Few months ago, I embarked on a full stack spatial data project at work.
The project kicked off amazingly, until I was almost backed to the wall
when I discovered that some of the data sources were served via a
GraphQL API. Before now, I haven't worked with GraphQL. But, I have
heard a lot about it and how amazing it is for querying data.

GraphQL is a query language for application programming interfaces
(APIs) that prioritizes giving clients exactly the data they request.
It's designed to make APIs flexible, fast and friendly. Basically, it is
used to load data from a server to a client and it does this in a much
more efficient manner than traditional methods and services.

I started off installing the [GraphQL
playground](https://www.graphqlbin.com/v2/new), which is now my go to
tool to interact with GraphQL API and it offers a great workflow to
understand the schema and structure of any GraphQL API. I recommend you
try it out.

After hours of trying out the GraphQL playground, I finally understood
the schema of the GraphQL API endpoint for the project I'm working on.
Well, that was the easy part I must say.

Now that I can query the API, everything should seem great. Right? Well,
it's not. Why? Well, the GraphQL API I was working with fetches the
geospatial data as json file instead of geojson. Probably, because of my
lack of understanding of how GraphQL truly worked with spatial data. The
issue now is, I can see my data. But, just not in the format that I can
work with. I am tempted to talk about the technicality behind it, but I
won't... maybe another time.

### How did I find out about ghql?

At work, our scripting language of choice is R. R is a free software
environment for statistical computing and graphics. Well, that's how
it's officially defined. But, trust me its gradually evolving to do more
than statistical computing and making of beautiful graphics. It was time
for me to programmatically access data with GraphQL API from R. I felt
excited because, I'm an R lover. But, I knew it was a going to be a huge
work, I guess I was ready for the huge work.

After spending days researching about the R packages that can interact
with a GraphQL API, I found three packages. I picked `ghql` over the
others because, it was an rOpenSci package.

### Moving on

In order for R to interact with any GraphQL API, it requires a GraphQL
client. That's where `ghql`, a GraphQL client for R, developed by Scott
Chamberlain comes into play. Still confused? Well, so was I at first. I
tried interacting with a GraphQL server in R and felt like giving up
even before getting anywhere. Hopefully this blog post will assist
others.

<figure class="center">
<img src="GHQL.png" width="350"
alt="GraphQL client and R connection flow" />
<figcaption aria-hidden="true"><p>GraphQL client and R connection
flow</p></figcaption>
</figure>

### Working with `Countries List`, a GraphQL public API

Let's say you were working on a project that required country-specific
data, such as currency, or language. You could get such data from the
[Countries GraphQL API](https://github.com/trevorblades/countries) which
is a public GraphQL API for information about countries, continents, and
languages. This public API uses [Countries
List](https://annexare.github.io/Countries/) and
[provinces](https://github.com/substack/provinces) as data sources, so
the schema follows the shape of those data, with a few exceptions:

- The codes used to key the objects in the original data are available
  as a code property on each item returned from the API.
- The country.continent and country.languages are objects and arrays of
  objects, respectively.
- Each Country has an array of states populated by their
  states/provinces, if any.

Loading the libraries

::: {.highlight}
``` {.chroma tabindex="0"}
library(ghql)library(jsonlite)library(dplyr)
```
:::

Link to the GraphQL schema api

::: {.highlight}
``` {.chroma tabindex="0"}
link <- 'https://countries.trevorblades.com/'
```
:::

Create a new graphqlClient object

::: {.highlight}
``` {.chroma tabindex="0"}
conn <- GraphqlClient$new(url = link)
```
:::

Define a Graphql Query

::: {.highlight}
``` {.chroma tabindex="0"}
query <- 'query($code: ID!){  country(code: $code){    name    native    capital    currency    phone    languages{      code      name    }  }}'
```
:::

The `ghql` query class and define query in a character string

::: {.highlight}
``` {.chroma tabindex="0"}
new <- Query$new()$query('link', query)
```
:::

Inspecting the schema

::: {.highlight}
``` {.chroma tabindex="0"}
new$link
```
:::

``` {tabindex="0"}
##  ##  ## query($code: ID!){##   country(code: $code){##     name##     native##     capital##     currency##     phone##     languages{##       code##       name##     }##   }## }
```

Define a variable as a named list

::: {.highlight}
``` {.chroma tabindex="0"}
variable <- list(  code = "DE")
```
:::

Making a request, passing in the query and then the variables. Then you
convert the raw object to a structured json object

::: {.highlight}
``` {.chroma tabindex="0"}
result <- conn$exec(new$link, variables = variable) %>%  fromJSON(flatten = F)result
```
:::

``` {tabindex="0"}
## $data## $data$country## $data$country$name## [1] "Germany"#### $data$country$native## [1] "Deutschland"#### $data$country$capital## [1] "Berlin"#### $data$country$currency## [1] "EUR"#### $data$country$phone## [1] "49"#### $data$country$languages##   code   name## 1   de German
```

Convert the json data into a tibble object

::: {.highlight}
``` {.chroma tabindex="0"}
country_data <- result$data$country %>%  as_tibble()country_data
```
:::

``` {tabindex="0"}
## # A tibble: 1 x 6##   name    native      capital currency phone languages$code $name##   <chr>   <chr>       <chr>   <chr>    <chr> <chr>          <chr>## 1 Germany Deutschland Berlin  EUR      49    de             German
```

## More examples

Working with a GraphQL API without a defined variable named list

::: {.highlight}
``` {.chroma tabindex="0"}
link <- 'https://countries.trevorblades.com/'# R6 class for constructing graphql queriesconn <- GraphqlClient$new(url = link)## Define query## Create a query class firstqry <- Query$new()## The graphQL server schemaqry$query('x', '{continent(code: "AF") {    countries{    code    name    native    capital    currency    phone    languages {      name    }    }}  }          ')## Execute the queryres <- conn$exec(qry$queries$x)# Convert the the output from raw to json formatres <- jsonlite::fromJSON(res,                          flatten = TRUE)## convert the from json to dataframe objectres_data <- res$data$continent$countries %>%  as_tibble()## Inspect the first 6 rows of the datares_data
```
:::

``` {tabindex="0"}
## # A tibble: 58 x 7##    code  name             native            capital   currency phone languages  ##    <chr> <chr>            <chr>             <chr>     <chr>    <chr> <list>     ##  1 AO    Angola           Angola            Luanda    AOA      244   <df[,1] [1~##  2 BF    Burkina Faso     Burkina Faso      Ouagadou~ XOF      226   <df[,1] [2~##  3 BI    Burundi          Burundi           Bujumbura BIF      257   <df[,1] [2~##  4 BJ    Benin            Bénin             Porto-No~ XOF      229   <df[,1] [1~##  5 BW    Botswana         Botswana          Gaborone  BWP      267   <df[,1] [2~##  6 CD    Democratic Repu~ République démoc~ Kinshasa  CDF      243   <df[,1] [5~##  7 CF    Central African~ Ködörösêse tî Bê~ Bangui    XAF      236   <df[,1] [2~##  8 CG    Republic of the~ République du Co~ Brazzavi~ XAF      242   <df[,1] [2~##  9 CI    Ivory Coast      Côte d'Ivoire     Yamousso~ XOF      225   <df[,1] [1~## 10 CM    Cameroon         Cameroon          Yaoundé   XAF      237   <df[,1] [2~## # ... with 48 more rows
```

### The last dance (conclusion)

So, you have stuck with me this far? Thanks!

My final thought. I think GraphQL can greatly simplify data needs for
both client product developers, server-side engineers and data
scientist. It's still early to assertain the extent of it's impact in
the technological world. But, it seems very promising since the Team
behind GraphQL are continously improving the technology, and there is a
growing community.


Introduction   Few months ago, I embarked on a full stack spatial data project at work. The project kicked off amazingly, until I was almost backed to the wall when I discovered that some of the data sources were served via a GraphQL API. Before now, I haven’t worked with GraphQL. But, I have heard a lot about it and how amazing it is for querying data.

Accessing GraphQL from R

Jeroen Ooms

The new ropensci
[graphql](https://cran.r-project.org/web/packages/graphql/) package is
now on CRAN. It implements R bindings to the [libgraphqlparser C++
library](https://github.com/graphql/libgraphqlparser) to parse GraphQL
syntax and export the syntax tree in JSON format:

::: {.highlight}
``` {.chroma tabindex="0"}
graphql2json("{ field(complex: { a: { b: [ $var ] } }) }")
```
:::

A syntax parser is perhaps not super useful to most end-users, but can
be used to validate graphql queries or implement a GraphQL API in R. We
hope to add more related functionality later on.

## What is GraphQL

From the [GraphQL website](https://graphql.org):

> GraphQL is a query language for APIs and a runtime for fulfilling
> those queries with your existing data. GraphQL provides a complete and
> understandable description of the data in your API, gives clients the
> power to ask for exactly what they need and nothing more, makes it
> easier to evolve APIs over time, and enables powerful developer tools.

GraphQL is a new approach and syntax to structuring API queries. It
provides a modern, and perhaps more natural alternative to e.g. REST or
SQL (which were designed for hypertext and relational data,
respectively). A GraphQL query on the other hand resembles the structure
of the data. For example the query:

``` {tabindex="0"}
{  me {    name  }}
```

Could produce the JSON result:

::: {.highlight}
``` {.chroma tabindex="0"}
{  "me": {    "name": "Luke Skywalker"  }}
```
:::

The Github API [supports GraphQL](https://developer.github.com/v4//) and
illustrates how GraphQL reduces the number of requests and data in
comparison with traditional REST queries.


The new ropensci graphql package is now on CRAN.

Messaggi di Rogue Scholar

GraphQL for WikiData (WikiCite)

It's been a while...

Accessing GraphQL from R

New package graphql: A GraphQL Query Parser