Unraveling complex biology with biological knowledge graphs. Content licensed under CC BY 4.0.

Charles Tapley Hoyt

I’ve just returned from the 17

 th

Annual International Biocuration Conference at the Indian Biological Data Centre (IBDC) in Faridabad, India.

Discussions and Follow-ups from Biocuration 2024

Using Pydantic for encoding data models and FastAPI for implementing APIs on top of them has become a staple for many Python programmers.

Semantic Pydantic

I finally got back into reading! Over winter break 2022, I started the Stormlight Archive then followed up in 2023 by reading the entirety of Brandon Sanderson’s

 Cosmere

, as well as a some other fantasy, science fiction, and literary fiction. Here’s the list.

Books I Read in 2023

The Unified Medical Language System (UMLS) is a widely used biomedical and clinical vocabulary maintained by the United States National Library of Medicine. However, it is notoriously difficult to access and work with due to licensing restrictions and its complex download system. In the same vein as my previous posts about DrugBank and ChEMBL, this post describes open source software I’ve developed for downloading and working with this data.

Unlocking UMLS

Today’s short post is about three SPARQL queries I wrote to get bibliometric information about journals and publishers out of Wikidata. Each of the following queries can be readily copy-pasted into the Wikidata Query Service and run in the browser.

Querying Journals and Publishers in Wikidata

I was recently nominated for the International Society for Biocuration’s Excellence in Biocuration Early Career Award (results will be announced on June 14

 th

!). This made me curious about how to model nominations and awards on Wikidata. In this post, I’ll describe how to curate awards, nominations, recipients, and how to make SPARQL queries to get them.

Modeling and Querying Awards in Wikidata

Archival Resource Keys (ARKs) are flavor of persistent identifiers like DOIs, URNs, and Handles that have the benefit of being free, flexible with what metadata gets attached, and natively able to resolve to web pages. Name-to-Thing (N2T) implements a resolver for a variety of ARKs, so this blog post is about how that resolver can be re-implemented with the curies Python package. In a lot of ways, ARKs look and act like CURIEs.

Re-implementing the N2T ARK Resolver

I’ve been working on improving reproducibility in the field of cheminformatics for some time now. For example, I’ve written posts about making data from DrugBank and ChEMBL more actionable. Over the last year, I’ve been preparing a concept with the editors of the Journal of Cheminformatics on how to include an assessment of reproducibility to reviews of manuscripts submitted to the journal.

Reproducibility Pilot in the Journal of Cheminformatics

Several controlled vocabularies and ontologies that aren’t themselves OBO Foundry ontologies use unsanctioned OBO PURLs. This post is about how to use the Bioregistry to identify which resources are doing this and to give some insight into how we arrived in this situation. Background on the OBO Foundry   The OBO Foundry is a set of independent, interoperable biomedical ontologies that aspire to using shared development principles.

Resources masquerading as OBO Foundry ontologies

This blog is normally about

 very serious science

, but I’m taking a break from that for the evening to advertize my band’s upcoming show on April 8

 th

in the SPH Music Masters Finale (aka, the German Battle of the Bands). We need your support!

The Representatives of Monkey Jack - German Battle of the Bands Finale

The Open Researcher and Contributor Identifier (ORCID) database is an invaluable resource that supports the unambiguous identification of researchers. However, its first party data dump is too complex, verbose, and unstandardized for many use cases. This post describes open source software I wrote that automates downloading, processing, and exporting ORCID into a more usable form. I put the results on Zenodo under the CC0 license.

Easier ORCID

Biopragmatics

[Archival Resource Keys (ARKs)](https://arks.org/) are flavor of
persistent identifiers like DOIs, URNs, and Handles that have the
benefit of being free, flexible with what metadata gets attached, and
natively able to resolve to web pages. [Name-to-Thing
(N2T)](https://n2t.net) implements a resolver for a variety of ARKs, so
this blog post is about how that resolver can be re-implemented with the
[`curies`{.language-plaintext
.highlighter-rouge}](https://github.com/cthoyt/curies/) Python package.

In a lot of ways, ARKs look and act like CURIEs. For example,
`ark:/53355/cl010277627`{.language-plaintext .highlighter-rouge} could
be interpreted as having the prefix `ark`{.language-plaintext
.highlighter-rouge} and the local unique identifier
`/53355/cl010277627`{.language-plaintext .highlighter-rouge}. The first
part of each ARK between the first two slashes corresponds to the
provider. In this example, `53355`{.language-plaintext
.highlighter-rouge} corresponds to the
[Louvre](https://www.louvre.fr/en) museum in Paris, France and
`cl010277627`{.language-plaintext .highlighter-rouge} is the local
unique identifier corresponding to the Vénus de Milo statue.

However, I might have just committed ARK blasphemy. In N2T, it appears
that the ARK prefix and provider code stay grouped together in the front
half like `ark:/53355/`{.language-plaintext .highlighter-rouge} and then
the back half `cl010277627`{.language-plaintext .highlighter-rouge}
represents the local unique identifier. This is very similar to the
two-layer identifiers in DOI and the arbitrary number of layer
identifiers in OID.

The point is, if we can interpret this enough like CURIEs, we can use
the `curies`{.language-plaintext .highlighter-rouge} package to
implement a resolver. The first step we can take is to download the N2T
data from <https://n2t.net/e/n2t_full_prefixes.yaml>. Then we can parse
out the ARKs (there are other things in N2T we'll disregard) with the
following code:

:::: {.language-python .highlighter-rouge}
::: {.highlight}
``` highlight
import pystow
import yaml

URL = "https://n2t.net/e/n2t_full_prefixes.yaml"
PROTOCOLS = {"https://", "http://", "ftp://"}


def get_prefix_map():
 """Get the prefix map from N2T, not including redundant ``ark:/`` in prefixes."""
 with pystow.ensure_open("n2t", url=URL) as file:
 records = yaml.safe_load(file)
 prefix_map = {}
 for key, record in records.items():
 uri_prefix = record.get("redirect")
 if (
 not uri_prefix
 or all(not uri_prefix.startswith(protocol) for protocol in PROTOCOLS)
 or uri_prefix.count("$id") != 1
 or not uri_prefix.endswith("$id")
 or not key.startswith("ark:/")
 ):
 continue
 key = key.removeprefix("ark:/")
 prefix_map[key] = uri_prefix.removesuffix("$id") + "/" + key + "/"
 return prefix_map
```
:::
::::

This prefix map removes `ark:/`{.language-plaintext .highlighter-rouge}
from the beginning of the prefixes in N2T and also adds the provider
code into the URI prefix to make the URIs more focused on the local
unique identifiers within each provider, rather than the entire ARK
space.

Once we have a prefix map, we can make a
`curies.Converter`{.language-plaintext .highlighter-rouge} and a Flask
web application for resolving in a few lines:

:::: {.language-python .highlighter-rouge}
::: {.highlight}
``` highlight
from curies import Converter, get_flask_app


def get_app():
 """Get an ARK resolver app, noting that it uses a non-standard delimiter and URL prefix."""
 prefix_map = get_prefix_map()
 print(prefix_map)
 converter = Converter.from_prefix_map(prefix_map, delimiter="/")
 app = get_flask_app(converter, blueprint_kwargs=dict(url_prefix="/ark:"))
 return app
```
:::
::::

The two tricks here are:

1. We want to remove the redundant `ark:/`{.language-plaintext
 .highlighter-rouge} then interpret the ARK provider code as the
 prefix and the rest as the local unique identifier. However, we
 still want to be able to write URLs in our resolver that have the
 `ark:/`{.language-plaintext .highlighter-rouge} prefix. Luckily,
 Flask has the facility to define a default
 `url_prefix`{.language-plaintext .highlighter-rouge} before a given
 blueprint that we invoke directly.
2. Unlike CURIEs that use a colon `:`{.language-plaintext
 .highlighter-rouge} as the delimiter between the prefix and local
 unique identifier, ARKs use a slash `/`{.language-plaintext
 .highlighter-rouge}. We can also set this in the Converter's
 settings.

Now, all we need to do is instantiate the app and serve it with any WSGI
tool like Gunicorn, Uvicorn, or Flask's built-in development server
(from Werkzeug). Navigating to
http://localhost:5000/ark:/53355/cl010277627 redirects to
https://collections.louvre.fr/ark:/53355/cl010277627 and gets some nice
art from the Louvre. In general, you can stick any ARK after
http://localhost:5000/ark: that is resolvable via N2T when running this
server.

All of this code is on
[GitHub](https://github.com/cthoyt/n2t-ark-resolver/tree/main) and can
be run with the following:

:::: {.language-shell .highlighter-rouge}
::: {.highlight}
``` highlight
git clone https://github.com/cthoyt/n2t-ark-resolver
cd n2t-ark-resolver
python -m pip install -r requirements.txt
python wsgi.py
```
:::
::::

------------------------------------------------------------------------

Update: since posting this, I have heard from John Kunze that the ARK
format is currently being updated to look more like URNs and therefore
not have the slash after `ark:/`{.language-plaintext .highlighter-rouge}
anymore. If/when that happens, there are only a few bits of string
pre-processing in this script that need to be updated to keep everything
running.