Exploring metadata, communities, and new ideas. 

Metadata Game Changers

Ted Habermann

The Doe family consists of mother Jane and her son John. Identifying people and relationships is an important function of metadata. The ISO TC211 metadata standards provide a comprehensive framework for many important metadata use cases.

 Could ISO metadata be used to describe the Doe family?

ISO metadata describes objects with types and properties.

Understanding ISO Metadata - Objects and Properties

The ROR web interface and API work well if you have a small number of affiliation strings that need identifiers, but what if you have several hundred organization names or thousands of affiliation strings? Try RORRetriever if you have a big ROR search and don’t know where to start!

Cite this blog as Habermann, T. (2022). Need help searching for RORs? Try RORRetriever! Front Matter.

Need help searching for RORs? Try RORRetriever!

California Digital Library (CDL) and Dryad recently announced a partnership to address researcher needs and to “move the needle” on digital data adoption throughout the scholarly research community by working together to understand and respond to researcher needs for high-quality scientific data publishing infrastructures.

Dryad Data Packages and Files

I recently introduced a simple visualization of data from the CrossRef Participation Reports that provides quantitative insight into how completeness of CrossRef metadata collections with respect to eleven key metadata elements has changed between the backfile and the current record collections.

The Big Picture - Has CrossRef metadata completeness improved?

Domain repositories build strong communities of people and organizations that contribute data, expertise and scientific results. These communities thrive on active connections between data, papers, people, and organizations. Multiple contributions and real-world connections make domain repositories great places for adoption of identifiers and virtual connections that build the PID graph.

As I have been working with domain repositories to understand and describe their practices and apply for Core Trust Seal certification, I have been struck by the close, long-term relationships that these repositories form with their communities.

Improving Domain Repository Connectivity: Establishing a Baseline

In honor of PIDapalooza 2021 and 25 hours of Persistent Identifier discussion happening today, we thought a post celebrating the power of PIDs for increasing connectivity and how to measure that connectivity is in order. Let’s party on 🥳 Discovery

 Before Conectivity (BC)

: For many years we have thought about discovery in terms of web portals with text search boxes, maps, timelines, facets, or some other kind of interface.

Power of PIDS: Measuring Connectivity

Measuring metadata completeness is a great way to identify good examples and low-hanging fruit for improvement. A Pidapalloza talk from last week described results from a recent assessment of 144 repositories from the TIB DataCite Consortium with respect to recommendations for FAIR metadata. Overall, the seven mandatory DataCite fields dominate these repositories, but several great examples that go beyond the mandatory were identified.

We just finished 24 hours of PIDapalooza  last week with talks and demos and an amazing amount of information presented by PID experts and users all over the world. It was a truly international meeting happening all over the world with presentations and discussions in many languages.

A PID Feast for Research – PIDapalooza 2021

Erin Robinson

We are thrilled to announce that Metadata Game Changers is partnering with Openscapes, housed at the National Center for Ecological Analysis and Synthesis on a new NASA-funded, three year project!

Metadata Game Changers Partners with Openscapes in New NASA-funded Project

Non-unique organizations and acronyms can cause problems in ROR searches. Use supporting information if you have it to get better results. If not, beware of theses gotchas.

Last October I pulled together a list of organization names that occur multiple times in the ROR database. These non-unique organization names represent the most extreme case of the essential problem that RORs are trying to address, ambiguous affiliations.

Sometimes a Name is Not Enough - Update

Identifying good examples of consistent affiliation information providers within the Crossref community and automating the identification of organizations are important steps towards the goal of assigning identifiers (e.g. RORs) to these organizations.

One of the interesting challenges encountered in the migration of Dryad metadata technology into the California Digital Library is the identification of organizations associated with the research data archived in the Dryad Repository. This information is not included in the original Dryad metadata, so digging around for it, i.e. “metadata archeology”, is required.

Metadata Archeology – Finding Affiliations in Crossref Metadata

I will describe how the DataCite metadata facets help me improve my understanding of DataCite metadata usage in this blog and introduce some software that can help you answer your questions about DataCite metadata usage.

Cite this blog as Habermann, T. (2022). DataCite Facets: Understanding DataCite Usage. Front Matter.

DataCite Facets: Understanding DataCite Usage

ORCIDs for UCAR employees include many more works than other kinds of items and publishers and PID infrastructure providers provide most of the data.

Cite this blog as Habermann, T. (2023). ORCID Profiles – What’s There and Where Did It Come From? Front Matter.

ORCID Profiles – What’s There and Where Did It Come From?

All scientific communities have been linking research together for many years using references to related work in articles. Recently these communities are exploring options for linking to datasets and software. As part of this effort, the CodeMeta Project recently proposed a vocabulary for metadata for code based on schema.org.

Metadata for Using and Understanding Software

We are in the early days of documenting output management plans using DataCite metadata so it is an important time to identify and adopt common practices that help realize the exciting goals of machine-actionable DMPs.  Common practices will facilitate the creation of interoperable DMPs and the development of tools across the research community that help us all reap the on-going benefits of these plans throughout the research life-cycle.

One of the exciting new capabilities in Version 4.4 of the DataCite Metadata Schema is the capability to create unique identifiers (DOIs) for Output Management Plans, i.e., resourceTypeGeneral = OutputManagementPlan. This new resource type recognizes that research outputs can include many kinds of resources beyond just data: journal articles, software, computational notebooks, presentations, etc.

Connecting DMPs and Research Outputs in DataCite Metadata

Let’s recognize leaders in the adoption of organizational identifiers in DataCite metadata. They are making tracks for us to follow!

Version 4.3 of the DataCite Metadata Schema released during August, 2019 included (among other things), the capability to provide persistent identifiers for affiliated organizations in the metadata (Dasler and deSmaele, Identify your affiliation with Metadata Schema 4.3, 2019). This capability builds on the work and enthusiasm generated by the ROR Community that has championed the concept of open organization identifiers for several years

Early Adopters of Organizational Identifiers in DataCite

Many datasets and products are documented using approaches and tools developed by data collectors to support their analysis and understanding. This documentation exists in notebooks, scientific papers, web pages, user guides, word processing documents, spreadsheets, data dictionaries, PDF’s, databases, custom binary and ASCII formats, and almost any other conceivable form, each with associated storage and preservation strategies.

Documentation or Metadata?

The DataCite metadata model includes rich information about over 18 million items with DOIs. The schema includes at least four elements that describe organizations: publisher, creator, contributor, and fundingReference. Each of these elements has interesting characteristics. Let’s explore them!

In my last blog I introduced Metadata Archeology with a description of digging around in Crossref metadata for affiliations associated with authors of work published in the Dryad Data Repository. I showed how the Crossref Participation Reports could be used for an initial survey of Crossref metadata like satellite images are being used in the GlobalXplorer project to find ancient sites in Peru.

Metadata Archeology: Hunting Affiliations and RORs in DataCite Metadata

Need facet data in a machine-readable format? The new option —facetdata can help!

Cite this blog as Habermann, T. (2022). How Many When? Front Matter.

How Many When?

DataCite has a long history of registering DOIs for a variety of resource types.

Cite this blog as Habermann, T. (2022). How Many When (All Time). Front Matter.

How Many When (All Time)

The UNAVCO community is the most common creator of datasets in the UNAVCO repository. They need to be recognized for those contributions. Adding an identifier for the community increases the % of DOIs with complete connectivity from 6 to 30%.

Most current metadata standards recognize that people and organizations play similar roles in the creation and management of datasets and other research objects. This dichotomy has been managed with the introduction of the concept of ‘party’ which, for example, could be a person, organization, or position in the ISO TC211 metadata standards for geographic data.

Improving Domain Repository Connectivity: Person or Organization?

I recently introduced a simple metric for measuring metadata collection completeness with respect to elements in the CrossRef Participation Reports. The suggestion of this metric immediately led to speculation about relationships between collection size and completeness. Small collections include fewer records – are they more likely to be complete? Publishers with large collections have more resources – do they have more complete metadata?

Metadata Evolution - Metadata Completeness, Agility, and Collection Size

DataCite subject metadata can play an important role in dataset classification and discovery, but these elements are uncommon in the repository and need some improvements. Establishing metrics for completeness and quality is an important first step in the improvement process.

Martin Fenner from DataCite recently described the benefits of some standardization of sources for vocabularies used for three DataCite metadata elements: language, rights, and subjects. All of these elements:  can play important roles in the dataset discovery and selection processes at DataCite  are implemented using shared vocabularies that are identified by associated Scheme elements, e.g. subjectScheme, and  are optional.

DataCite Subject Metadata

Metadata records are high-value information hubs that connect users to related resources in journals, books, and on the web.

High-quality metadata can help users discover data and other resources in many ways. Good titles, abstracts, and keywords, along with spatial and temporal extents, go a long way with many search strategies. Adding schema.org metadata to landing pages can also help. A second discovery strategy, more like browsing, comes into play once someone finds something interesting.

Discovery Through Links

“Existing metadata are an exciting learning set for discerning patterns that can be used to streamline the metadata creation process. We want to ensure that data are being submitted to the best fit repository with the right metadata.

Metadata Game Changers, the Center for Expanded Data Annotation and Retrieval (CEDAR) at Stanford University, and Dryad are thrilled to announce their joint National Science Foundation two-year EAGER award focused on increasing the quality of disciplinary metadata and bridging the gap between generalist and disciplinary data repositories.

Metadata Game Changers, Stanford University, and Dryad receive NSF Funding to improve metadata quality and connect repositories

The CrossRef Participation Reports provide a wealth of information about completeness of CrossRef metadata collections and, equally interesting, comparisons between two time periods (backfile and current). These comparisons provide unique opportunities to examine metadata evolution through time. This blog introduces a tool that can be used to visualize this evolution.

Over the last several years CrossRef has grown into one of the largest and most significant metadata repositories in the world. It contains over 100,000,000 registered content items and offers many services to help members and other users take advantage of that content. In addition to managing registered content, CrossRef makes links between resources. It is the connective tissue for the web of scholarly communications.

Metadata Evolution - CrossRef Participation Reports

The idea that the language we use to talk about things shapes the way we think or can think about those things has been around since the 1800’s and even has a name, the Sapir–Whorf hypothesis, proposed during 1954.

Talking and Thinking About Metadata

The


 presentation


 given at the AGU fall meeting by Ted Habermann focused on how repository re-curation could help repositories of all kinds respond to the guidance in the


 OSTP Public Access Memo


 published during August 2022.

Cite this blog as Habermann, T. (2023). Repository Re-Curation for Open Science. Front Matter.

Repository Re-Curation for Open Science

“Project” is being added to the DataCite ResourceTypeGeneral vocabulary in the next release. Many DataCite members are already describing projects with DataCite metadata. Establishing a baseline is a step towards establishing community guidelines and examples of good ideas for how projects might be used.

Cite this blog as Habermann, T. (2024). Projects@DataCite Revisited. Front Matter. https://doi.org/10.59350/ejd15-gvd72.

Projects@DataCite Revisited

Version 4.5 of the DataCite Metadata Schema\ includes several changes supporting the identification and description of instruments. Several DataCite members were describing instruments in DataCite metadata before this capability was introduced and others are beginning to do it now. These existing efforts can inform the development of community conventions and help the broader community understand how to use instrument metadata effectively.

Ted Habermann, Metadata Game Changers   Cite this blog as Habermann, T. (2024). Instruments@DataCite. Front Matter. https://doi.org/10.59350/8fjx7-y3229. Version 4.5 of the DataCite Metadata Schema, released during January 2024, includes several changes supporting the identification and description of instruments.

Instruments@DataCite

Universities with the most complete FAIR DataCite metadata are identified using a community convention for FAIR DataCite metadata that support findability with text and identifiers, connections and contacts.

Ted Habermann and Erin Robinson, Metadata Game Changers   Cite this blog as: Habermann, T. and Robinson, E., (2024). FAIR DataCite Metadata - University and College Bright Spots. Front Matter.

FAIR DataCite Metadata - University and College Bright Spots

Funder metadata is becoming more important as the global research infrastructure is engaged as a tool for quantifying impact of funders on research results and as interest in open science increases. Using acronyms in funder names makes it difficult to find identifiers and make the connections that are needed to realize benefits.

Ted Habermann, Metadata Game Changers  Jamaica Jones, University of Pittsburgh  Howard Ratner, Tara Packer, CHORUS Introduction   Cite this blog as Habermann, T., Jones, J., Ratner, H., Packer, T., (2024). Funder Acronyms Are Still Not Enough. Front Matter.

Funder Acronyms Are Still Not Enough

Metadata Game Changers and the INFORMATE Project had the opportunity to present some of our recent work during the recent culminating conference to showcase the outcomes, coalition-building efforts, and ongoing work stemming from the 2023 Year of Open Science (YOS).  Some highlights are described here and a recording of the talk is also available.

Ted Habermann, Metadata Game Changers  Jamaica Jones, University of Pittsburgh  Cite this blog as Habermann, T. (2024). Making the Invisible Visible: Celebrating the Year of Open Science. Front Matter.

Making the Invisible Visible: Celebrating the Year of Open Science

The CHORUS Dashboard provides visualizations and a variety of reports from the Global Research Infrastructure for Federal agencies and other users. The INFORMATE project focuses on three of the CHORUS Reports: All, Author Affiliation and Dataset. The goal of the first phase of the project is to understand the contents of the CHORUS reports and the data collection and processing that bring the data to the reports, i.e. the CHORUS Data Journey.

::::: {#block-4d0d36c5315635abda26 .sqs-block .html-block .sqs-block-html block-type=“2” border-radii=“{"topLeft":{"unit":"px","value":0.0},"topRight":{"unit":"px","value":0.0},"bottomLeft":{"unit":"px","value":0.0},"bottomRight":{"unit":"px","value":0.0}}”} Ted Habermann, Metadata Game Changers  Jamaica Jones, University of Pittsburgh  Howard Ratner and Tara Packer, CHORUS   ::::: Introduction   Cite this blog as Habermann, T. , Jones, J.,

CHORUS Data Journeys

A recent talk introduces the concept of repository re-curation as a means of improving connectivity with identifiers while complying with the Nelson Memo. Examples from three types of research organizations are described: Dryad, a generalist repository, institutional repositories, and biologic field stations. Slides and a video of the talk are included.

Cite this blog as Habermann, T. (2023). Responding to the Nelson Memo: Repository Re-Curation for Open Science. Front Matter.

Responding to the Nelson Memo:Repository Re-Curation for Open Science

DataCite metadata with the term Project in the resourceType occur in at least ten repositories, with the majority in the Center for Open Science (cos.osf) and Zenodo (cern.zenodo). Samples of these metadata were examined to determine how projects are represented in DataCite. While single repositories show consistent patterns, no clear overall pattern emerged.

Cite this blog as Habermann, T. (2023). Project Metadata in DataCite. Front Matter.

Project Metadata in DataCite

The Dryad data publishing platform and community have provided secure storage, curation, discovery and access for datasets associated with published scientific papers since 2008 and now holds nearly 50,000 datasets. The platform is supported by an active community of researchers, research organizations, and scientific journals that submit data and support infrastructure development and sustenance.

Cite this blog as Habermann, T. (2022). Journal Connectivity@Dryad: The University of Chicago Press. Front Matter.

Journal Connectivity@Dryad: The University of Chicago Press

Identifying diverse contributions made to research objects is the first step in acknowledging those contributions. DataCite includes the contributorType metadata element and a list of twenty types to support this step. A type of Other is included to allow recognition of contributions not included in the list. Understanding how Other is used might help evaluate possible extensions to the contributorType list.

::::: {#block-4ce0cb5334f301eba820 .sqs-block .html-block .sqs-block-html block-type=“2” border-radii=“{"topLeft":{"unit":"px","value":0.0},"topRight":{"unit":"px","value":0.0},"bottomLeft":{"unit":"px","value":0.0},"bottomRight":{"unit":"px","value":0.0}}”}  Connecting research is a critical element of the DataCite vision and two kinds of connections are supported by DataCite metadata.

The Others

Adopting organizational identifiers (RORs) in existing metadata may not be as hard as we think for DataCite members that only have a few affiliations in their metadata. However, even in these cases, some challenges remain.

The scholarly communications and open science communities are getting excited about the benefits of metadata that includes persistent identifiers for organizations. Now that an open registry of organizational identifiers exists, the community is facing two adoption challenges: 1) evolving metadata dialects to include identifiers and 2) implementing them in existing and future metadata.

How Many RORs Do We Need?

Affiliations and identifiers are critical for building a connected research community across the world. The contribution made to that connected resource for University datasets in DataCite is limited by the paucity of the required organizational identifiers.

Cite this blog as Habermann, T. (2022). Universities@DataCite. Front Matter.

Universities@DataCite

Writing out an acronym the first time you use it works well in technical writing. Use the same rule in your affiliations! Is it Metadata Game Changers or Mitsubishi Gas Chemical?

I have been working on finding RORs from affiliation strings again and have been struck by the number of people that provide acronyms as affiliation strings. We all know the rule for using acronyms in scientific writing is “write out the first occurrence completely” so the reader knows what the acronym stands for and most of us try to follow this rule.

Acronyms Are Definitely Not Enough

Can communities around domain repositories be used to increase connectivity for researchers and organizations in those communities? The UNAVCO repository suggests that the answer is yes.

The fundamental hypothesis being explored in this series of blogs is that identifiers of any kind, even if they occur just once, can be useful in domain repository metadata because individuals and organizations make multiple contributions to the repository and the corpus of scientific literature based on it. The goal of the work is to increase the number of identifiers in DataCite metadata associated with datasets created or archived at UNAVCO.

Improving Domain Repository Connectivity: Closing the Circle

Adding identifiers for organizations is an important step in the process of increasing connectivity in metadata collections. Organizations make many contributions to domain repositories so each identifier is used many times. In the UNAVCO case, thirty-five identifiers were used over two thousand times.

The UNAVCO DataCite Repository has over 5000 records that describe datasets created by researchers from many organizations, all of which are members of the tight-knit and well-established UNAVCO community. In the first blog of this series, I proposed that connecting these organizations to the PID Graph depends on having unique identifiers, i.e., RORs, for these organizations.

Improving Domain Repository Connectivity: Identifying Organizations

It has been ten years since the release of the first DataCite metadata schema. Many improvements since then provide a roadmap for community evolution toward making DataCite metadata more FAIR.

Metadata schema evolution reflects the progression of needs, ideas, and practices of the community that creates and uses the metadata. Version 2.0 of the DataCite Metadata Schema was released ten years ago. During this time, the schema has evolved considerably, adding capabilities and supporting many new use cases.

DataCite Metadata: Evolving to FAIRness

Looking for New Year’s metadata resolutions? How about:

 Stop using sentences that include the words “minimum metadata” without specifying a use case.

Sentences that include the words “minimum metadata” come up frequently in metadata discussions, usually in the context of what a data provider wants to provide or, even more common, in the context of what should be expected of them.

Minimum Metadata

Many groups are interested in augmenting their affiliation metadata with RORs and it seems that a list of organization names is the right input to that process... Alas, we know that affiliations are a swamp filled with gotcha's. It turns out that an unexpected (really?) number of organization names resolve to multiple RORs. Actually, almost 800 of them!

A quick post about non-unique ROR Organization names. Many groups are interested in augmenting their affiliation metadata with RORs and it seems that a list of organization names is the right input to that process... Alas, we know that affiliations are a swamp filled with gotcha's. It turns out that an unexpected (really?) number of organization names resolve to multiple RORs. Actually, almost 800 of them!

Sometimes a Name is Not Enough

Clear and specific community recommendations for FAIR metadata in multiple dialects are rare and necessary for measuring progress towards evaluating and improving metadata in many repositories. We propose an initial set of concepts and an evaluation of the DataCite metadata collection for essential metadata elements that support Findability.

This post was first published on the


 DataCite blog


 on September 27, 2019.

Since Wilkenson et al., 2016 introduced the FAIR Principles, discussions and implementation guidelines have been published in almost every possible context.

MetaDIG recommendations for FAIR DataCite metadata

Join the community playing The Metadata Game!

Last summer John Chodaki challenged me to develop something fun for a Metadata2020 presentation during the FORCE18 meeting in Montreal. I have always enjoyed games, so a collaborative game that could engage teams in the metadata creation process seemed like a natural candidate. There were many metadata geeks and lots of ideas about what such a game would look like.

The Metadata Game

Comparison of content from the global research infrastructure retrieved from CHORUS and the NSF Public Access Repository indicates that using current infrastructure effectively could significantly increase PAR content and, subsequently, our understanding of the impact of NSF funding across many domains.

Ted Habermann, Metadata Game Changers  Jamaica Jones, University of Pittsburgh  Cite this blog as Habermann, T. and Jones, J. (2024). The Global Research Infrastructure and the NSF Public Access Repository. Front Matter. https://doi.org/10.59350/s2mgr-mwt45  Understanding the impact of research funded by U.S. Federal agencies and other funders is an important, on-going task.

The Global Research Infrastructure and the NSF Public Access Repository

Instruments can now be identified and connected to other resources using the DataCite metadata schema. Instruments are behind every set of scientific observations so this is a big step towards improved science.

Ted Habermann, Metadata Game Changers   Cite this blog as: Habermann, T. (2024). Instrument Identifiers = Better Science. Front Matter. https://doi.org/10.59350/ehsbf-gwe11  Measuring ocean temperatures accurately is a critical step towards understanding heat in the oceans.

Instrument Identifiers = Better Science

How can the global research infrastructure increase understanding of the myriad contributions made to global knowledge by U.S. Federal agencies? How can we use this infrastructure to increase understanding of connections across the U.S. and global research landscape? How can this infrastructure be used to increase completeness, consistency, and connectivity within agency repositories and search tools?

::::: {#block-f7d3b3ee807c0c6d4345 .sqs-block .html-block .sqs-block-html block-type=“2” border-radii=“{"topLeft":{"unit":"px","value":0.0},"topRight":{"unit":"px","value":0.0},"bottomLeft":{"unit":"px","value":0.0},"bottomRight":{"unit":"px","value":0.0}}”}  Ted Habermann, Metadata Game Changers  Jamaica Jones, University of Pittsburgh  Tara Packer and Howard Ratner.

INFORMATE: Metadata Game Changers and CHORUS Collaborate to Make the Invisible Visible

DataCite includes many types of resources and the distribution of those has evolved considerably over the last twenty years. During 2023, 75% of the new resources  are physicalObjects.

Cite this blog as Habermann, T. (2023). How Many When (Update). Front Matter.

How Many When (Update)

The FAIR Island Project has recently been exploring how the global research infrastructure can be used to connect all kinds of research resources related to scientifically significant places back to the place that enabled the work (i.e. the field station). We recently explored connecting data management plans, protocols, and datasets to projects at field stations in French Polynesia.

Cite this blog as Robinson, E. and Habermann, T. (2023). FAIR Island Experiments with Connecting Project Resources in DataCite. Front Matter.

FAIR Island Experiments with Connecting Project Resources in DataCite

Nearly 900 RORs created during the last several years have been dropped from the ROR database and no longer resolve. These missing RORs have been identified. What should be done with them?

Cite this blog as Habermann, T. (2022). ROR Identifiers That Have Disappeared. Front Matter.

ROR Identifiers That Have Disappeared

::::: {#block-0510d12464991b845e5f .sqs-block .html-block .sqs-block-html block-type=“2” border-radii=“{"topLeft":{"unit":"px","value":0.0},"topRight":{"unit":"px","value":0.0},"bottomLeft":{"unit":"px","value":0.0},"bottomRight":{"unit":"px","value":0.0}}”}  It is hard to believe that we are closing in on the last few days of 2023.

2023 Recap

I use the database version of the ROR data to explore data characteristics that might affect my results, usually by making it harder to find the correct RORs in a big pile of affiliations.

Cite this blog as Habermann, T. (2022). ROR in a Database. Front Matter.

ROR in a Database

The idea that DataCite exists only to provide DOIs is deeply embedded in repository thought processes and this idea needs to evolve. The research community needs to think about DataCite (and other elements of the global research infrastructure) as powerful resources for describing and connecting the myriad of resources that make up the modern research world.

Cite this blog as Habermann, T. (2022). Metadata Life Cycle: Mountain or Superhighway? Front Matter.

Metadata Life Cycle: Mountain or Superhighway?

Improving the completeness and the machine-readability of funder metadata in the global research infrastructure, i.e. DataCite and Crossref, is a critical step along the path of using that infrastructure to identify and characterize research results supported by funders all over the world. A set of 854 funder descriptions from the DRUM repository were processed into 1482 affiliation strings.

Research activities depend on monetary and other support from a wide variety of state and federal agencies, universities, foundations, and individuals.

Funder Metadata: Identifiers and Award Numbers

Can metadata from journal articles be used to augment dataset metadata?

In previous blogs we used DataCite metadata for UNAVCO to demonstrate how identifiers could be found and spread through the metadata collection to improve connectivity for people and organizations. The community built around UNAVCO over time was a critical part of this process as community members, both individuals and organizations, make many contributions over time.

Improving Domain Repository Connectivity: Article Metadata Archeology

Metadata metrics over the last five years demonstrate that communities and repositories can work together to improve metadata.

With Chris Beltz, S. Jeanette Clark, Peter Slaughter and Matt Jones from the Arctic Data Center (


 https://ror.org/055hrh286


 ) and NCEAS (


 https://ror.org/0146z4r19


 ).

I have written several blogs about measuring metadata with goals of identifying good examples and quantifying improvements in metadata over time.

Can Communities Improve Metadata?

The road to complete and consistent metadata can be long and arduous – digging through piles of metadata and other kinds of data to find small gems of information that can be added to metadata records, contacting recalcitrant researchers to fill in blanks, slowly building content across a collection… Does it really need to be that hard?

Can Repositories Improve Metadata?

The University of Bath Research Data Archive has added ORCIDs and RORs to their metadata. They were already a brightSpot among Universities at DataCite. Now they are even brighter!

Ted Habermann, Metadata Game Changers   Cite this blog as Habermann, T. (2024). BrightSpots Get Brighter. Front Matter. https://doi.org/10.59350/prqvd-2f082. Introduction   During June we discussed results of FAIR metadata assessments for 387 DataCite repositories associated with universities and colleges.

BrightSpots Get Brighter

Blog - Metadata Game Changers

::: {#item-5c61d6fb7817f7b585cf720e .sqs-layout .sqs-grid-12 .columns-12 layout-label="Post Body" data-type="item" updated-on="1549915960351"}
::: {.row .sqs-row}
::: {.col .sqs-col-12 .span-12}
::: {#block-e5946b87978cc1fd9d12 .sqs-block .html-block .sqs-block-html block-type="2"}
::: {.sqs-block-content}
::: {.sqs-html-content}
[[California Digital
Library]{style="text-decoration:underline"}](https://www.cdlib.org/)
(CDL) and [[Dryad
]{style="text-decoration:underline"}](https://datadryad.org/)recently
[[announced a
partnership]{style="text-decoration:underline"}](https://www.cdlib.org/cdlinfo/2018/05/30/letter-to-the-community-cdl-and-dryad-partnership/)
to address researcher needs and to "move the needle" on digital data
adoption throughout the scholarly research community by [[working
together]{style="text-decoration:underline"}](https://uc3.cdlib.org/2018/12/18/we-cant-succeed-alone/)
to understand and respond to researcher needs for [[high-quality
scientific data publishing
infrastructures]{style="text-decoration:underline"}](https://uc3.cdlib.org/2018/10/24/community-owned-data-publishing-infrastructure/).
Metadata that supports accessibility to and understanding of published
datasets plays a critical role in driving this adoption.

I have been working with CDL's University of California Curation Center
([[UC3]{style="text-decoration:underline"}](https://uc3.cdlib.org/)) on
[[Metadata
2020]{style="text-decoration:underline"}](http://www.metadata2020.org/)
and this new partnership provides a great opportunity to apply some of
the ideas developed
[[there]{style="text-decoration:underline"}](http://www.metadata2020.org/projects/)
to a challenging real-world metadata evaluation, guidance and
improvement problem. This initial post explores **current Dryad metadata
models** and usage to lay a foundation for understanding these metadata
and forming data-driven recommendations for improvements in the future.

### Dryad Metadata Model {#dryad-metadata-model style="white-space:pre-wrap;"}

The [[Dryad Metadata
Model]{style="text-decoration:underline"}](http://wiki.datadryad.org/Metadata_Profile)
describes two types of objects: dataFiles, and dataPackages that contain
them. This metadata hierarchy is common in other metadata models as
well, sometimes called granule and collection metadata. All of these
models include terms that are shared by both types of objects and others
that are unique to one or the other.

Curated examples of how Dryad metadata are used are available on the
[[Dryad
Wiki]{style="text-decoration:underline"}](http://wiki.datadryad.org/Sample_Dryad_Content).
Existing Dryad metadata provides a rich source for understanding these
metadata and how they are used in real-life. In order to understand the
usage, we examined a random sample of metadata for 114 dataPackages and
335 dataFiles from Dryad. Hopefully this sample is large enough to
identify similarities and differences.

The Dryad metadata terms naturally divide into four groups shown in the
Table below along with the % of the records of each type (dataPackage =
blue, dataFile = orange) that include each of the terms on the Y axis.
For example, the first group (Data Files) includes four terms (isPartOf,
provenance, rights, and format) listed on the X axis. Three of these are
required (marked by \*) and two are repeatable (marked by +). The term
isPartOf occurs in 100% of the dataFile metadata records (shown as
orange) and 0% of the dataPackage records (shown as blue). This usage is
expected, as this term gives the dataPackage that a dataFile is part of.
:::
:::
:::

::: {#block-yui_3_17_2_1_1549914831074_91513 .sqs-block .gallery-block .sqs-block-gallery block-json="{\"existingGallery\":null,\"hSize\":null,\"newWindow\":false,\"floatDir\":null,\"methodOption\":\"transient\",\"aspect-ratio\":\"square\",\"aspectRatio\":null,\"auto-crop\":true,\"blockAnimation\":\"none\",\"collectionId\":\"5c61da7fb208fc015e56439c\",\"design\":\"grid\",\"lightbox\":true,\"lightboxTheme\":\"light\",\"padding\":20,\"show-meta\":true,\"show-meta-basic\":true,\"show-meta-only-title\":false,\"show-meta-only-description\":false,\"square-thumbs\":false,\"thumbnails-per-row\":4,\"vSize\":null,\"transientGalleryId\":\"5c61da7fb208fc015e56439c\"}" block-type="8"}
::: {.sqs-block-content}
::: {.sqs-gallery-container .sqs-gallery-block-grid .sqs-gallery-aspect-ratio-square .sqs-gallery-thumbnails-per-row-4 .sqs-gallery-block-show-meta .block-animation-none .clear}
::: {.sqs-gallery}
::: {.slide data-type="image" animation-role="image"}
::: {.margin-wrapper}
[View fullsize]{.v6-visually-hidden}

![Data
Files](https://images.squarespace-cdn.com/content/v1/52ffa419e4b05b374032e6d9/1549916799465-ZY5826UGSJ2H06D2H9UA/DataFiles.png)
:::
:::

::: {.slide data-type="image" animation-role="image"}
::: {.margin-wrapper}
[View fullsize]{.v6-visually-hidden}

![Shared
Terms](https://images.squarespace-cdn.com/content/v1/52ffa419e4b05b374032e6d9/1549916885830-O2PSDIRUGFCFOFX6JEK5/Shared.png)
:::
:::

::: {.slide data-type="image" animation-role="image"}
::: {.margin-wrapper}
[View fullsize]{.v6-visually-hidden}

![Mostly Data
Packages](https://images.squarespace-cdn.com/content/v1/52ffa419e4b05b374032e6d9/1549916902653-F016OO0RFEXOJRRLHZYZ/MostlyDataPackages.png)
:::
:::

::: {.slide data-type="image" animation-role="image"}
::: {.margin-wrapper}
[View fullsize]{.v6-visually-hidden}

![All Files and
Packages](https://images.squarespace-cdn.com/content/v1/52ffa419e4b05b374032e6d9/1549916942422-YU2D1RXBOILTOTGACEMK/AllFilesAndPackages.png)
:::
:::
:::
:::
:::
:::

::: {#block-yui_3_17_2_1_1549914831074_115172 .sqs-block .html-block .sqs-block-html block-type="2"}
::: {.sqs-block-content}
::: {.sqs-html-content}
**DataFile terms**: Four terms that pertain only to dataFiles.

Three of these (isPartOf, provenance, and rights) are required (marked
with \*) and occur in virtually all dataFile records. They contain the
identifier of dataPackage that the dataFile is part of (typically a
DOI), the state of the dataFile in the Dryad workflow, and the license
for the dataPackage (all Creative Commons Zero, CC0).

The format term is optional and occurs in 86% of the dataFile records.
It typically contains the size of the dataFile in bytes.

**Shared terms**: Five optional terms that are used in roughly the same
number of dataPackages and dataFiles to provide information about
several types of keywords (time period, spatial location,
scientificNames, and subject).

The similarity of these numbers reflects the fact that dataPackages
aggregate the keywords from the dataFiles that they contain.

**DataPackage terms**: Five terms, three of which are required, that are
used in essentially all dataPackages and varying numbers of dataFiles to
provide submission history and descriptions and to connect dataPackages
to dataFiles (hasPart) and to published papers (references).

**All Files and Packages**: Four required terms and the XML
schemaLocation that provide basic discovery and access information
(title, creator, identifier) and give the types for all dataPackages and
dataFiles.

Several of these terms can occur more than once in a single metadata
record (marked with + in the Table above). The most numerous terms in
the dataPackage and dataFile metadata records are creators, averaging
\~5 / record. Subject terms come in a close second with an average of
just fewer than 5 / record. Most dataPackages in this sample (54%)
contain a single dataFile and 77% contain less than three dataFiles. The
maximum number of dataFiles in a dataPackage is 72 in this sample.

### Collection Homogeneity {#collection-homogeneity style="white-space:pre-wrap;"}

I proposed several
[[metrics]{style="text-decoration:underline"}](https://blog.datacite.org/metrics-for-metadata/)
for characterizing and comparing metadata collections. One of these
measures the collection homogeneity calculated as the number of terms
that are in all records / the total number of terms in the collection.
The Dryad dataPackage sample contains fourteen terms and nine of them
are included in all records so homogeneity is 65%. The dataFile sample
includes seventeen terms and six are included in all records so
homogeneity is 35%. This difference may reflect the fact that much of
the dataPackage metadata is managed as part of the central repository
workflow while the dataFile metadata are provided by the researchers
that use Dryad to publish data.

### Conclusion {#conclusion style="white-space:pre-wrap;"}

This initial exploration identifies the two types of Dryad metadata,
outlines some interesting similarities and differences between them, and
helps form a baseline for understanding intended metadata use cases. The
metadata were designed to connect elements of information packages that
include scientific papers (identified in the references element) and
related dataFiles. They provide identifiers and discovery information
that includes titles, creators, paper abstracts, and three types of
keywords (subject, scientific name, spatial, and temporal) and track the
data publication process (processing dates and provenance). The
scientific papers are the primary discovery path for the data and are
required for data users to understand details of how the data were
collected and processed. This is different than the model used in many
repositories where these details can be included in the dataset metadata
and users can access them independently.

  
:::
:::
:::
:::
:::
:::
