University of Glasgow

Roderic Page

Following on from the discussion of the [African chameleon
data](http://iphylo.blogspot.com/2014/08/guest-post-response-to-discussion-on.html),
I\'ve started to explore Angelique Hjarding\'s data in more detail. The
data is available from figshare
([doi:10.6084/m9.figshare.1141858](http://dx.doi.org/10.6084/m9.figshare.1141858)),
so I\'ve grabbed a copy and put it in
[github](https://github.com/rdmpage/gbif-chameleon). Several things are
immediately apparent.  
  

1.  There is a lot of ungeoreferenced data. With a little work this
    could be geotagged and hence placed on a map.
2.  There are some errors with the georeferenced data (chameleons in
    Soutb America or off the coast, a locality in Tanzania that is now
    in Ethiopia, etc.).
3.  Rather alarmingly, most of the URLs to GBIF records that Angelique
    gives in the dataset **no longer resolve**.

  
The last point is worrying, and reflects the fact that at present you
can\'t trust GBIF occurrence URLs to be stable over time. Most of the
specimens in Angelique\'s data are probably still in GBIF, but the GBIF
occurrenceID (and hence URL) will have changed. This pretty much kills
any notion of reproducibility, and it will require some fussing to be
able to find the new URLs for these records.  
  
That the GBIF occurrenceIDs are no longer valid also makes it very
difficult to make use of any data cleaning I or anyone else attempts
with this data. If I georeference some of the specimens, I can\'t simply
tell GBIF that I\'ve got improved data. Nor is it obvious how I would
give this information to the original providers using, say [VertNet\'s
github
repositories](http://iphylo.blogspot.com/2014/01/vertnet-starts-issue-tracking-using.html).
All in all a mess, and a sad reflection on our inability to have
persistent identifiers for occurrences.  
  
To help explore the data I\'ve created some GeoJSON files to get a sense
of the distribution of the data. Here are the point localities, a few
have clearly got issues.  
  

  
Taxa represent by less than three distinct localities are presented by
place marker, the rest by polygons.  
  
I\'ll keep playing with this data as time allows, and try to get a sense
of how hard it would be to go from what GBIF provides to what is
actually going to be useful.  


Following on from the discussion of the African chameleon data, I've started to explore Angelique Hjarding's data in more detail. The data is available from figshare (doi:10.6084/m9.figshare.1141858), so I've grabbed a copy and put it in github. Several things are immediately apparent.There is a lot of ungeoreferenced data.

Exploring the chameleon dataset: broken GBIF links and lack of georeferencing

![AHjarding](http://lh5.ggpht.com/-IaI0Pb9t3yU/U_MU3KtZ7rI/AAAAAAAADgY/OCwLyZNK22M/AHjarding.JPG?imgmax=800 "AHjarding.JPG"){border="0"
width="128" height="170" style="float:right;"}This is guest post by
Angelique Hjarding in response to [discussion on this
blog](http://iphylo.blogspot.com/2014/08/seven-percent-of-gbif-data-is-usable.html)
about the paper below.  

> Hjarding, A., Tolley, K. A., & Burgess, N. D. (2014, July 10). Red
> List assessments of East African chameleons: a case study of why we
> need experts. Oryx. Cambridge University Press (CUP).
> [doi:10.1017/s0030605313001427](http://dx.doi.org/10.1017/s0030605313001427){target="_new"}
>
> ::: {.altmetric-embed badge-type="donut" doi="10.1017/s0030605313001427"}
> :::

Thank you for highlighting our recent publication and for the very
interesting comments. We wanted to take the opportunity to address some
of the issues brought up in both your review and from reader comments.  
  
One of the most important issues that has been raised is the sharing of
cleaned and vetted datasets. It has been suggested that the datasets
used in our study be uploaded to a repository that can be cited and
shared. This is possible for data that was downloaded from GBIF as they
have already done the legwork to obtain data sharing agreements with the
contributing organizations. So as long as credit is properly given to
the source of the data, publicly sharing data accessed through GBIF
should be acceptable. At the time the manuscript was submitted for
publication, we were unaware of sites such as <http://figshare.com>
where the data could be stored and shared with no additional cost to the
contributor. The dataset used in the study that used GBIF data has now
been made available in this way.  

> Angelique Hjarding. (2014). Endemic Chameleons of Kenya and Tanzania.
> Figshare.
> [doi:10.6084/m9.figshare.1141858](http://dx.doi.org/10.6084/m9.figshare.1141858){target="_new"}

  
  
It starts to get tricky with doing the same for the expert vetted data.
This dataset consists primarily of data gather by the expert from museum
records and published literature. So in this case it is not a question
of why the expert doesn't share. The question is why the museum data and
any additional literature records are not on GBIF already. As has been
pointed out in our analysis (and confirmed by Rod) most of these museums
do not currently have data sharing agreements with GBIF. Therefore, the
expert who compiled the data does not have the permission of the museums
to share their data second hand. Bottom line, all of the data used in
this study that was not accessed through GBIF is currently available
from the sources directly. That is, for anyone who wants to take the
time contact the museums for permission to use their data for research
and to compile it. We also do not believe there is blame on museums that
have not yet shared their data with forums such as GBIF. Mobilisation of
data is an enormous task, and near impossible if funds and staff are not
available. With regards to the particular comment regarding the lack of
data sharing by NHML and other museums, we need to recognise what the
task at hand would mean, and rather address ways such a monumental, and
valuable, collection could be mobilised. A further issue should be
raised around literature records that are not necessarily encapsulated
in museum collections, but are buried in old and obscure manuscripts. To
our knowledge, there is no way to mobilise those records either, because
they are not attached to a specimen. Further, because there are no
specimens, extreme care must be taken if such records were to be
mobilised in order to ensure quality control. Again, assistance of
expert knowledge would be highly beneficial, yet these things take time
and require funds.  
  
Another issue that was raised is why didn't we go directly to GBIF to
fix the records? The point of our research was not to clean and update
GBIF/museum data but to evaluate the effect of expert vetting and museum
data mobilization in an applied conservation setting. As it has been
pointed out, the lead author was working at GBIF during the course of
the research. An effort was made to provide a checklist of the updated
taxonomy to GBIF at the time, but there was no GBIF mechanism for
providing updates. This appears to still be the case. In addition, two
GBIF staff provided comments on the paper and were acknowledged for
their input. We are happy to provide an updated taxonomy to help improve
the data quality, should some submission tool for updates be made
available.  
  
Finally we would like to address the question, why use GBIF data if we
know it needs some work before it can be used? We believe this is a very
important debate for at least two reasons. First, when data is made
public, we believe there are many researchers who work under the
assumption that the data is ready for use with minimal further work. We
believe they assume that the taxonomy is up to date; that the records
are in the right place; and that the records provided relate to the name
that is attached to those records. Many of the papers that have used
GBIF data have undertaken broad scale macroecological analyses where,
perhaps, the errors we have shown matter little. But some of these
synthetic studies have also proposed that their results can be used for
decision making by companies, which starts to raise concerns especially
if the company wants to know the exact species that its activities could
impact. As we have shown, for chameleons at least, such advice would be
hard to provide using the raw GBIF data.  
  
Second, we are aware that there is another group of researchers using
GBIF data who \"know that to use GBIF\'s data you need to do a certain
amount of previous work and run some tests, and if the data does not
pass the tests, you don\'t use it.\" We are not sure of the tests that
are run, and it would be useful to have these spelled out for broader
debate and potentially the development of some agreed protocols for data
cleaning for various uses.  
  
Our underlying reason for writing the paper was not to enter into debate
of which data are best between GBIF and an expert compiled dataset. We
are extremely pleased that GBIF data exist, and are freely available for
the use of all. This certainly has to be part of the future of \'better
data for better decisions\', but we are concerned that we should not
just accept that the data is the best we can get, but should instead
look for ways to improve it, for all kinds of purposes. As such, we
would like to suggest that the discussion focuses some energy on ways to
address the shortcomings of the present system, but also that the
community who would benefit from the data address ways to assist the
dataholders to mobilise their information in terms of accessing the
resources required to digitise and make data available, and maintain
updated taxonomy for their holdings. In an era of declining funding for
Museum-based taxonomy in many parts of the world this is certainly a
challenge that needs to be addressed.  
  
We welcome further discussion as this is a very important topic, not
only for conservation but also in terms of improved access to
biodiversity knowledge, which is critical for many reasons.  
  
Angelique Hjarding
[http://orcid.org/0000-0002-9279-4893](http://orcid.org/0000-0002-9279-4893){target="_new"}  
[Krystal
Tolley](https://www.researchgate.net/profile/Krystal_Tolley){target="_new"}  
Neil Burgess  
  


This is guest post by Angelique Hjarding in response to discussion on this blog about the paper below.Thank you for highlighting our recent publication and for the very interesting comments. We wanted to take the opportunity to address some of the issues brought up in both your review and from reader comments. One of the most important issues that has been raised is the sharing of cleaned and vetted datasets.

Guest post: Response to the discussion on Red List assessments of East African chameleons

::: {style="background-color:#FFFBB1;padding:10px;border:1px solid #666666;"}
**Update:** Angelique Hjarding and her co-authors have responded in a
[guest post on
iPhylo.](http://iphylo.blogspot.com/2014/08/guest-post-response-to-discussion-on.html)
:::

  
The quality and fitness for use of [GBIF-mobilised
data](http://www.gbif.org) is a topic of interest to anyone that uses
GBIF data. As an example, a recent paper on African chameleons comes to
some rather alarming conclusions concerning the utility of GBIF data:  
  

> Hjarding, A., Tolley, K. A., & Burgess, N. D. (2014, July 10). Red
> List assessments of East African chameleons: a case study of why we
> need experts. Oryx. Cambridge University Press (CUP).
> [doi:10.1017/s0030605313001427](http://dx.doi.org/10.1017/s0030605313001427)

  
Here\'s the abstract (unfortunately the paper is behind a paywall):  
  

> The IUCN Red List of Threatened Species uses geographical distribution
> as a key criterion in assessing the conservation status of species.
> Accurate knowledge of a species' distribution is therefore essential
> to ensure the correct categorization is applied. Here we compare the
> geographical distribution of 35 species of chameleons endemic to East
> Africa, using data from the Global Biodiversity Information Facility
> (GBIF) and data compiled by a taxonomic expert. **Data screening
> showed 99.9%of GBIF records used outdated taxonomy and 20% had no
> locality coordinates**. Conversely the expert dataset used
> 100%up-to-date taxonomy and only seven records (3%) had no
> coordinates. Both datasets were used to generate range maps for each
> species, which were then used in preliminary Red List categorization.
> There was disparity in the categories of 10 species, with eight being
> assigned a lower threat category based on GBIF data compared with
> expert data, and the other two assigned a higher category. Our results
> suggest that before conducting desktop assessments of the threatened
> status of species, aggregated museum locality data should be vetted
> against current taxonomy and localities should be verified. We
> conclude that available online databases are not an adequate
> substitute for taxonomic experts in assessing the threatened status of
> species and that Red List assessments may be compromised unless this
> extra step of verification is carried out.

  
The authors used two data sets, one from GBIF, the other provided by an
expert to compute the conservation status for each chameleon species
endemic to Kenya and/or Tanzania. After screening the GBIF data for
taxonomic and geographic issues, **a mere 7% of the data remained** -
93% of the 2304 records downloaded from GBIF were discarded.  
  
This study raises a number of questions, some of which I will touch on
here. Before doing so, it\'s worth noting that it\'s unfortunate that
neither of the two data sets used in this study (the data downloaded
from GBIF, and the expert data set assembled by Colin Tilbury) are
provided by the authors, so our ability to further explore the results
is limited. This is a pity, especially now that citable data
repositories such as [Dryad](http://datadryad.org) and
[Figshare](http://figshare.com) are available. The value of this paper
would have been enhanced if both datasets were archived.  
  
Below is Table 1 from the paper, \"Museums from which locality records
for East African chameleons were obtained for the expert and GBIF
datasets\":  
  

Museum

Expert dataset

GBIF

Afrika Museum, The Netherlands

x

[American Museum of Natural History,
USA](http://www.gbif.org/occurrence/search?taxon_key=9453&dataset_key=71d0dff0-f762-11e1-a439-00145eb45e9a)

x

[Bishop Museum,
USA](http://www.gbif.org/occurrence/search?taxon_key=9453&dataset_key=b929f23d-290f-4e85-8f17-764c55b3b284)

x

British Museum of Natural History, UK

x

Brussels Museum of Natural Sciences, Belgium

x

[California Academy of Sciences,
USA](http://www.gbif.org/occurrence/search?taxon_key=9453&dataset_key=cece4fc2-1fec-4bb5-a335-7252548e3f0b)

x

[Ditsong Museum, South
Africa](http://www.gbif.org/occurrence/search?taxon_key=9453&dataset_key=d51cd455-6b16-4255-9fcf-b567fe2f09ca)

x

x

[Los Angeles County Museum of Natural History,
USA](http://www.gbif.org/occurrence/search?taxon_key=9453&dataset_key=7a25f7aa-03fb-4322-aaeb-66719e1a9527)

x

[Museum für Naturkunde,
Germany](http://www.gbif.org/publisher/10980920-6dad-11da-ad13-b8a03c50a862)

x

[Museum of Comparative Zoology (Harvard University),
USA](http://www.gbif.org/occurrence/search?taxon_key=9453&dataset_key=4bfac3ea-8763-4f4b-a71a-76a6f5f243d3)

x

Naturhistorisches Museum Wien, Austria

x

Smithsonian Institution, USA

x

[South African Museum, South
Africa](http://www.gbif.org/dataset/7b468209-8d5c-477b-b118-cdf04dde9912)

x

Trento Museum of Natural Sciences, Italy

x

[University of Dar es Salaam,
Tanzania](http://www.gbif.org/dataset/9bf16572-a62e-4a87-80b5-4847ca7ff31f)

x

[Zoological Research Museum Alexander Koenig,
Germany](http://www.gbif.org/dataset/263bbfe7-f643-43bb-b448-f0705d1c0e18)

x

  
  
It is striking that there is virtually no overlap in data sources
available to GBIF and the sources used by the expert. Some of the
museums have no presence in GBIF, including some major collections (I\'m
looking at you, [The Natural History Museum](http://www.nhm.ac.uk)), but
some museums do contribute to GBIF, but not their herpetology specimens.
So, GBIF has some work to do in mobilising more data (Why is this data
not in GBIF? What are the impediments to that happening?). Then there
are museums that have data in GBIF, but not in a form useful for this
study. For example, the American Museum of Natural History has [327,622
herpetology specimens in
GBIF](http://www.gbif.org/dataset/71d0dff0-f762-11e1-a439-00145eb45e9a),
but **not one of these is georeferenced!** Given that there are records
in GenBank for AMNH specimens that are georeferenced, I suspect that the
AMNH collection has deliberately not made geographic coordinates
available, which raises the obvious question - why?  
  

### GBIF coverage

  
I had a quick look at GBIF to get some idea of the geographic coverage
of the relevant herpetology collections (or animal collections if herps
weren\'t separated out). Below are maps for some of these collections.
The AMNH is empty, as is the smaller Zoological Research Museum
Alexander Koenig collection (which supplied some of the expert data).  
  
American Museum of Natural History, USA  

::: {style="position:relative;height:128px;"}
![](http://a.tiles.mapbox.com/v3/timrobertson100.map-x2mlizjd/0/0/0.png){style="position:absolute;top:0px;left:0px;z-index:0;"
width="128"
height="128"}![](http://api.gbif.org/v1/map/density/tile?x=0&y=0&z=0&type=DATASET&key=71d0dff0-f762-11e1-a439-00145eb45e9a&resolution=2){style="background:none;position:absolute;top:0px;left:0px;z-index:10"
width="128" height="128"}
:::

  
Bishop Museum, USA  

::: {style="position:relative;height:128px;"}
![](http://a.tiles.mapbox.com/v3/timrobertson100.map-x2mlizjd/0/0/0.png){style="position:absolute;top:0px;left:0px;z-index:0;"
width="128"
height="128"}![](http://api.gbif.org/v1/map/density/tile?x=0&y=0&z=0&type=DATASET&key=b929f23d-290f-4e85-8f17-764c55b3b284&resolution=2){style="background:none;position:absolute;top:0px;left:0px;z-index:10"
width="128" height="128"}
:::

  
California Academy of Sciences, USA  

::: {style="position:relative;height:128px;"}
![](http://a.tiles.mapbox.com/v3/timrobertson100.map-x2mlizjd/0/0/0.png){style="position:absolute;top:0px;left:0px;z-index:0;"
width="128"
height="128"}![](http://api.gbif.org/v1/map/density/tile?x=0&y=0&z=0&type=DATASET&key=cece4fc2-1fec-4bb5-a335-7252548e3f0b&resolution=2){style="background:none;position:absolute;top:0px;left:0px;z-index:10"
width="128" height="128"}
:::

  
Ditsong Museum, South Africa  

::: {style="position:relative;height:128px;"}
![](http://a.tiles.mapbox.com/v3/timrobertson100.map-x2mlizjd/0/0/0.png){style="position:absolute;top:0px;left:0px;z-index:0;"
width="128"
height="128"}![](http://api.gbif.org/v1/map/density/tile?x=0&y=0&z=0&type=DATASET&key=d51cd455-6b16-4255-9fcf-b567fe2f09ca&resolution=2){style="background:none;position:absolute;top:0px;left:0px;z-index:10"
width="128" height="128"}
:::

  
Los Angeles County Museum of Natural History, USA  

::: {style="position:relative;height:128px;"}
![](http://a.tiles.mapbox.com/v3/timrobertson100.map-x2mlizjd/0/0/0.png){style="position:absolute;top:0px;left:0px;z-index:0;"
width="128"
height="128"}![](http://api.gbif.org/v1/map/density/tile?x=0&y=0&z=0&type=DATASET&key=7a25f7aa-03fb-4322-aaeb-66719e1a9527&resolution=2){style="background:none;position:absolute;top:0px;left:0px;z-index:10"
width="128" height="128"}
:::

  
Museum für Naturkunde, Germany  

::: {style="position:relative;height:128px;"}
![](http://a.tiles.mapbox.com/v3/timrobertson100.map-x2mlizjd/0/0/0.png){style="position:absolute;top:0px;left:0px;z-index:0;"
width="128"
height="128"}![](http://api.gbif.org/v1/map/density/tile?x=0&y=0&z=0&type=PUBLISHER&key=10980920-6dad-11da-ad13-b8a03c50a862&resolution=2){style="background:none;position:absolute;top:0px;left:0px;z-index:10"
width="128" height="128"}
:::

  
Museum of Comparative Zoology (Harvard University), USA  

::: {style="position:relative;height:128px;"}
![](http://a.tiles.mapbox.com/v3/timrobertson100.map-x2mlizjd/0/0/0.png){style="position:absolute;top:0px;left:0px;z-index:0;"
width="128"
height="128"}![](http://api.gbif.org/v1/map/density/tile?x=0&y=0&z=0&type=DATASET&key=4bfac3ea-8763-4f4b-a71a-76a6f5f243d3&resolution=2){style="background:none;position:absolute;top:0px;left:0px;z-index:10"
width="128" height="128"}
:::

  
Smithsonian Institution, USA  

::: {style="position:relative;height:128px;"}
![](http://a.tiles.mapbox.com/v3/timrobertson100.map-x2mlizjd/0/0/0.png){style="position:absolute;top:0px;left:0px;"
width="128"
height="128"}![](http://api.gbif.org/v1/map/density/tile?x=0&y=0&z=0&type=DATASET&key=5df38344-b821-49c2-8174-cf0f29f4df0d&resolution=2){style="background:none;position:absolute;top:0px;left:0px;"
width="128" height="128"}
:::

  
Zoological Research Museum Alexander Koenig, Germany  

::: {style="position:relative;height:128px;"}
![](http://a.tiles.mapbox.com/v3/timrobertson100.map-x2mlizjd/0/0/0.png){style="position:absolute;top:0px;left:0px;"
width="128"
height="128"}![](http://api.gbif.org/v1/map/density/tile?x=0&y=0&z=0&type=DATASET&key=63bbfe7-f643-43bb-b448-f0705d1c0e18&resolution=2){style="background:none;position:absolute;top:0px;left:0px;"
width="128" height="128"}
:::

  
  
Some collections are relevant, such as the California Academy of
Sciences, but a number of the collections in GBIF simply don\'t have
georeferenced data on chameleons. Then there are several museums that
are listed as sources for the expert database and which contribute to
GBIF, but haven\'t digitised their herp collections, or haven\'t made
these available to GBIF.  
  

### Taxonomy

  
The other issue encountered by Hjarding et al. 2014 is that the GBIF
taxonomy for chameleons is out of date (2302 of 2304 GBIF-sourced
records needed to be updated). Chameleons are a fairly small group, and
it\'s not like there are hundreds of new species being discovered each
year (see [timeline in
BioNames](http://bionames.org/timeline/Animalia/Chordata/Vertebrata/Reptilia/Lepidosauria/Squamata/Sauria/Chamaeleontidae)),
2006 was a bumper year with 12 new taxonomic names added. But there has
been a lot of recent phylogenetic work which has clarified
relationships, and as a result species get shuffled around different
genera, resulting in a plethora of synonyms. GBIF\'s taxonomy has lagged
behind current research, and also manages to horribly mangle the
chameleon taxonomy is does have. For example, the genus
[*Trioceros*](http://www.gbif.org/species/4818567) is not even placed
within the chameleon family Chamaeleonidae but is simply listed as a
reptile, which means anyone searching for data on the family
Chamaeleonidae will all the *Trioceros* species.  
  

### Summary

  
The use case for this study seems one of the most basic that GBIF should
be able to meet - given some distributions of organisms, compute an
assessment of their conservation status. That GBIF-mobilised data is so
patently not up to the task in this case is cause for concern.  
  
However, I don\'t see this is simply a case of expert data set versus
GBIF data, I think it\'s more complicated than that. A big issue here is
data availability, and also the extent of data release (assuming that
the AMNH is actively withholding geographic coordinates for some, if not
most of its specimens). GBIF should be asking those museums that provide
data why they\'ve not made georeferenced data available, and if its
because the museums simply haven\'t been able to do this, then how can
it help this process? It should also be asking why museums which are
part of GBIF haven\'t mobilised their herpetology data, and again, what
can it do to help? Lastly, in an age of rapid taxonomic change driven by
phylogenetic analysis, GBIF needs to overhaul the glacial pace at which
it incorporates new taxonomic information.


<b>
 Update:
</b>
Angelique Hjarding and her co-authors have responded in a guest post on iPhylo.The quality and fitness for use of GBIF-mobilised data is a topic of interest to anyone that uses GBIF data.

Publicaciones de Rogue Scholar

Exploring the chameleon dataset: broken GBIF links and lack of georeferencing

Guest post: Response to the discussion on Red List assessments of East African chameleons

Seven percent of GBIF data is usable - quick thoughts on Hjarding et al. 2014