University of Glasgow

Roderic Page

While working with linked data and ways to explore and visualise
information, I keep coming back to the [Haystack
project](https://en.wikipedia.org/wiki/Haystack_(MIT_project)), which is
now over a decade old. Among the tools developed was the Haystack
application, which enabled a user to explore all sorts of structured
data. Below is a screen shot of Haystack showing a sequence for [Homo
sapiens cyclin T1 (CCNT1), transcript variant a,
mRNA](https://www.ncbi.nlm.nih.gov/nuccore/NM_001240.4/). Note the use
of a LSID to identify the sequence (LSIDs were actively being used to
identify bioinformatics resources)
**urn:lid:ncbi.nlm.nih.gov.lsid.i3c.org:genbank:nm_001240**.  
  

::: {.separator style="clear: both; text-align: center;"}
[![](https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEg-Xexcu9cE_ywW4s-BggThXoJ6rgwm7JMWsZvZL-DFCEji-zLn5ln4rlyl6NSYb5elCMSFYh5qQ6eoZPyILOSrLHBWH1R0HKFq3MGywct4NdTFKvBl7KxyDbKWOrqaGaMFXkZC/s400/3-Figure1-1.png){border="0"
original-height="896" original-width="1364" height="262"
width="400"}](https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEg-Xexcu9cE_ywW4s-BggThXoJ6rgwm7JMWsZvZL-DFCEji-zLn5ln4rlyl6NSYb5elCMSFYh5qQ6eoZPyILOSrLHBWH1R0HKFq3MGywct4NdTFKvBl7KxyDbKWOrqaGaMFXkZC/s1600/3-Figure1-1.png){imageanchor="1"
style="margin-left: 1em; margin-right: 1em;"}
:::

  
  
For some background on the Haystack project see **How to Make a Semantic
Web Browser**
[DOI:10.1145/988672.988707](https://doi.org/10.1145/988672.988707)
([PDF](http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.4.5356&rep=rep1&type=pdf))
and **Haystack: A Customizable General-Purpose Information Management
Tool for End Users of Semistructured Data**
[PDF](http://www-db.cs.wisc.edu/cidr/cidr2005/papers/P02.pdf).  
One reason I keep coming back to the Haystack project is the notion of
having a **personal** space for exploring linked data. One of the
challenges of having a large knowledge graph is that it becomes hard to
have \"local\" queries. That is, queries which are restricted to a
subset of things that you care about.  
  
For example, while playing around with Ozymandias I keep coming across
interesting species, such as *Milyeringa justitia* (see [FIGURE 5 in A
new species of the blind cave gudgeon Milyeringa (Pisces: Gobioidei,
Eleotridae) from Barrow Island, Western Australia, with a redescription
of M. veritas
Whitley](https://ozymandias-demo.herokuapp.com/?uri=https://doi.org/10.5281/zenodo.220763)).  
  

::: {.separator style="clear: both; text-align: center;"}
[![](https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj5oczE7jd1tMz1eW3kCqnofnVLJGftiwbFObhl0cYElkqOUjwCqrSvMo49MhY3L5XA4tm3sx6Tb7SlpN9TgOOrFsJqL_O-BwTxrjZHO4YTFm6UzmNtkPesep9o476JZptoKocB/s400/figure.png){border="0"
original-height="461" original-width="1055" height="173"
width="400"}](https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj5oczE7jd1tMz1eW3kCqnofnVLJGftiwbFObhl0cYElkqOUjwCqrSvMo49MhY3L5XA4tm3sx6Tb7SlpN9TgOOrFsJqL_O-BwTxrjZHO4YTFm6UzmNtkPesep9o476JZptoKocB/s1600/figure.png){imageanchor="1"
style="margin-left: 1em; margin-right: 1em;"}
:::

  
If I want to explore this taxon in more detail I\'d like to have the
original description, any relevant DNA sequences (e.g.,
[MG543430](https://www.ncbi.nlm.nih.gov/nuccore/MG543430.1)), any papers
publishing those sequences (e.g., [Multiple molecular markers reinforce
the systematic framework of unique Australian cave fishes
(Milyeringa : Gobioidei)](https://doi.org/10.1071/ZO18008)), and
phylogenetic analyses such as the paper [The First Record of a
Trans-Oceanic Sister-Group Relationship between Obligate Vertebrate
Troglobites](https://doi.org/10.1371/journal.pone.0044083) which
establishes a link between *Milyeringa* and a genus of cave fish endemic
to Madagascar (*Typhleotris*).  
  
What I\'d like to be able to do is collect all these sources (ideally by
simply bookmarking the links), saving them as a \"collection\", then at
some point exploring what the knowledge graph can tell me. The
importance of having a collection is so that I can tell the knowledge
graph that I just want to explore a subset of information. Without a
collection it can be tricky to limit the scope of queries. For example,
given a global knowledge graph such as Wikidata, how would you query
just species found in Australia? You would typically rely on the species
having either a property (\"found in Australia\"), or perhaps an
identifier that is only used for Australian species. Neither of these is
particularly satisfactory, especially if there isn\'t a property that
fortuitously matches the scope or your inquiry.  
Hence, I\'m interested in having collections: lists of entities that I
want to know more about. I need ways to create these collections, ways
to describe them, and ways to explore them. In some ways the collections
feature of EOL was close to what I\'m after. In the previous version of
EOL you could \"collect\" taxa that you were interested in (for example,
species that were blue) (see [I think I now \"get\" the Encylopedia of
Life](https://iphylo.blogspot.com/2011/09/i-think-i-now-encylopedia-of-life.html)).
Sadly, collections (along with JSON-LD export and stable image URLs)
have vanished from the new EOL (which seems to be in a death spiral
driven by some really unfortunate decisions). And collections need to be
able to contain any entity, not just taxa.  
  
One way to represent collections in the linked data world is using RSS
feeds, or their schema.org descendant, the
[DataFeed](https://schema.org/DataFeed) (see also Google\'s [Data Feed
Validation Tool](https://actions.google.com/tools/feed-validator/u/0/)).
So, we could collect a series of things we are interested in, create the
corresponding DataFeed, import that into our Knowledge Graph and that
would give us a way to scope our queries (using membership of the
DataFeed to select the species, papers, sequences, etc. that we are
interested in). As an aside, there\'s also some overlap with another MIT
project of old, David Huynh\'s
[Parallax](http://people.csail.mit.edu/dfhuynh/projects/parallax/)
project which explored querying on a set of objects, rather than one
object at a time. This is the functionality that a collection gives you
(if you have a query language like SPARQL which can work on sets of
things).  
  
Returning to Haystack, I\'m intrigued by the idea of building a personal
linked data browser. In other worlds, a browser that stores data that is
relevant to projects you are working on (e.g., blind fish) as
collections (data feeds), but can query a global knowledge graph to
augment that information. SPARQL supports federated queries, so this is
eminently doable. The local browser would have its own triple store,
which could be implemented using [Linked Data
Fragments](http://linkeddatafragments.org/).  
  
For now this is just a jumble of poorly articulated ideas, but I think
much of the power of linking data together will be lost until we have
simple tools that enable us to explore the data in ways that are
relevant to what we actually want to know. Haystack gives us one model
of what such a tool could look like.


While working with linked data and ways to explore and visualise information, I keep coming back to the Haystack project, which is now over a decade old. Among the tools developed was the Haystack application, which enabled a user to explore all sorts of structured data. Below is a screen shot of Haystack showing a sequence for Homo sapiens cyclin T1 (CCNT1), transcript variant a, mRNA.

Notes on collections, knowledge graphs, and Semantic Web browsers

![VN Short
Logo](http://lh4.ggpht.com/-CH8bqcF-Whg/UuJKkNRlovI/AAAAAAAADP8/dAyO9A7yUzo/VN_Short_Logo.png?imgmax=800 "VN_Short_Logo.png"){border="0"
width="180" height="187" style="float:right;"}VertNet has announced that
they have implemented [issue tracking using
GitHub](http://blog.vertnet.org/post/74297660841/issue-tracking-available-in-the-vertnet-portal-at).
This is a really interesting development, as figuring out how to capture
and make use of annotations in biodiversity databases is a problem
that\'s attracting a lot of attention. VertNet have decided to use
[GitHub](http://github.com) to handle annotations, but in a way that
hides most of GitHub from users (developers tend to love things like
GitHub, regular folks, not so much, see [The Two Cultures of
Computing](http://pgbovine.net/two-cultures-of-computing.htm)).  
  
The [VertNet
blog](http://blog.vertnet.org/post/74297660841/issue-tracking-available-in-the-vertnet-portal-at)
has a detailed walk through of how it works. I\'ve made some comments on
that blog, but I\'ll repeat them here.  
  
At the moment the VertBet interface doesn\'t show any evidence of issue
tracking (there\'s a link to add an issue, but you can\'t see if there
are any issues). For example, visiting an example [CUMV Amphibian
1766](http://portal.vertnet.org/o/cornell-university-museum-of-vertebrates/cumv-amphibian-collection?id=b056a508-34c5-46ca-9414-1b2e8696409b)
I don\'t see any evidence on that page that there is an issue for this
record (there is an issue, see
<https://github.com/cumv-vertnet/cumv-amph/issues/1>). It think it\'s
important that people see evidence of interaction (that way you might
encourage others to participate). This would also enable people to gauge
how active collection managers are in resolving issues (\"gee, they
fixed this problem in a couple of days, cool\").  
  
Likewise, it would be nice to have a collection-level summary in the
portal. For example, looking at [CUMV Amphibian
1766](http://portal.vertnet.org/o/cornell-university-museum-of-vertebrates/cumv-amphibian-collection?id=b056a508-34c5-46ca-9414-1b2e8696409b)
I\'m not able to click through to a page for CUMV Amphibians (why can\'t
I do this at the moment - there needs to be a way for me to get to the
collection from a record) to see how many issues there are for the whole
collection, and how fast they are being closed.  
  
I think the approach VertNet are using has a lot of potential, although
it sidesteps some of the most compelling features of GitHub, namely
forking and merging code and other documents. I can\'t, for example,
take a record, edit it, and have those edits merged into the data. It\'s
still a fairly passive \"hey, there\'s a problem here\", which means
that the burden is still on curators to fix the issue. This raises the
whole question of what to do with user-supplied edits. There\'s a nice
paper regarding validating user input into Freebase that is relevant
here, see \"Trust, but Verify: Predicting Contribution Quality for
Knowledge Base Construction and Curation\"
(<http://dx.doi.org/10.1145/2556195.2556227> \[not live yet\], [PDF
here](http://www.ipeirotis.com/wp-content/uploads/2014/01/wsdm2014-cqual.pdf)).


VertNet has announced that they have implemented issue tracking using GitHub. This is a really interesting development, as figuring out how to capture and make use of annotations in biodiversity databases is a problem that's attracting a lot of attention.

VertNet starts issue tracking using GitHub

If youâ€™ve been around either myself or Deepak Singh you will almost certainly have heard the Jeff Jonas/Jon Udell soundbite: â€˜Data finds data. Then people find peopleâ€™. The naÃ¯ve analysis of the success of consumer social networks and the weaknesses of science communication has lead to efforts that almost precisely invert the Jonas/Udell concept.

Cameron Neylon

::: {.zemanta-img}
<div>

[![Picture1](http://cameronneylon.net/wp-content/uploads/2010/05/2719657214_949bddd5fd_m.jpg "Picture1"){decoding="async"}](http://www.flickr.com/photos/24801682@N08/2719657214)
:   Image by
    [cameronneylon](http://www.flickr.com/photos/24801682@N08/2719657214)
    via Flickr

</div>
:::

If youâ€™ve been around either myself or [Deepak
Singh](http://mndoci.com) you will almost certainly have heard the [Jeff
Jonas](http://jeffjonas.typepad.com/)/[Jon
Udell](http://blog.jonudell.net) soundbite: â€˜[Data finds data. Then
people find
people](http://blog.jonudell.net/2007/07/02/data-finds-data-then-people-find-people/)â€™.
Jonas is referring to data management frameworks and knowledge discovery
and Udell is referring to the power of integrated data to bring people
together.

At some level Jonasâ€™ vision (see his
chapter\[[pdf](http://jeffjonas.typepad.com/DataFindsDataCreativeCommons.pdf)\]
in [Beautiful Data](http://oreilly.com/catalog/9780596157111/)) is what
the [semantic
web](http://en.wikipedia.org/wiki/Semantic_Web "Semantic Web"){.zem_slink
rel="wikipedia"} ought to enable, the automated discovery of data or
objects based on common patterns or characteristics. Thus far in
practical terms we have signally failed to make this a reality,
particularly for research data and objects.

Udellâ€™s angle (or rather, myÂ interpretationÂ of his overall stance)
is more linked to the social web â€" the discovery of common contexts
through shared data frameworks. These contexts might be social groups,
as in conventional social networks, a particular interest or passion, or
-- in the case of Jonâ€™s championing of the
[iCalendar](http://tools.ietf.org/html/rfc5545) standard -- Â a date and
place as demonstrated by the Â the [elmcity
project](http://blog.jonudell.net/elmcity-project-faq/) supporting
calendar curation and aggregation. Shared context enables the making of
new connection, the creation of new links. But still mainly links
between people.

> Itâ€™s not the scientists who are social; itâ€™s the data â€" [Neil
> Saunders](http://nsaunders.wordpress.com/)

The naÃ¯ve analysis of the success of consumer social networks and the
weaknesses of science communication has lead to efforts that almost
precisely invert the Jonas/Udell concept. In the case of most of these
â€œFacebooks for Scientistsâ€? the idea is that people find people, and
then they connect with data through those people.

My belief is that it is this approach that has led to the almost
complete failure of these networks to gain traction. Services that place
the object Â research at the centre; the reference management and
bookmarking services, to some extent
[Twitter](http://twitter.com "Twitter"){.zem_slink rel="homepage"} and
[Friendfeed](http://friendfeed.com "FriendFeed"){.zem_slink
rel="homepage"}, appear to gain much more real scientific use because
they mediate the interactions that researchers are interested in, those
between themselves and research objects. Friendfeed in particular seems
to support this discovery pattern. Objects of interest are brought into
your stream, which then leads to discovery of the person behind them.Â 
I often use [Citeulike](http://citeulike.org) in this mode. I find a
paper of interest, identify the tags other people have used for it and
the papers that share those tags. If these seems promising, I *then*
might look at the library of the person, but I get to that person
through the shared context of the research object, the paper, and the
tags around that object.

> Data, data everywhere, but not a lot of links â€" [Simon
> Coles](http://www.soton.ac.uk/chemistry/research/coles/coles.html)

A common complaint made of research data is that people donâ€™t make it
available. This is part of the problem but increasingly it is a smaller
part. It is easy enough to put data up that many researchers are doing
so, in supplementary data of journal articles, on personal websites, or
on community or consumer sites. From a linked data perspective we ought
to be having a field day with this, even if it represents only a small
proportion of the total. However little of this data is easily
discoverable and most of it is certainly not linked in any meaningful
way.

A fundamental problem that I feel like Iâ€™ve been banging on about for
years now is that dearth of well built tools for creating these links.
Finally these tools are starting to appear with [Freebase
Gridworks](http://blog.freebase.com/2010/05/10/announcing-the-release-of-freebase-gridworks-1-0/)
being an early example. There is a good chance that it will become
easier over time for people to create links as part of the process of
making their own record. But the fundamental problems we always face,
that this is hard work, and often unrewarded work, are limiting
progress.

> Data friends dataâ€¦then knowledge becomes discoverable

Human interaction is unlikely to work at scale. We are going to need
automated systems to wire the web of data together. The human process
simply cannot keep up with the ongoing annotation and connection of data
at the volumes that are being generated today. And we canâ€™t afford not
to if we want to optimize the opportunities of research to deliver
useful outcomes.

When we think about social networks we always place people at their
centre. But there is nothing to stop us replacing people with data or
other research objects. Software that wants to find data, data that
wants to find complementary or supportive data, or wants to find the
right software to convert or analyze it. Instead of
[Farmville](http://www.farmville.com/) or [Mafia
Wars](http://www.facebook.com/MafiaWars) imagine useful tools that make
these connections, negotiate content, and identify common context. As
pointed out to me by [Paul Walk](http://blog.paulwalk.net/) this is very
similar to what was [envisioned in the 90s as the role of software
agents](http://www.sce.carleton.ca/netmanage/docs/AgentsOverview/ao.html).
In this view the human research users are the poorly connected users on
the outskirts of the web.

The point is that the hard part of creating linked data is making the
links, not publishing the data. The semantic web has always suffered
from the chicken and egg problem of a lack of user-friendly ways to
generate RDF and few tools that could really use that RDF in exciting
ways even if it did exist. I still can't do a useful search on which
restaurants in Bath will be open next Sunday. The reality is that the
innards of this should be hidden from the user, the making of
connections needs to be automated as far as possible, and as natural as
possible when the user has to be involved. As easy as hitting that
"like" button, or right clicking and adding a citation.

We have learnt a lot about the principles of when and how social
networks work. If we can apply those lessons to the construction of open
data management and discovery frameworks then we may stand some chance
of actually making some of the original vision of the web work.

###### Related articles by Zemanta {#related-articles-by-zemanta .zemanta-related-title}

- [Facebook and the Open Graph: good for Linked
  Data?](http://blogs.talis.com/nodalities/2010/05/facebook-and-the-open-graph-good-for-linked-data.php)
  (blogs.talis.com)
- [Freebase Gridworks: A power tool for data scrubbers " Jon
  Udell](http://faganm.com/blog/2010/03/26/1021/) (faganm.com)

::: {.zemanta-pixie}
[![Reblog this post \[with
Zemanta\]](http://img.zemanta.com/reblog_e.png?x-id=37493daa-cc68-4a8d-8b63-ddde92d6ab8f){.zemanta-pixie-img
decoding="async"}](http://reblog.zemanta.com/zemified/37493daa-cc68-4a8d-8b63-ddde92d6ab8f/ "Reblog this post [with Zemanta]"){.zemanta-pixie-a}[]{.zem-script
.more-related .pretty-attribution .paragraph-reblog}
:::


Image by cameronneylon via Flickr  If youâ€™ve been around either myself or Deepak Singh you will almost certainly have heard the Jeff Jonas/Jon Udell soundbite: â€˜Data finds data. Then people find peopleâ€™. Jonas is referring to data management frameworks and knowledge discovery and Udell is referring to the power of integrated data to bring people together.

Why the web of data needs to be social

[Freebase Parallax](http://mqlx.com/~david/parallax/index.html) is a
very cool interface to [Freebase](http://www.freebase.com).  
  

  
[Freebase Parallax: A new way to browse and explore
data](http://www.vimeo.com/1513562?pg=embed&sec=1513562) from [David
Huynh](http://www.vimeo.com/user392740?pg=embed&sec=1513562) on
[Vimeo](http://vimeo.com?pg=embed&sec=1513562).


Freebase Parallax is a very cool interface to Freebase.  Freebase Parallax: A new way to browse and explore data from David Huynh on Vimeo.

Messaggi di Rogue Scholar

Notes on collections, knowledge graphs, and Semantic Web browsers

VertNet starts issue tracking using GitHub

Why the web of data needs to be social

Freebase Parallax