Cameron Neylon

On Monday I am speaking as part of a meeting on [Use Cases for
Provenance](http://www.nesc.ac.uk/esi/events/983/)
([Programme](http://wiki.esi.ac.uk/UseCasesForProvenanceWorkshop)),
which has a lot of interesting talks scheduled. I appear to be last. I
am not sure whether that means I am the comedy closer or the pre-dinner
entertainment. This may, however, be as a result of the title I chose:

> *In your worst nightmares: How experimental scientists are doing
> provenance for themselves*

> On the whole experimental scientists, particularly those working in
> traditional, small research groups, have little knowledge of, or
> interest in, the issues surrounding provenance and data curation.
> There is however an emerging and evolving community of practice
> developing the use of the tools and social conventions related to the
> broad set of web based resources that can be characterised as "Web
> 2.0". This approach emphasises social, rather than technical, means of
> enforcing citation and attribution practice, as well as maintaining
> provenance. I will give examples of how this approach has been
> applied, and discuss the emerging social conventions of this community
> from the perspective of an insider.

The meeting will be webcast (link should be available from
[here](http://www.nesc.ac.uk/esi/events/983/)) and my slides will with
any luck be up at least a few minutes before my talk [in the usual
place](http://www.slideshare.net/cameronneylon).


On Monday I am speaking as part of a meeting on Use Cases for Provenance (Programme), which has a lot of interesting talks scheduled. I appear to be last. I am not sure whether that means I am the comedy closer or the pre-dinner entertainment.

Use Cases for Provenance – eScience Institute – 20 April

I've been mulling over this for a while, and seeing as I am home sick
(can't you tell from the rush of posts?) I'm going to give it a go. This
definitely comes with a health warning as it goes way beyond what I know
much about at any technical level. This is therefore handwaving of the
highest order. But I haven't come across anyone else floating the same
ideas so I will have a shot at explaning my thoughts.

The Semantic Web, RDF, and XML are all the product of computer
scientists thinking about computers and information. You can tell this
because they deal with straightforward declarations that are absolute. X
has property Y. Putting aside all the issues with the availability of
tools and applications, the fact that triple stores don't scale well,
regardless of all the technical problems a central issue with applying
these types of strategy to the real world is that absolutes don't exist.
I may assert that X has property Y, but what hppens when I change my
mind, or when I realise I made a mistake, or when I find out that the
underlying data wasn't taken properly. How do we get this to work in the
real world?

I am going to try and frame this around RDF but I think the issues are
fairly generalisable. The concept of the triple is a natural one that
fits with out understading of language and is comprehensible to most
people. Within a defined system it is possible to make simple
assertions. However in the real world it becomes obvious that authority
is an issue. Who are you to make an assertion?

**The triple to the quad**

I think the following is reasonably well accepted and uncontentious.
That is; provenance matters. Any assertion is of limited value if it
isn't clear who who has made that assertion. The concept of the quad
(subject predicate object: provenance) I think is reasonably well
established. I first saw it in a t[alk given by Paul
Miller](http://feeds.feedburner.com/~r/PetermrsBlog/~3/167964990/) at
the Unilever Centre in Cambridge (via Peter Murray-Rusts's blog). If
this is to work we need absolute control over identity and respected and
useable clearing houses for identity mamangement. It is to be hoped that
a combination of foaf and openid will make some of this possible. We
already have good systems for rating the value of different people's
contributions. Expanding that is reasonably clear.

**The quad to the quint: confidence levels  
**

Provenance isn't the killer though and the issues associated with it are
being worked through. The big problem is that you cannot make absolute
assertions. I am very confident that the sun will rise tomorrow, but
there are plausible, if hopefully very unlikely, circumstances through
which it might not. Closer to home I can't think of any set of
experimental data that has ever been entirely consistent. The skill in
science lies in figuring out why the inconsistencies are there. More
generally it is fundamental to scientific practice that any assertion is
only provisional.

We therefore require a way of providing confidence levels on assertions.
This doesn't fit will within an RDF framework, as I understand it, as
you are applying an operator to the predicate. It is straightforward to
make assertions about your confidence in an object or subject, to make a
comment about a node on the graph (via a predicate). However what we
need here is to apply a weight to an edge. The visual analogy is a good
one. We have gone from a graph with edges to a graph in which the
various edges (predicates) connecting objects (nodes) have different
strengths. Indeed if we implement the provenance ideas have a set of
graphs with perhaps very different weights depending on who we do, or
don't, trust.

None of this is fatal, although it complicates things a great deal. In
principle (again based on my somewhat vague understanding) there is a
whole branch of mathematics based on Bayesian approaches that deals with
incomplete and even contradictory data by building the most highly
likely structure and placing bounds on the confidence or probability of
specific states within a general model. Indeed this actually mirrors the
practise of science more closely than the notion of perfectly logical
machine reasoning on a perfect abstracted data set. We muddle through a
set of messy data with the aim of trying to come up with the best model
that fits the most data (conflicting protein-protein association data
sets anyone?). Using a Bayeisan approach to generate synthetic
assertions with precise confidence intervals would be hugh step forward.
I shudder to imagine the computational costs but hey, that's someone
else's problem.

**The quint to the sext and on: what does my assertion rely on, and is
it out of date?**

Another aspect of how we practice science isn't captured well by the
triple. What is the basis on which I am making an assertion? What is the
underlying data? Again this is a characteristic of the predicate rather
than an object so is not straightforward to capture. If we imagine our
perfectly honed scientific machine (with added probabilistic reasoning)
then we would want it automatically to reassess assertions, either
confidence or correctness, if the body of data these rely on are found
to be flawed in some way (e.g. if the confidence level drops because a
trusted authority disagrees).

It may be possible to capture the reasoning behind an assertion by
creating a synthetic predicate. The properties of X, Y, and Z, when
procesed this way give me answer A, which justifies my assertion. But in
the real world you can only do this after the event, when you've
winnowed out the irrelevant or misleading data. The natural language
statement is X has relationship A with Y *because* this set of other
data suggests it. This may in fact be very difficult to capture in any
reasonably practical framework. Tripes are good at capturing
relationships between simple objects and simple concepts. Whether they
can cope with the somewhat wooly objects and concepts that make up
scientific models is another question.

If it could be achieved this would be tremendously exciting because it
would provide a much more objective way of weighing different types of
data, that might be contradictory together. It can never be completely
objective, because at the end of the day there is an element of trust in
the quality of the raw data and its description, but it has the
potential to be *rigorous* and *transparent*; two things that don't
often feature in clashes over scientific models. If automated reasoning
fails when it hits a contradiction it will be close to useless in the
real world. Being able to circumvent these problems within the system
will be much better than *ad hoc* discarding of specific contradictory
elements within a data set allowing a balancing to take place of
different pieces of data.

There is another simple but critical aspect that we need to incorporate
into our thinking, and that is a date stamp. A common occurrence in
science is that something which is out of date or misleading makes it
into a review, and then into a textbook, and from there becomes received
wisdom. This is simply because secondary and tertiary literature is
always 2-10 years out of date. If we can connect the literature up,
ideally ultimately back down to the primary data then it is possible to
imagine a future text book system in which the correction or updating of
a concept in the primary data sweeps through updating the textbook
(possibly even the lectures might one hope?) on the fly. These no doubt
already exist, but are the being effectively used?  
**Conclusions**

None of this should be read as suggesting that someone ought to
implement this. It is more a thought experiment. How do we make the data
structure more closely relate to what we really do in practice? Out
there in the wet lab world we rarely have anything as absolute as a
sequence or a simple ID number. We have messy samples and messy concepts
out of which we try to abstract something a little simpler and hopefully
a little bit more beautiful. How might we connect the way we work and
think into these systems that have the potential to make our life a lot
easier?

But the concept of probabilistic or weighted assertions, and in Part II
probabilistic ontologies and vocabularies, is something that I think
bears thinking about. My life doesn't fit inside a triple store, at
least not one that doesn't contain multiple direct contradictions. How
can we go about turning that into something useful?


I’ve been mulling over this for a while, and seeing as I am home sick (can’t you tell from the rush of posts?) I’m going to give it a go. This definitely comes with a health warning as it goes way beyond what I know much about at any technical level. This is therefore handwaving of the highest order. But I haven’t come across anyone else floating the same ideas so I will have a shot at explaning my thoughts.

Messaggi di Rogue Scholar

Use Cases for Provenance – eScience Institute – 20 April

Semantics in the real world? Part I – Why the triple needs to be a quint (or a sext, or…)