Xuzeng He

Latest findings in multiple research directions for handling graph
prediction and optimization

<figure>
<img src="https://cdn-images-1.medium.com/max/894/0*sb13bRUAq0OUErOQ" />
<figcaption>Source: Generated using DeviantArt’s DreamUp</figcaption>
</figure>

### Author

· Xuzeng He (**ORCID:**
[0009--0005--7317--7426](https://orcid.org/0009-0005-7317-7426))

A graph, in short, is a description of items linked by relations, where
the items of a graph are called **nodes** (or vertices) and their
relations are called **edges** (or links). Examples of graphs can
include social networks (e.g. Instagram) or knowledge graphs (e.g.
Wikipedia). Nowadays, there is a rising trend in the research of using
Machine Learning techniques on graphs to solve various kinds of
problems.

In this post, we introduce two latest findings where the technique of
Machine Learning is applied to Graphs.

### UFKGC

Knowledge graphs (**KGs**) represent information pieces consisting of
entities and their relations, usually organised in the form of
triples ---** head entity**, **relation**, and **tail entity**. For
example, a KG might show that "Chicago Bulls" (head entity) is "located"
(relationship) in "America" (tail entity).

In real-world scenarios, relations in the already-existing KGs generally
suffer from the so-called long-tail problem, meaning that a large number
of relations only occur occasionally in a KG (limited number of sets of
head and tail entities for these relations), which makes the traditional
KG completion methods impractical with these insufficient
training samples.

Most KGs contain many more **rare relations** than **common relations**.

- A common relation, like "located in," might connect countless cities
 and countries.
- Rare relations, on the other hand, might describe very specific
 things, like "known for the invention of the stapler" (person,
 stapler). These relations only connect a handful of entities.

<figure>
<img
src="https://cdn-images-1.medium.com/max/1024/0*-SavSdENTjH-SCVr" />
<figcaption>An example of the few-shot KG completion task. (a) The task
is to predict tail entities on KG. (b) The task utilises only K triples
for the specific relation to predicting others. (Qian Li et al., 2018)
Article Link: <a
href="https://arxiv.org/abs/2403.04521">https://arxiv.org/abs/2403.04521</a></figcaption>
</figure>

In this case, the few-shot KG completion (**FKGC**) task, which predicts
the tail entity from a given head entity and a specific relation as well
as a few triples having the same relation, has gained wide attention.
FKGC aims to predict the missing tail entity in a KG, but with a
different approach: it only uses **a few examples** (called "few-shot")
of the specific relation it's dealing with. These few examples are
provided as a set of triples (head, relation, tail) that share the same
relation you are trying to predict for.

However, uncertainty usually arises when the model is provided with a
limited number of samples. To further enhance the performance of FKGC, a
framework called Uncertainty-aware Few-Shot Knowledge Graph Completion
(**UFKGC**) was proposed recently to model uncertainty for a better
understanding of the limited data and expand the data under the same
distribution.

<figure>
<img
src="https://cdn-images-1.medium.com/max/1024/0*QnFmZChHuJCSpxwz" />
<figcaption>The framework of UFKGC.(Qian Li et al., 2018) Article Link:
<a
href="https://arxiv.org/abs/2403.04521">https://arxiv.org/abs/2403.04521</a></figcaption>
</figure>

The framework of UFKGC contains three different modules, including
Uncertainty Representation, UR-GNN(uncertainty-aware relational graph
neural network), and Uncertainty Optimization.

**Uncertainty Representation** --- *Estimate the uncertainty of
entities/relations in the form of Gaussian Distribution.*

- Consider a knowledge graph statement like "Albert Einstein wrote the
 Theory of Relativity." There might be some uncertainty about other
 works Einstein wrote (less certain) compared to the well-established
 fact that he wrote the Theory of Relativity (more certain).
- This module aims to quantify this uncertainty by representing it as a
 Gaussian Distribution, a mathematical way to describe the probability
 of different values occurring.

**UR-GNN **--- *Get better integration using the learned Gaussian
Distribution*

- This module builds upon the concept of a Graph Neural Network
 (**GNN**), a type of AI model adept at learning from interconnected
 data like a knowledge graph.
- The "uncertainty-aware" part comes in here. The UR-GNN incorporates
 the uncertainty information (Gaussian Distributions) estimated in the
 previous module.
- By considering this uncertainty, the UR-GNN can potentially achieve a
 better understanding of the relationships between entities in the KG,
 even with limited data.

**Uncertainty Optimization **--- *Using multiple sampling from the
learned distributions to evaluate the completion prediction*

- This module leverages the uncertainty information to improve the final
 prediction of the missing entity.
- It utilizes a technique called "multiple sampling." The UR-GNN model,
 informed by the uncertainty, proposes multiple possible tail entities
 (the missing part) for a given head entity and relation, considering
 the probability distribution learned earlier.
- By evaluating these multiple proposals, the Uncertainty Optimization
 module aims to identify the most likely tail entity to complete the
 knowledge graph statement accurately.

Experimental results show that UFKGC achieves state of the art results
in terms of accuracy on two FKGC benchmark datasets, opening exciting
possibilities for future research.

### Bilevel Optimization

Graph machine learning covers a wide range of modelling tasks involving
graph-structured data, where cross-instance/node dependencies are
reflected by edges. Cross-instance or node dependencies refer to the
fact that information about one node can be crucial for understanding
another node, especially if they're connected by an edge. For instance,
in a social network, knowing your friend's interests (one node) might
influence what content you see (another node). Among all the graph
machine learning techniques, Graph Neural Networks (**GNNs**) have
emerged as a promising class of predictive models for handling tasks
such as node classification (assign labels to nodes in the graph) or
link prediction (predict the existence of a link between two entities).

However, the graph machine learning models in the past have primarily
been motivated from diverse perspectives, without necessarily a single,
transparent lens with which to evaluate their commonalities. In this
case, the idea of operating from a unifying vantage point afforded by
bilevel optimization was recently proposed.

Here, bilevel optimization refers to an optimization problem
characterised by two interconnected levels, whereby the optimal solution
of a lower-level objective serves as input features to an upper-level
decision. For example, we can consider the following scenario:

**Upper-level Problem** --- *Teacher*

- A teacher wants to design a quiz that maximises the average score of
 the students in the class. The teacher can choose the difficulty level
 and the number of questions for the quiz.

**Lower level Problem **--- *Student*

- Each student aims to maximise their individual score on the quiz by
 choosing how much time to allocate for studying and which topics to
 focus on.

In this scenario, the study efforts and topic choices made by individual
students can influence their quiz scores, thus impacting the teacher's
objective of maximising the class's average score as input features.

Based on the same idea, a broadly applicable bilevel optimization
framework called **BloomGML** was proposed, which is capable of handling
generic GNN tasks besides node classification. Experimental results show
that BloomGML achieves state-of-the-art performance in terms of accuracy
when given tasks such as node classification, which can provide avenues
for future research.

### Conclusion

The usage of Machine Learning has witnessed significant advancement,
particularly when combined with various kinds of graphs. This can open
multiple exciting possibilities for different research directions. In
this post, we discussed two latest findings for the use of Machine
Learning when combining different graphs. Moreover, we have also shared
some promising scopes that could be used for future exploration of these
approaches. Through continuous investigation and refinement, we believe
that the use of graphs can open up exciting opportunities for us along
with different machine learning techniques.

### References

\[1\] Li, Q., Guo, S., Chen, Y., Ji, C., Sheng, J., & Li, J. (2024).
Uncertainty-Aware Relational Graph Neural Network for Few-Shot Knowledge
Graph Completion.

\[2\] Zheng, A. Y., He, T., Qiu, Y., Wang, M., & Wipf, D. (2024).
BloomGML: Graph Machine Learning through the Lens of Bilevel
Optimization.

![](https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=5a4745e3a3d6){width="1"
height="1"}

Latest findings in multiple research directions for handling graph prediction and optimization Author · Xuzeng He (

 ORCID:

0009–0005–7317–7426) A graph, in short, is a description of items linked by relations, where the items of a graph are called

 nodes

(or vertices) and their relations are called

 edges

(or links). Examples of graphs can include social networks (e.g. Instagram) or knowledge

Recent Advances in using Machine Learning with Graphs

Swinburne University of Technology

Nakul Nambiar

Zhuochen Wu

<figure>
<img
src="https://cdn-images-1.medium.com/max/1024/1*7Fy1vUav7gAiy9CAWanAvQ.png" />
<figcaption>Created using DALLE on 6 Feb 2024</figcaption>
</figure>

#### Authors:

* Nakul Nambiar (ORCID: [0009-0009-9720-9233](https://orcid.org/0009-0009-9720-9233))
* Zhuochen Wu (ORCID: [0009-0000-5642-5348](https://orcid.org/0009-0000-5642-5348))

Research Graph is a structured representation of research objects that
captures information about entities and the relationships between
Researcher, Organisation, Publication, Grant and Research Data.
Currently, the publications are available as PDF files, and due to the
free-form text, it is difficult to parse a PDF file to extract
structured information. In this article, we will try to create a
research graph from a PDF of a publication by extracting relevant
information from the text and organising it into a graph structure
using OpenAI.

<figure>
<img
src="https://cdn-images-1.medium.com/max/1024/1*40j-_agywNgUsJ_h_kXslA.jpeg" />
<figcaption>Pipeline to create the graph from PDF</figcaption>
</figure>

### OpenAI

In this work, we use OpenAI API and the new Assistant functionality of
GPT (Currently in Beta) to transform PDF documents to a set of
structured JSON files based on Research Graph Schema.

#### Assistants API

The Assistants API allows you to build AI assistants within your
applications. An assistant can answer user questions by using models,
tools, and information per predetermined guidelines. It is a beta API
that is actively under development. Using the assistants API, we can use
the OpenAI-hosted tools like code interpreter and knowledge retrieval.
For this blog post, we will focus on knowledge retrieval.

#### Knowledge Retrieval

Sometimes, we need the AI model to answer queries based on unknown
knowledge, like user-provided documents or sensitive information. We can
augment the model with this information using the assistant APIs
knowledge retrieval tool. We can upload files to the assistant, and it
automatically chunks the documents and creates and stores embeddings to
implement vector search on the data.

#### Example

For our example, we will upload PDF files of publications to the OpenAI
assistant and the knowledge retrieval tool to get a JSON output of the
graph schema for the given publications. The publication we used for
this example can be accessed from the following
[link](https://doi.org/10.5166/jroi-2-1-6).

<figure>
<img
src="https://cdn-images-1.medium.com/max/1024/1*CrAVHSNrRmQiBay81_Bi0Q.png" />
<figcaption>Screenshot from the publication (<a
href="https://doi.org/10.5166/jroi-2-1-6">https://doi.org/10.5166/jroi-2-1-6</a>)</figcaption>
</figure>

#### Step 1

Read the input path where the Publications PDFs are stored and the
output path where the JSON output will be stored.

 import configparser
 config = configparser.ConfigParser()
 config.read('{}/config.ini'.format(current_path))
 input_path = config['DEFAULT']['Input-Path']
 output_path = config['DEFAULT']['Output-Path']
 debug = config['DEFAULT']['Debug']

#### Step 2

Get all the PDF files from the Input path.

 onlyfiles = [f for f in os.listdir(input_path) if os.path.isfile(os.path.join(input_path, f))]

#### Step 3

Then, we need to initialise the assistant to use the Knowledge retrieval
tool. To do this, we need to specify the type of tools in the API for
"retrieval". We also specify the instructions for the assistant and the
OpenAI model to be used.

 my_file_ids = []
 if client.files.list().data==[]:
 for f in onlyfiles:
 file = client.files.create(
 file=open(input_path + f, "rb"),
 purpose='assistants'
 )
 my_file_ids.append(file.id)

 # Add the file to the assistant
 assistant = client.beta.assistants.create(
 instructions = "You are a publication database support chatbot. Use pdf files uploaded to best respond to user queries in JSON.",
 model = "gpt-4-1106-preview",
 tools = [{"type": "retrieval"}],
 # Do not attach all files to the assistant, otherwise, it will mismatch the answers even though specify file ID in query messages.
 # We will attach to each message instead
 )

#### Step 4

Then, we specify the information we need to extract from the
publications files and pass them as user queries to the assistant. After
experimenting with the assistant instructions, we found that requesting
JSON format in each user message generates the most consistent output.

 user_msgs = ["Print the title of this paper in JSON",
 "Print the authors of this paper in JSON",
 "Print the abstract section of the paper in JSON",
 "Print the keywords of the paper in JSON",
 "Print the DOI number of the paper in JSON",
 "Print the author affiliations of the paper in JSON",
 "Print the reference section of the paper in JSON"]

#### Step 5

The next is to pass the queries to the assistant to generate the output.
We need to create an individual thread object for each user query, which
contains the query as a user message. Then, we run the thread and
retrieve the assistant's answer.

 all_results = []

 for i in my_file_ids:
 print('\n#####')
 # the JSON result it can extract and parse, hopefully
 file_result = {}
 
 for q in user_msgs:
 # create thread, user message and run objects for each query
 thread = client.beta.threads.create()
 msg = client.beta.threads.messages.create(
 thread_id=thread.id,
 role="user",
 content=q,
 file_ids=[i] # specify the file/publication we want to extract from
 )
 print('\n',q)
 run = client.beta.threads.runs.create(
 thread_id=thread.id,
 assistant_id=assistant.id,
 additional_instructions="If answer cannot be found, print 'False'" # not very useful at the time of this presentation
 )
 # checking run status by retrieving updated object each time
 while run.status in ["queued",'in_progress']:
 print(run.status) 
 time.sleep(5)
 run = client.beta.threads.runs.retrieve(
 thread_id=thread.id,
 run_id=run.id
 )
 # usually a rate limit error
 if run.status=='failed': logging.info("Run failed: ", run)
 if run.status=='completed':
 print("<Complete>")
 # extract updated message object, this includes user messages
 messages = client.beta.threads.messages.list(
 thread_id=thread.id
 )
 for m in messages:
 if m.role=='assistant':
 value = m.content[0].text.value # get the text response
 if "json" not in value:
 if value=='False':logging.info("No answer found for ", str(q))
 else:
 logging.info("Not JSON output, maybe no answer found in the file or model is outdated: ", str(value))
 else:
 # clean the response and try to parse as json
 value = value.split("```")[1].split('json')[-1].strip()
 try: 
 d = json.loads(value)
 file_result.update(d)
 print(d)
 except Exception as e:
 logging.info(f"Query {q} \nFailed to parse string to JSON: ", str(e))
 print(f"Query {q} \nFailed to parse string to JSON: ", str(e))
 
 all_results.append(file_result)

The JSON output generated for the above publication file is:

 [{'title': 'Dodes (diagnostic nodes) for Guideline Manipulation',
 'authors': [{'name': 'PM Putora',
 'affiliation': 'Department of Radiation-Oncology, Kantonsspital St. Gallen, St. Gallen, Switzerland'},
 {'name': 'M Blattner',
 'affiliation': 'Laboratory for Web Science, Zürich, Switzerland'},
 {'name': 'A Papachristofilou',
 'affiliation': 'Department of Radiation Oncology, University Hospital Basel, Basel, Switzerland'},
 {'name': 'F Mariotti',
 'affiliation': 'Laboratory for Web Science, Zürich, Switzerland'},
 {'name': 'B Paoli',
 'affiliation': 'Laboratory for Web Science, Zürich, Switzerland'},
 {'name': 'L Plasswilma',
 'affiliation': 'Department of Radiation-Oncology, Kantonsspital St. Gallen, St. Gallen, Switzerland'}],
 'Abstract': {'Background': 'Treatment recommendations (guidelines) are commonly represented in text form. Based on parameters (questions) recommendations are defined (answers).',
 'Objectives': 'To improve handling, alternative forms of representation are required.',
 'Methods': 'The concept of Dodes (diagnostic nodes) has been developed. Dodes contain answers and questions. Dodes are based on linked nodes and additionally contain descriptive information and recommendations. Dodes are organized hierarchically into Dode trees. Dode categories must be defined to prevent redundancy.',
 'Results': 'A centralized and neutral Dode database can provide standardization which is a requirement for the comparison of recommendations. Centralized administration of Dode categories can provide information about diagnostic criteria (Dode categories) underutilized in existing recommendations (Dode trees).',
 'Conclusions': 'Representing clinical recommendations in Dode trees improves their manageability handling and updateability.'},
 'Keywords': ['dodes',
 'ontology',
 'semantic web',
 'guidelines',
 'recommendations',
 'linked nodes'],
 'DOI': '10.5166/jroi-2-1-6',
 'references': [{'ref_number': '[1]',
 'authors': 'Mohler J Bahnson RR Boston B et al.',
 'title': 'NCCN clinical practice guidelines in oncology: prostate cancer.',
 'source': 'J Natl Compr Canc Netw.',
 'year': '2010 Feb',
 'volume_issue_pages': '8(2):162-200'},
 {'ref_number': '[2]',
 'authors': 'Heidenreich A Aus G Bolla M et al.',
 'title': 'EAU guidelines on prostate cancer.',
 'source': 'Eur Urol.',
 'year': '2008 Jan',
 'volume_issue_pages': '53(1):68-80',
 'notes': 'Epub 2007 Sep 19. Review.'},
 {'ref_number': '[3]',
 'authors': 'Fairchild A Barnes E Ghosh S et al.',
 'title': 'International patterns of practice in palliative radiotherapy for painful bone metastases: evidence-based practice?',
 'source': 'Int J Radiat Oncol Biol Phys.',
 'year': '2009 Dec 1',
 'volume_issue_pages': '75(5):1501-10',
 'notes': 'Epub 2009 May 21.'},
 {'ref_number': '[4]',
 'authors': 'Lawrentschuk N Daljeet N Ma C et al.',
 'title': "Prostate-specific antigen test result interpretation when combined with risk factors for recommendation of biopsy: a survey of urologist's practice patterns.",
 'source': 'Int Urol Nephrol.',
 'year': '2010 Jun 12',
 'notes': 'Epub ahead of print'},
 {'ref_number': '[5]',
 'authors': 'Parmelli E Papini D Moja L et al.',
 'title': 'Updating clinical recommendations for breast colorectal and lung cancer treatments: an opportunity to improve methodology and clinical relevance.',
 'source': 'Ann Oncol.',
 'year': '2010 Jul 19',
 'notes': 'Epub ahead of print'},
 {'ref_number': '[6]',
 'authors': 'Ahn HS Lee HJ Hahn S et al.',
 'title': 'Evaluation of the Seventh American Joint Committee on Cancer/International Union Against Cancer Classification of gastric adenocarcinoma in comparison with the sixth classification.',
 'source': 'Cancer.',
 'year': '2010 Aug 24',
 'notes': 'Epub ahead of print'},
 {'ref_number': '[7]',
 'authors': 'Rami-Porta R Goldstraw P.',
 'title': 'Strength and weakness of the new TNM classification for lung cancer.',
 'source': 'Eur Respir J.',
 'year': '2010 Aug',
 'volume_issue_pages': '36(2):237-9'},
 {'ref_number': '[8]',
 'authors': 'Sinn HP Helmchen B Wittekind CH.',
 'title': 'TNM classification of breast cancer: Changes and comments on the 7th edition.',
 'source': 'Pathologe.',
 'year': '2010 Aug 15',
 'notes': 'Epub ahead of print'},
 {'ref_number': '[9]',
 'authors': 'Paleri V Mehanna H Wight RG.',
 'title': "TNM classification of malignant tumours 7th edition: what's new for head and neck?",
 'source': 'Clin Otolaryngol.',
 'year': '2010 Aug',
 'volume_issue_pages': '35(4):270-2'},
 {'ref_number': '[10]',
 'authors': 'Guarino N.',
 'title': 'Formal Ontology and Information Systems',
 'source': '1998 IOS Press'},
 {'ref_number': '[11]',
 'authors': 'Uschold M Gruniger M.',
 'title': 'Ontologies: Principles Methods and Applications.',
 'source': 'Knowledge Engineering Review',
 'year': '1996',
 'volume_issue_pages': '11(2)'},
 {'ref_number': '[12]',
 'authors': 'Aho A Garey M Ullman J.',
 'title': 'The Transitive Reduction of a Directed Graph.',
 'source': 'SIAM Journal on Computing',
 'year': '1972',
 'volume_issue_pages': '1(2): 131–137'},
 {'ref_number': '[13]',
 'authors': 'Tai K',
 'title': 'The tree-to-tree correction problem.',
 'source': 'Journal of the Association for Computing Machinery (JACM)',
 'year': '1979',
 'volume_issue_pages': '26(3):422-433'}]}]

#### Step 6

File objects and assistant object needs to be cleaned up because they
cost money in 'retrieval' mode. Also, it is a good coding practice.

 for f in client.files.list().data:
 client.files.delete(f.id)

 # Retrieve and delete running assistants
 my_assistants = client.beta.assistants.list(
 order="desc",
 )
 for a in my_assistants.data: 
 response = client.beta.assistants.delete(a.id)
 print(response)

#### Step 7

The next step is to generate a graph visualisation using the Python
[Networkx](https://networkx.org/documentation/stable/tutorial.html) package.

 import networkx as nx
 import matplotlib.pyplot as plt

 G = nx.DiGraph()
 node_colors = []

 key = "jroi/" + all_results[0]['title']
 G.add_nodes_from([(all_results[0]['title'], {'doi': all_results[0]['DOI'], 'title': all_results[0]['title'], 'source': 'jroi', 'key': key})])
 node_colors.append('#4ba9dc')

 for author in all_results[0]['authors']:
 key = "jroi/" + author['name']
 G.add_nodes_from([(author['name'], {'key': key, 'local_id': author['name'], 'full_name': author['name'], 'source': 'jroi'})])
 G.add_edge(all_results[0]['title'], author['name'])
 node_colors.append('#63cc9e')


 for reference in all_results[0]['references']:
 key = "jroi/" + reference['title']
 G.add_nodes_from([(reference['title'].split('.')[0][:25] + '...', {'title': reference['title'], 'source': 'jroi', 'key': key})])
 G.add_edge(all_results[0]['title'], reference['title'].split('.')[0][:25] + '...')
 node_colors.append('#4ba9dc')

 pos = nx.spring_layout(G)
 labels = nx.get_edge_attributes(G, 'label')
 nx.draw(G, pos, with_labels=True, node_size=1000, node_color=node_colors, font_size=7, font_color='black')
 nx.draw_networkx_edge_labels(G, pos, edge_labels=labels)
 plt.savefig("graph_image.png")
 plt.show()

The graph visualisation is as below:

<figure>
<img
src="https://cdn-images-1.medium.com/max/640/1*-a5y19GUSyQZv63KMXKnZA.png" />
<figcaption>Graph Visualisation generated using Networkx</figcaption>
</figure>

**Note:** Please note that the output structure generated by OpenAI may
be different for different executions. So you may need to update the
above code according to that structure.

### Conclusion

In conclusion, leveraging the GPT API to extract a Research Graph from
PDF publications offers a powerful and efficient solution for
researchers and data analysts. This workflow streamlines the process of
converting PDF publications into structured and accessible Research
Graphs. But we must also pay careful attention to the inconsistencies of
the responses generated by LLM. Over time, accuracy and relevance may be
further enhanced by routinely updating and improving the extraction
model.

![](https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=4655f64337db){width="1"
height="1"}

Authors:  Nakul Nambiar (ORCID: 0009-0009-9720-9233) Zhuochen Wu (ORCID: 0009-0000-5642-5348)  Research Graph is a structured representation of research objects that captures information about entities and the relationships between Researcher, Organisation, Publication, Grant and Research Data.

How to use GPT API to export a research graph from PDF publications

University of Glasgow

Roderic Page

I keep returning to the problem of viewing large graphs and trees, which
means my hard drive has accumulated lots of failed prototypes. Inspired
by some recent discussions on comparing taxonomic classifications I
decided to package one of these (wildly incomplete) prototypes up so
that I can document the idea and put the code somewhere safe.

> Very cool, thanks for sharing this\-- the tree diff is similar to what
> J Rees has been cooking up lately with his \'cl diff\' tool. I\'ll tag
> [@beckettws](https://twitter.com/beckettws?ref_src=twsrc%5Etfw) in
> here too so he can see potential crossover. The goal is autogenerate
> diffs like this as 1st step to mapping taxo name-to concept
>
> --- Nate Upham (@n8_upham) [December 28,
> 2021](https://twitter.com/n8_upham/status/1475834371131289608?ref_src=twsrc%5Etfw)

## Google Maps-like viewer

::: {.separator style="clear: both;"}
[![](https://blogger.googleusercontent.com/img/a/AVvXsEiGDVVKGmwRi0gn-kEtpxBCUEdK-WCLBSpmkALAUDHGeTPkTSICSYVgnsj5N7zUeUfQALfFFHJJCsfeFvRULKbmqxLz51rW5hp_11dVXh-FHrnlRA7RJTA7I82l7sERF5jAjlah0LyEheVayO9nAfHTGZDuw5rnCe9iEO3dHQmA8_5AFIlJJg=s400){border="0"
width="400" original-height="448"
original-width="500"}](https://blogger.googleusercontent.com/img/a/AVvXsEiGDVVKGmwRi0gn-kEtpxBCUEdK-WCLBSpmkALAUDHGeTPkTSICSYVgnsj5N7zUeUfQALfFFHJJCsfeFvRULKbmqxLz51rW5hp_11dVXh-FHrnlRA7RJTA7I82l7sERF5jAjlah0LyEheVayO9nAfHTGZDuw5rnCe9iEO3dHQmA8_5AFIlJJg=s500){style="display: block; padding: 1em 0; text-align: center; "}
:::

I\'ve created a simple viewer that uses a tiled map viewer (like Google
Maps) to display a large graph. The idea is to draw the entire graph
scaled to a 256 x 256 pixel tile. The graph is stored in a database that
supports geospatial queries, which means the queries to retrieve the
individual tiles need to display the graph at different levels of
resolution are simply bounding box queries to a database. I realise that
this description is cryptic at best. The GitHub repository
<https://github.com/rdmpage/gml-viewer> has more details and the code
itself. There\'s a lot to do, especially adding support for labels(!)
which presents some interesting challenges ([levels of
detail](https://en.wikipedia.org/wiki/Level_of_detail_(computer_graphics))
and
[generalization](https://en.wikipedia.org/wiki/Cartographic_generalization)).
The code doesn\'t do any layout of the graph itself, instead I\'ve used
the [yEd](https://www.yworks.com/products/yed) tool to compute the x,y
coordinates of the graph.

Since this exercise was inspired by a discussion of the [ASM Mammal
Diversity Database](https://www.mammaldiversity.org), the graph I\'ve
used for the demonstration above is the ASM classification of extant
mammals. I guess I need to solve the labelling issue fairly quickly!

I keep returning to the problem of viewing large graphs and trees, which means my hard drive has accumulated lots of failed prototypes. Inspired by some recent discussions on comparing taxonomic classifications I decided to package one of these (wildly incomplete) prototypes up so that I can document the idea and put the code somewhere safe.

Large graph viewer experiments

Cameron Neylon

When it comes to getting data up on the web, I am actually a great
optimist. I think things are moving in the right direction and with high
profile people like [Tim Berners-Lee making the
case](http://www.ted.com/index.php/talks/tim_berners_lee_on_the_next_web.html),
the with meme of "Linked Data" spreading, and a steadily improving set
of tools and interfaces that make all the complexities of
[RDF](http://en.wikipedia.org/wiki/Resource_Description_Framework),
[OWL](http://en.wikipedia.org/wiki/Web_Ontology_Language),
[OAI-ORE](http://www.openarchives.org/ore/), and other standards
disappear for the average user, there is a real sense that this might
come together. It will take some time; certainly years, possibly a
decade, but we are well along the road.

I am less sanguine about our ability to record the processes that lead
to that data or the process by which raw data is turned into
information. Process remains a real recording problem. Part of the
problem for this is just that scientists are rather poor at recording
process in most cases, often assuming that the data file generated is
enough. Part of the problem is the lack of tools to do the recording
effectively, the reason why we are interested in developing electronic
laboratory recording systems.

Part of it is that the people who have been most effective at getting
data onto the web tend to be informaticians rather than
experimentalists. This has lead in a number of cases to very
sophisticated systems for describing data, and for describing
experiments, that are a bit hazy about what it is they are actually
describing. I mentioned this when [I talked
about](http://blog.openwetware.org/scienceintheopen/2008/08/31/linking-up-open-science-online/)
Peter Murray-Rust and Egon Willighagen's efforts at putting at
[UsefulChem](http://usefulchem.wikispaces.com) experiments into
[Chemistry Markup
Languag](http://en.wikipedia.org/wiki/Chemical_Markup_Language)e. While
we could describe a set of results we got awfully confused because we
weren't sure whether we were describing a specific experiment, the
average of several experiments, or a description of what we thought was
happening in the experiments. This is not restricted to CML. Similar
problems arise with many of the
[MIBBI](http://www.mibbi.org/index.php/Main_Page) guidelines where the
process of generating the data gets wrapped up with the data file. None
of these things are bad, it is just that we may need some extra thought
about distinguishing between classes and instances.

In the world of linked data Process can be thought of either as a link
between objects, or as an object in its own right. The distinction is
important. A link between, say, a material and data files, points to a
vocabulary entry (e.g. PCR, HPLC analysis, NMR analysis). That is the
link creates a pointer to the class of a process, not to a specific
example of carrying out that process. If the process is an object in its
own right then it refers reasonably clearly to a specific instance of
that process (it links this material, to that data file, on this date)
but it breaks the links between the inputs and outputs. While RDF can
handle this kind of break by following the links though it creates the
risk of a rapidly expanding vocabulary to hold all the potential
relationships to different processes.There are other problems with this
view. The object that represents the process. Is it a a log of what
happened? Or is it the instructions to transform inputs into outputs?
Both are important for reproducibility but they may be quite different
things. They may even in some cases be incompatible if for some reason
the specific experiment went off piste (power cut, technical issues, or
simply something that wasn't recorded at the time, but turned out to be
important later).

The problem is we need to think clearly about what it is we mean when we
point at a process. Is it merely a nexus of connections? Things coming
out, things going in. Is it the log of what happened? Or is it the set
of instructions that lets us convert these inputs into those outputs.
People who have thought about this from a vocabulary of informatics
perspective tend to worry mostly about categorization. Once you've
figure out what the name of this process is and where it falls in the
heirachy you are done. People who work on minimal description frameworks
start from the presumption that we understand what the process is (a
microarray experiment, electrophoresis etc). Everything within that is
then internally consistent but it is challenging in many cases to link
out into other descriptions of other processes. Again the distinction
between what was planned to be done, and what was actually done, what
instructions you would give to someone else if they wanted to do it, and
how you would do it yourself if you did it again, have a tendency to be
somewhat muddled.

Basically process is complicated, and multifaceted, with different
requirements depending on your perspective. The description of a process
will be made up of several different objects. There is a collection of
inputs and a collection of outputs. There is a pointer to a
classification, which may in itself place requirements on the process.
The specific example will inherit requirements from the abstract class
(any PCR has two primers, a specific PCR has two specific primers).
Finally we need the log of what happened, as well as the "script" that
would enable you to do it again. On top of this for a complex process
you would need a clear machine-readable explanation of how everything is
related. Ideally a set of RDF statements that explain what the role of
everything is, and links the process into the wider web of data and
things. Essentially the RDF graph of the process that you would use in
an idealized machine reading scenario.

![Diagramatic representation of
process](http://farm4.static.flickr.com/3661/3388812135_91181f34f4_o.jpg){decoding="async"
loading="lazy" width="600" height="450"}

This is all very abstract at the moment but much of it is obvious and
the rest I will try to tease out by example in the next post. At the
core is the point that process is a complicated thing and different
people will want different things from it. This means that it is
necessarily going to be a composite object. The log is something that we
would always want for any process (but often do not at the moment). The
"script", the instructions to repeat the process, whether it be
experimental or analytical is also desirable and should be accessible
either by stripping the correct commands out of the log, or by adapting
a generic set of instructions for that type of process. The new things
is the concept of the graph. This requires two things. Firstly that
every relevant object has an identity that can be referenced. Secondly
that we capture the context of each object and what we do to it at each
stage of the process. It is building systems that capture this as we go
that will be the challenge and what I want to try and give an example of
in the next post.


When it comes to getting data up on the web, I am actually a great optimist. I think things are moving in the right direction and with high profile people like Tim Berners-Lee making the case,  the with meme of “Linked Data” spreading, and a steadily improving set of tools and interfaces that make all the complexities of RDF, OWL, OAI-ORE, and other standards disappear for the average user, there is a real sense that this might come together.

Capturing the record of research process &#8211; Part I

I just wanted to point to [a post that Tim O'Reilly
wrote](http://radar.oreilly.com/2008/11/political-endorsement-appropriate-tech-sites.html)
just before the US election a few weeks back. There was an interesting
discussion about the rights and wrongs of him posting on his political
views and the rights and wrongs of that being linked to from the
O'Reilly Media front page. In amongst the abuse that you have come to
expect in public political discussions there is some thought provoking
stuff. But what I wanted to point out and hopefully revive a discussion
of is a point he makes right near the bottom.

\[I have conflated two levels of comments here (Tim is quoting his own
comment) -- see the original post for the context\]

> > "Thanks to everyone for wading in, especially those of you who are
> > marshalling reasoned arguments and sharing actual sources and
> > references, showing you've done your homework, and helping other
> > people to see the data that helped to shape your point of view. We
> > need a LOT more of that in this discussion, rather than slinging
> > unsupported allegations back and forth.
> >
> > Bringing this back to tech -- showing the data behind your argument
> > is a lot like open source. It's a way of verifying the "code" that's
> > inside your head. If you can't show us your code, it's a lot harder
> > to trust your results!"
>
> Links as source code for your thinking: that's a meme that should
> survive the particulars of this particular debate!

In a sense Tim is advocating the wholesale adoption of the very strong
attribution culture we (like to think we) have in academic research. The
importance of acknowedging your sources is clear but it also has much
more value than that. By tracing back the influences that have brought
someone to a specific conclusion or belief it is possible for other
people to gain a much deeper insight into how those ideas evolved. Being
able to parse the dependencies between ideas, data, samples, papers, and
knowledge in an automatic, machine readable, way is the promise of the
semantic web, but in the meantime just helping the poor old humans to
trace back and understand where someone is coming from is very helpful.


I just wanted to point to a post that Tim O’Reilly wrote just before the US election a few weeks back. There was an interesting discussion about the rights and wrongs of him posting on his political views and the rights and wrongs of that being linked to from the O’Reilly Media front page. In amongst the abuse that you have come to expect in public political discussions there is some thought provoking stuff.

Rogue Scholar Beiträge

Recent Advances in using Machine Learning with Graphs

How to use GPT API to export a research graph from PDF publications

Large graph viewer experiments

Capturing the record of research process – Part I

Links as the source code of our thinking – Tim O’Reilly