Swinburne University of Technology

Nakul Nambiar

Australian National University

Aishwarya Nambissan

A novel approach to improving the efficiency of text search in graph databases utilizing Neo4j, OpenAI, and Typesense. Authors  Nakul Nambiar (ORCID: 0009–0009–9720–9233) Aishwarya Nambissan (ORCID: 0009–0003–3823–6609)  The ability to use cutting-edge tools and frameworks is essential for staying ahead in the ever-changing field of technology.

Typesense and Neo4j in a hybrid information retrieval solution

Qingqin Fang

Exploring the OpenAlex Data Structure and Visualization Author: Qingqin Fang (

 ORCID:

0009–0003–5348–4264)

 Introduction to OpenAlex

In today’s world, the realm of research papers is brimming with countless hot topics, and the sheer volume of publications can be overwhelming.

Unveiling Research Trends through OpenAlex Visualization

Wenyi Pi

Neo4j APOC Library Use Case

 Author

Wenyi Pi (

 ORCID:

0009–0002–2884–277)

 Introduction

In the realm of Neo4j, the APOC (Awesome Procedures on Cypher) library stands as a powerful tool. Previously, We have talked about the importance of APOC in optimising Cypher queries and improving query efficiency in our article Exploring Methods of Cypher Query Optimisations.

Introduction to APOC: Enhancing Neo4j Capabilities

Vaibhav Khobragade

Unlocking the Future of AI: The Transformative Journey of Large Language Models Author · Vaibhav Khobragade (

 ORCID:

0009–0009–8807–5982)

 Introduction

Human language development is innate and evolves throughout life. Machines lack this ability to evolve without advanced AI algorithms.

The Journey of Large Language Models: Evolution, Application, and Limitations

How to efficiently retrieve information for different applications

Author Wenyi Pi (ORCID: 0009-0002-2884-2771) This article aims to explore various ways in which Retrieval-Augmented Generation (RAG) can be utilised to retrieve information and generate responses effectively within the dialogue system. The rationale behind utilising RAG as well as potential ways in which it can be employed effectively will be covered.

Efficient Information Retrieval and Response Generation with Retrieval-Augmented Generation (RAG)

Zhuochen Wu

Authors:  Nakul Nambiar (ORCID: 0009-0009-9720-9233) Zhuochen Wu (ORCID: 0009-0000-5642-5348)  Research Graph is a structured representation of research objects that captures information about entities and the relationships between Researcher, Organisation, Publication, Grant and Research Data.

How to use GPT API to export a research graph from PDF publications

Amanda Kau

Unlocking the Power of Questions — A deep dive into Question Answering Systems

Author:

 Amanda Kau (ORCID:


 0009–0004–4949–9284


 )

Virtual assistants have popped up on numerous websites over the years.

Harnessing The Power of Knowledge Graphs in Question Answering

Amir Aryani

Authors  Nakul Nambiar (ORCID: 0009-0009-9720-9233) Amir Aryani (ORCID: 0000-0002-4259-9774)  Knowledge graphs, which offer a structured representation of data and its relationships, are revolutionising how we organise and access information.

Mini RAG using Neo4j

Author  Amir Aryani (ORCID: 0000-0002-4259-9774) Introduction   In this article we look at Research Graph as an information model , and an approach to connect and capture the connections between research outputs, researchers and research activities. We explore the metadata model, and we discuss how to capture this graph in a Neo4j Graph Database.

Research Graph 101

Aland Astudillo

How to use GROBID to extract text from PDF  Author  Aland Astudillo (ORCID: 0009-0008-8672-3168)  GROBID is a powerful and useful tool based on machine learning that can extract text information from PDF files and other files to a structured format. One of the key challenges in knowledge mining from academic articles is reading the content of PDF files.

How to use GROBID

Dhruv Gupta

Unlocking the power of language models: A deep dive into BERT


 Author:


 Dhruv Gupta (ORCID:


 0009–0004–7109–5403


 )

Clive Humby, in 2006 rightly said, “Data is the new oil”. With data being present everywhere, it has never been more valuable.

Language Models: Deep Dive into BERT

Xuzeng He

An introduction to tools supporting PubMed Author: Xuzeng He (

 ORCID:

0009–0005–7317–7426) PubMed is a free Web literature search service developed and maintained by the National Center for Biotechnology Information (

 NCBI

), and it is also a part of NCBI’s Entrez retrieval system that provides access to a diverse set of 38 databases.

Enhancing PubMed: from a Medical Database to beyond

Exploring the potentials and limitations of Vision Language Models

Author:

 Amanda Kau (ORCID:


 0009–0004–4949–9284


 )

The human brain is more extraordinary than any machine we could build. From an early age, many of us gain the ability to comprehend what our eyes tell us and articulate it. Furthermore, we combine evidence from all our senses to reason.

How Much Can Vision Language Models Really “See”?

Addressing Misinformation, Bias, and Privacy Concerns

Author:

 Wenyi Pi (ORCID:


 0009–0002–2884–2771


 )

Recently, the spread of false or misinformation has become an important concern for people all over the world, especially in the era of big data and multimedia, where people are faced with a flood of information that makes it difficult to determine its authenticity.

The Ethical and Social Impact of Artificial Intelligence

Science in the age of large language models

Generative AI entails a credit–blame asymmetry

Exploring AI’s Ethical Terrain: Addressing Bias, Security, and Beyond

Author: Vaibhav Khobragade (

 ORCID:

0009–0009–8807–5982) Large language models (LLMs) like OpenAI’s GPT-4, Meta’s LLaMA, and Google Gemini (previously called Bard) have showcased their vast capabilities, from passing bar exams and crafting articles to generating images and website code.

Ethics and AI: Confronting the Challenges Ahead

The AI Helper Turning Mountains of Data into Bite-Sized Instructions

Author Aland Astudillo (ORCID: 0009-0008-8672-3168) LLMs have been changing the way the entire world deals with problems and day-by-day tasks. To make them better for specific applications, they need huge amounts of data and complex and expensive approaches to training them.

What is LLMLingua?

Tools and Platform for Integration of Knowledge Graph with RAG pipelines.

Authors Aland Astudillo (ORCID: 0009-0008-8672-3168) Aishwarya Nambissan (ORCID: 0009-0003-3823-6609) Many users of chatbots such as ChatGPT, have encountered the problem of receiving inappropriate or incompatible responses. There are several reasons why this might happen.

Unveiling the Synergy: Retrieval Augmented Generation (RAG) Meets Knowledge Graphs

Author: Vaibhav Khobragade (

 ORCID:

0009–0009–8807–5982) Introduction Large language models (LLMs) are becoming increasingly popular in natural language processing for their superior competence in various applications.

Enhancing Language Models: The Role of Knowledge Graph Augmentation in Overcoming LLM Challenges

Amir Aryani

A brief overview of different types of clustering techniques and their algorithms. Authors Aishwarya Nambissan (ORCID: 0009-0003-3823-6609) Amir Aryani (ORCID: 0000-0002-4259-9774)

 Background

Clustering is a fascinating technique used in machine learning, where patterns or data points are grouped based on their similarities. It’s like finding hidden connections among different data points without predefined labels.

19 Clustering Techniques

Unlocking the power of knowledge graphs in research catalogues: A deep dive into OpenAlex


 Author:


 Dhruv Gupta (ORCID:


 0009–0004–7109–5403


 )

Clive Humby, in 2006 rightly said, “Data is the new oil”. With data being present everywhere, it has never been more valuable.

Knowledge Graphs: Deep Dive into OpenAlex

An Introduction to RA-CM3, MuRAG and RACE Author  Xuzeng He (ORCID: 0009-0005-7317-7426)  Generative Artificial Intelligence (GAI) has demonstrated impressive performances in tasks such as text generation and text-to-image generation.

Recent Advances in using Retrieving Multimodal Information for Augmented Generation

Improving the performance and application of Large Language Models

Author Amanda Kau (ORCID: 0009-0004-4949-9284) Large language models (LLMs) like GPT-4, the engine of products like ChatGPT, have taken centre stage in recent years due to their astonishing capabilities. Yet, they are far from perfect.

Understanding Retrieval Pitfalls: Challenges Faced by Retrieval Augmented Generation (RAG) models

Yunzhong Zhang

Efficient creation of a stoplight report with data dashboard images

Author: Yunzhong Zhang (ORCID: 0009–0002–8177–419X) Comparing data dashboards is crucial for understanding trends and performance differences. Traditionally, this task required manual effort, which was slow and sometimes inaccurate. Now, thanks to OpenAI’s GPT-4 with Vision (GPT-4V), we are able to automate and improve this process.

How to use GPT-4V For Stoplight Report

Author Dhruv Gupta (

 ORCID

: 0009–0004–7109–5403) Introduction Large Language Models (LLMs) have become the new face of Natural language processing (NLP). With their generative power and ability to comprehend human language, the human reliance on these models is increasing every day. However, the LLMs have been known to hallucinate and thus produce wrong outputs.

Prompt engineering: A Way to Smartly Use AI

Recent Advances in Using Machine Learning with Graphs — Part 2 Latest findings in multiple research directions for handling graph construction and network security issues Author · Xuzeng He (

 ORCID:

0009–0005–7317–7426) Introduction A graph, in short, is a description of items linked by relations, where the items of a graph are called nodes (or vertices) and their relations are called edges (or links). Examples of

Recent Advances in Using Machine Learning with Graphs — Part 2

Bridging Human Perception and AI’s Future: The Convergence of Visual Understanding and Semantic Networks Author · Vaibhav Khobragade (

 ORCID:

0009–0009–8807–5982)

 Introduction

The fusion of Vision-Language Models (

 VLMs

), Generative Models, and Knowledge Graphs (

 KGs

) is reshaping how artificial intelligence (AI) understands and interacts with the world.

Tuning Vision-Language Models and Generative Models with Knowledge Graph

Using intelligence to use artificial Intelligence: A deep dive into Prompt Engineering


 Author


 Dhruv Gupta (ORCID:


 0009–0004–7109–5403


 )

Introduction Large Language Models (LLMs) have become the new normal in the field of Natural Language Processing (NLP). With their improved performance and generative power, people around the world are relying on it for

Prompt Engineering

An Overview of Constructing a Knowledge Graph Author · Qingqin Fang (

 ORCID:

0009–0003–5348–4264) 1. Introduction 1.1 What is a Knowledge Graph Knowledge Graphs are structured semantic knowledge bases used to rapidly describe concepts and their relationships in the physical world.

Unlocking Intelligence: The Journey from Data to Knowledge Graph

Understanding Knowledge Networks From A Graph Perspective


 Author:


 Amanda Kau (ORCID:


 0009–0004–4949–9284


 )

Since 2020, over ten million scholarly articles have been published annually. To put that into perspective, say all ten million articles were released on the first day of the year.

Uncovering The Secrets Behind The Most Influential Scholarly Publications With AceMap

Boosting Performance for Knowledge Graphs with Neo4j APOC Library

 Author

Wenyi Pi (ORCID: 0009–0002–2884–2771) Introduction A

 knowledge graph

(graph database) captures information about main entities in a domain and the relationships between them. It was an augmented feature store for connected data which gave access to compute, access and operationalise structure features.

Exploring Methods of Cypher Query Optimisations

Aditya Iyengar

The Most Recent Cutting-Edge Models


 Author

Aditya Iyengar (ORCID: 0009–0005–1959–9724)

 Introduction

The past 6 months marked significant technological leaps in the field of artificial intelligence (AI), especially in the realm of image generation. In this six-month window, multiple innovations redefined the capabilities of text-to-image, video generation, and multimodal AI systems.

The New Era of AI Image Generation

Mingrui Gao

Working Examples


 Author

Mingrui Gao (

 ORCID

: 0009–0005–7271–2677)

 Introduction

Machine learning has revolutionised countless industries, but deploying ML models remains a significant challenge for many data scientists and developers.

What is Cog?

Wendi Fan

Let’s explore MLflow together. Author Wendi Fan (

 ORCID: 0000–0003–0284–9166

) Introduction MLflow is an open-source platform designed to manage the entire lifecycle of machine learning projects. It helps developers and data scientists streamline their workflows by tracking experiments, managing models, and deploying them efficiently.

MLflow Beginner’s Guide: How to Get Started with MLflow

Tarun Krishnan

Unleashing FLUX1.1 with the BFL API


 Author

Tarun Krishnan (

 ORCID

: 0009–0006–6647–127X)

 Introduction

With the growing demand for

 image generation models

in various applications, the need for

 APIs

that seamlessly integrate these models into developers’ workflows has skyrocketed.

The Fast Lane of Image Generation

Tohfa Siddika

The Godfather of AI Author Tohfa Siddika Barbhuiya (

 ORCID

: 0009–0007–2976–4601) In a monumental moment for the world of Artificial Intelligence (AI), Geoffrey Hinton, known as the “Godfather of AI,” has been awarded the Nobel Prize for his pioneering work in neural networks and deep learning. He has been awarded the 2024 Nobel Prize in physics by the Royal Swedish Academy of Sciences.

Geoffrey Hinton Wins Nobel Prize

Evaluating the Role of Self-Reflection in AI: Innovation or Overcomplication?

Author Tohfa Siddika Barbhuiya (

 ORCID

: 0009–0007–2976–4601) Introduction Artificial Intelligence (AI) has made remarkable strides over the past decade, with new advancements pushing the boundaries of what machines can achieve. One of the latest developments in the field is the introduction of “reflection” within AI models.

The Idea of Reflection Models

Zijian Yang

OpenAI Canvas and DevDay: Advancing AI Collaboration with Voice Realtime API, Model Distillation, and More Explore OpenAI’s latest tools and features — Canvas, Realtime API, Prompt Cache, Visual Fine-Tuning Author Zijian Yang (

 ORCID:

0009–0006–8301–7634) Introduction OpenAI recently introduced a new feature called Canvas, which is designed to provide ChatGPT users with a more powerful writing and coding collaboration

OpenAI Canvas and DevDay: Advancing AI Collaboration with Voice Realtime API, Model Distillation…

Working Examples Author Mingrui Gao (

 ORCID

: 0009–0005–7271–2677) Introduction In the rapidly evolving landscape of artificial intelligence, Large Language Models (LLMs) have emerged as powerful tools capable of understanding and generating human-like text. While cloud-based services offer convenient access to these models, there’s a growing demand for local, hands-on experimentation.

What is LM Studio?

A novel compression technique ensuring comparable performance with 70% less parameters

Author Amanda Kau (

 ORCID

: 0009–0004–4949–9284) Introduction The sizes of large language models (LLMs) have been steadily increasing over the last few years.

CompactifAI: Large Language Models Don’t Actually Have To Be Large

Let’s explore ChatGPT’s Advanced Voice Mode Author Wendi Fan (

 ORCID: 0000–0003–0284–9166

) Introduction In today’s fast-paced world, where convenience and efficiency are paramount, OpenAI’s latest addition to ChatGPT, the

 Advanced Voice Mode

, sets a new standard for human-computer interaction.

Unlocking the Power of ChatGPT’s Advanced Voice Mode

Automated Knowledge Graph Construction with Large Language Models — Part 2

 Harvesting the Power and Knowledge of Large Language Models

Author Amanda Kau (

 ORCID

:

 0009–0004–4949–9284

) Introduction Knowledge graphs (KGs) are a structured representation of data in a graphical format, in which entities are represented by nodes and are connected by edges representing relationships

Automated Knowledge Graph Construction with Large Language Models — Part 2

Understanding the Evolutionary Journey of LLMs Author Wenyi Pi (

 ORCID

: 0009–0002–2884–2771) Introduction When we talk about large language models (LLMs), we are actually referring to a type of advanced software that can communicate in a human-like manner. These models have the amazing ability to understand complex contexts and generate content that is coherent and has a human feel.

Brief Introduction to the History of Large Language Models (LLMs)

An improvement architecture superior to the Transformer, proposed by Meta Author · Qingqin Fang (

 ORCID:

0009–0003–5348–4264) Introduction Recently, researchers from Meta and the University of Southern California have introduced a model called Megalodon. They claim that this model can expand the context window of language models to handle millions of tokens without overwhelming your memory.

The longer the context, the better? Unlimited Context Length in Megalodon

Attention mechanism not getting enough attention Author Dhruv Gupta (

 ORCID

: 0009–0004–7109–5403) Introduction As discussed in this article, RNNs were incapable of learning long-term dependencies. To solve this issue both LSTMs and GRUs were introduced. However, even though LSTMs and GRUs did a fairly decent job for textual data they did not perform well.

Transformers Models in NLP

The Three Oldest Pillars of NLP

Author Dhruv Gupta (

 ORCID

: 0009–0004–7109–5403) Introduction Natural Language Processing (NLP) has almost become synonymous with Large Language Models (LLMs), Generative AI, and fancy chatbots. With the ever-increasing amount of textual data and exponential growth in computational knowledge, these models are improving every day.

RNNs vs GRUs vs LSTMs

Large Language Models for Fake News Generation and Detection Author Amanda Kau (

 ORCID

: 0009–0004–4949–9284) Introduction In recent years, fake news has become an increasing concern for many, and for good reason. Newspapers, which we once trusted to deliver credible news through accountable journalists, are vanishing en masse along with their writers.

Are Large Language Models Our Allies or Enemies in the Fight Against Fake News?

A Unified and Collaborative Framework for LLM Author · Qingqin Fang (

 ORCID:

0009–0003–5348–4264) Introduction In today’s rapidly evolving field of artificial intelligence, large language models (LLMs) are demonstrating unprecedented potential. Particularly, the Retrieval-Augmented Generation (RAG) architecture has become a hot topic in AI technology due to its unique technical capabilities.

RAG 2.0 is Coming?

Exploring the Potential of Temporal Feature-Logic Embedding (TFLEX) in Complex Query Resolution

Author · Vaibhav Khobragade (

 ORCID:

0009–0009–8807–5982)

 Introduction

Artificial intelligence (AI) and knowledge representation in the field of temporal knowledge graphs are rapidly gaining interest.

Harnessing Temporal Dynamics: Advanced Reasoning using Temporal Knowledge Graphs

Integrating temporal data into static knowledge graphs

 Author

Amanda Kau

 (ORCID:


 0009–0004–4949–9284


 )

Introduction Knowledge graphs (KGs) have proven to be an effective method of data representation that is increasingly popular. In KGs, entities and concepts are represented as nodes, while the relationships between nodes are depicted as edges.

Dynamic Knowledge Graphs: A Next Step For Data Representation?

Understanding the Balance between Internal Knowledge and External Sources

Author Qingqin Fang (

 ORCID:

0009–0003–5348–4264) Introduction Previous research often emphasized the limitations of LLM’s information acquisition pathways, focusing on enhancing its capabilities in this regard.

When RAG and LLM Conflict: Who Will AI Listen to?

Exploring the Boundaries of Creativity and Responsibility in the Age of AI-Driven Media

 Author

Vaibhav Khobragade (

 ORCID:

0009–0009–8807–5982)

 Introduction

In 2024, the discipline of Generative AI takes a big step forward with the launch of revolutionary models that convert text into dynamic films, altering the landscape of digital content creation.

Generative AI’s Leap into Video in 2024 and its Ethical Horizon

Author  Amir Aryani: (ORCID: 0000-0002-4259-9774) Definition   A research collaboration network is a group of researchers, and practitioners, or both, working together on joint research activities. These networks often span across disciplines, geographic boundaries, and sectors, enabling participants to share resources, expertise, and data to address common research goals more effectively than they could individually.

Research Collaboration Network

Understanding Sequential Data Modelling with Keras for Time Series Prediction

 Author

Wenyi Pi (

 ORCID

: 0009–0002–2884–2771) Introduction Recurrent Neural Networks (RNNs) are a special type of neural networks that are suitable for learning representations of sequential data like text in Natural Language Processing (NLP). We will walk through a complete example of using RNNs for time series prediction, covering

Beginner’s Guide to Recurrent Neural Networks (RNNs) with Keras

Incorporating Knowledge Graphs to explain reasoning processes


 Author

Amanda Kau (

 ORCID:

0009–0004–4949–9284) Introduction Large language models (LLMs) like GPT-4 possess remarkable language abilities, allowing them to function as chatbots, translators, and much more.

Combining Knowledge Graphs With Language Models for Interpretability

Latest findings in multiple research directions for handling graph prediction and optimization Author · Xuzeng He (

 ORCID:

0009–0005–7317–7426) A graph, in short, is a description of items linked by relations, where the items of a graph are called

 nodes

(or vertices) and their relations are called

 edges

(or links). Examples of graphs can include social networks (e.g. Instagram) or knowledge

Recent Advances in using Machine Learning with Graphs

Prompt Engineering — Part 2


 Using intelligence to use artificial Intelligence: A deep dive into Prompt Engineering

Author Dhruv Gupta

 (ORCID:

0009–0004–7109–5403

 )

Introduction In the previous article we discussed what prompt engineering and some of the techniques used for prompt engineering.

Prompt Engineering — Part 2

Using Mistral for Data tagging Author · Xuzeng He (

 ORCID:

0009–0005–7317–7426) Introduction Data tagging, in simple terms, is the process of assigning labels or tags to your data so that they are easier to retrieve or analyse.

How to use Large Language Models to tag your data: A complete tutorial

Ministral — The World’s Best Edge Language Model
 


 Revolutionising On-Device AI: High-Performance, Low-Resource Language Models for Edge Computing


 Author

Tarun Krishnan (

 ORCID

: 0009–0006–6647–127X)

 Introduction

In recent years, there’s been a notable shift in the AI landscape, with many companies moving away from developing massive,

Ministral — The World’s Best Edge Language Model

Transformative Advances in Language Models through External Knowledge Integration

Author: Qingqin Fang (

 ORCID:

0009–0003–5348–4264)

 Introduction

In the dynamic field of natural language processing, the integration of external knowledge has emerged as a pivotal strategy for enhancing the performance of language models.

Deeper understanding of In-Context Retrieval-Augmented Language Models

Enhancing Data Interactivity with LLMs and Neo4j Knowledge Graphs Author Wenyi Pi (

 ORCID

: 0009–0002–2884–2771) Introduction Since OpenAI launched ChatGPT, a

 large language model (LLM)

based chatbot, in 2023, it has set off a technological wave.

The integration of large language models (LLMs) with Neo4j-based knowledge graphs

Latest findings in multiple research directions for tackling reasoning and common sense challenges Author: Xuzeng He (

 ORCID:

0009–0005–7317–7426) Knowledge Graphs, such as Wikidata, contain rich relational information between entities and have been widely used as a structured format for storing and representing relational information.

Large Language Models and Knowledge Graphs: Ways to combine them

From Naive to Modular: Tracing the Evolution of Retrieval-Augmented Generation

Author · Vaibhav Khobragade (

 ORCID:

0009–0009–8807–5982) Introduction Large Language Models (LLMs) have achieved remarkable success.

Three Paradigms of RAG

Latest effort in assessing the security of the code generated by large language models Author · Xuzeng He (

 ORCID:

0009–0005–7317–7426) Introduction With the surge of Large Language Models (LLMs) nowadays, there is a rising trend among developers to use Large Language Models to assist their daily code writing. Famous products include GitHub Copilot or simply ChatGPT.

Large Language Models for Code Writing: Security Assessment

Exploring innovative Strategies in Combating Misinformation with Enhanced Multimodal Understanding Author Wenyi Pi (

 ORCID

: 0009–0002–2884–2771) Introduction Misinformation refers to false or inaccurate information that is often given to someone in a deliberate attempt to make them believe something that is not true. This has a significantly negative impact on public health, political stability and social trust and harmony.

Multimodal Large Language Models for Misinformation Detection and Reasoning

Techniques to integrate Knowledge Graphs into Language Models

 Author

Amanda Kau

 (ORCID:


 0009–0004–4949–9284


 )

Introduction Both knowledge graphs (KGs) and pre-trained language models (PLMs) have gained popularity due to their ability to comprehend world knowledge and their broad applicability.

A Deep Dive Into Knowledge Graph Enhanced Pre-trained Language Models

Understanding the Power and Applications of Natural Language Processing

 Author

Dhruv Gupta (

 ORCID:

0009–0004–7109–5403) Introduction We are living in the era of generative AI. In an era where you can ask AI models almost anything, they will most certainly have an answer to the query. With the increased computational power and the amount of textual data, these models are bound to improve their performance.

NLU vs. NLG: Unveiling the Two Sides of Natural Language Processing

Understanding how RNNs work and its applications


 Author

Wenyi Pi (

 ORCID

: 0009–0002–2884–2771) Introduction In the ever-evolving landscape of artificial intelligence (AI), bridging the gap between humans and machines has seen remarkable progress. Researchers and enthusiasts alike have tirelessly worked across numerous aspects of this field, bringing about amazing advancements.

An Introduction to Recurrent Neural Networks (RNNs)

Harvesting the Power and Knowledge of Large Language Models


 Author

Amanda Kau

 (ORCID:

0009–0004–4949–9284

 )

Introduction Knowledge Graphs are networks that represent data in a graphical format. The beauty of Knowledge Graphs lies in their representation of concepts, events and entities as nodes, and the relationships between them as edges.

Automated Knowledge Graph Construction with Large Language Models

Enhancing Open-Domain Conversational Question Answering with Knowledge-Enhanced Models and Knowledge Graphs


 How knowledge-enhanced language models and knowledge graphs are advancing open-domain conversational question answering

Author:

 Wenyi Pi (ORCID:


 0009-0002-2884-2771


 )

When searching for information on the website, it is common to come across a flood of

Enhancing Open-Domain Conversational Question Answering with Knowledge-Enhanced Models and…

Refining AI Vision: How Retrieval-Augmented Generation Transforms Image Captioning in Large Language Models   Leveraging External Knowledge to Enhance the Descriptive Capabilities of AI Systems Author  Vaibhav Khobragade (ORCID: 0009–0009–8807–5982) Introduction   Large Language Models (LLMs) are artificial intelligence models that are trained on massive amounts of text data in order to generate human-like language and produce coherent

Refining AI Vision: How Retrieval-Augmented Generation Transforms Image Captioning in Large…

An Introduction to Retrieval Augmented Generation (RAG) and Knowledge Graph

Author Qingqin Fang (ORCID: 0009–0003–5348–4264)

 Introduction

Large Language Models (LLMs) have transformed the landscape of natural language processing, demonstrating exceptional proficiency in generating text that closely resembles human language.

Hallucination in Large Language Models and Two Effective Alleviation Pathways

Dhruv Gupta

Improving the performance of Large Language Models Author  Dhruv Gupta (ORCID: 0009-0004-7109-5403)  ChatGPT, which first came out in late 2022, took the world by storm. Since then, various LLM models and LLM based products such as Meta’s Llama and Google’s Gemini have emerged, demonstrating the power of LLMs.

RAG: The next big thing after LLMs?

Latest findings for the use of Knowledge Graph in the field of QA in multiple research directions Author: Xuzeng He (

 ORCID:

0009–0005–7317–7426) Question Answering (QA), the ability to interact with data using natural language questions and obtaining accurate results, has been a long-standing challenge in computer science dating back to the 1960s.

Question Answering enhanced by Knowledge Graph

Latest findings in pre-training graphs and using them for link recommendation Author · Xuzeng He (

 ORCID:

0009–0005–7317–7426) Introduction A graph, in short, is a description of items linked by relations, where the items of a graph are called nodes (or vertices) and their relations are called edges (or links). Examples of graphs can include social networks (e.g. Instagram) or knowledge graphs (e.g. Wikipedia). In Instagram

Working with Graphs: Pre-training and Application

Anthropic’s Groundbreaking AI, Claude 3.5, Uses Computers Like a Human and Becomes a Game-Changer in Automation


 Author

Zijian Yang (

 ORCID:

0009–0006–8301–7634)

 Introduction

Claude 3.5 Receives a Major Upgrade Overnight! As anticipated, Anthropic AI made a significant move this week with the launch of Claude 3.5 Haiku.

New Claude 3.5 Can Control Computer: Outsmarts o1 in Coding and Redefines Agent Capabilities

Solutions to Enhance LLM Performance in Long Contexts Author · Qingqin Fang (

 ORCID:

0009–0003–5348–4264) Introduction In the era of AI breakthroughs, large language models (LLMs) are not just advancements; they are revolutions, transforming how we interact with technology, from casual conversations with chatbots to the intricate mechanisms behind sophisticated data analysis tools.

Navigating the Long Context Conundrum: Challenges in Language Models’ Information Processing

Supervised Fine-tuning, Reinforcement Learning from Human Feedback and the latest SteerLM Author · Xuzeng He (

 ORCID:

0009–0005–7317–7426) Introduction Large Language Models (LLMs), usually trained with extensive text data, can demonstrate remarkable capabilities in handling various tasks with state-of-the-art performance. However, people nowadays typically want something more personalised instead of a general solution.

Fine-tuning Large Language Models: A Brief Introduction

Let’s explore GR-2 together.


 Author

Wendi Fan (

 ORCID: 0000–0003–0284–9166

)

 Introduction

GR-2 is a cutting-edge generative model designed for versatile and generalisable robot manipulation, developed by the Robotics Research Team at ByteDance Research.

GR-2: A Generative Video-Language-Action Model for Robot Manipulation

How Apple’s Research Exposes AI’s Inability to Reason and OpenAI Highlights ChatGPT’s Name-Based Biases


 Author

Zijian Yang (

 ORCID:

0009–0006–8301–7634)

 Introduction

As artificial intelligence continues to evolve, the capabilities and limitations of large language models (LLMs) are under increasing scrutiny.

ChatGPT’s Name Bias and Apple’s Findings on AI’s Lack of Reasoning: Major Flaws Revealed

How AI is Transforming Medicine and Improving Lives
 
 AI in the Healthcare Domain
 


 Author

Aditya Iyengar (ORCID: 0009–0005–1959–9724)

 Introduction

The rapid integration of Artificial Intelligence (AI) into healthcare is no longer just a futuristic vision — it is happening now and at an extraordinary pace.

The Future of Healthcare

Working Examples


 Author

Mingrui Gao (

 ORCID

: 0009–0005–7271–2677)

 Introduction

In the fast-paced world of artificial intelligence, a new platform is gaining attention by making AI-powered content creation more accessible.

What is Glif?

Stability AI’s latest release offering New Models and Features


 Author

Tohfa Siddika Barbhuiya (

 ORCID

: 0009–0007–2976–4601)

 Stable Diffusion 3.5

by

 Stability AI

marks a major step forward in AI-driven image generation.

Stable Diffusion 3.5

Research Graph

Google’s Groundbreaking AI Model Explained

Introduction In the rapidly evolving landscape of artificial intelligence, Google continues to stand at the forefront, consistently pushing the boundaries of what these technologies can achieve. One of their latest advancements, GEMMA2, represents a significant leap in AI capabilities. Let’s delve into the details of this new AI model by Google.

What is Gemma2?

Stories by Research Graph on Medium

> Research Graph is a heterogeneous network that connects research
> outputs, research activities and researchers.

<figure>
<img
src="https://cdn-images-1.medium.com/max/917/1*6SmzMUfWzbDuXSCWs0kgXA@2x.jpeg" />
<figcaption>Image from
https://blog.apo.org.au/2018/02/27/researchgraph-pilot-project</figcaption>
</figure>

## Author 

* Amir Aryani (ORCID: [0000-0002-4259-9774](https://orcid.org/0000-0002-4259-9774))

# Introduction

In this article we look at Research Graph as an information model , and
an approach to connect and capture the connections between research
outputs, researchers and research activities. We explore the metadata
model, and we discuss how to capture this graph in a Neo4j Graph
Database.

While scientific papers and research publications are key components of
the research ecosystem, understanding the research and innovation
ecosystem involves more than just studying research papers. Researchers
need to be aware of currently active projects in their field to avoid
duplication and to identify collaboration opportunities. Also, they need
to learn from completed projects and their outcomes, engage with experts
in their fields, and continuously seek potential funding and
collaborative opportunities.

Scientific papers and research publications are key components of the
research ecosystem. For a young PhD student, the journey begins by
reading academic papers to learn about their field, while established
researchers use the latest publications to keep up with developments,
discoveries, and innovations in their area of expertise. However, a
thorough understanding of the research and innovation ecosystem involves
more than just studying research papers. Researchers need to be aware of
currently active projects in their field to avoid duplication and to
identify collaboration opportunities. They should also learn from
completed projects and their outcomes, engage with experts in their
fields, and continuously seek potential funding and collaborative
opportunities. Arguably, understanding this interconnected ecosystem is
crucial for a successful academic career. The challenge for early career
researchers lies in navigating this ecosystem in a way that provides
insights into the best opportunities. Similarly, for governments,
policymakers, and administrations, comprehending the dynamic components
of innovation, staying abreast of the state of the art, and identifying
research capabilities and expertise represent significant challenges.

Research Graph is composed of five primary node types:

- **Researcher**: An individual who conducts scientific or scholarly
 study, often within a specific field of knowledge. Their role involves
 creating new knowledge through research, which often includes running
 experiments, contributing to studies, and writing articles, books,
 or reports.
- **Publications**: Academic publications include journal articles,
 books, patents, conference abstracts, and theses, serving as the
 primary medium for disseminating research findings. Academic
 publications are often subject to a peer review process to ensure
 quality and validity. In the domain of humanities and social sciences
 (HASS), publications can form media pieces, performance arts, and
 other non traditional research outputs. Publications are essential for
 sharing new knowledge and advancing scholarly discourse.
- **Research Data**: Data that is used or generated in the course of
 academic studies. They form the empirical basis for research in many
 disciplines, providing the raw information needed for analysis,
 conclusions, and validation of research hypotheses.
- **Grants**: Award and grants are financial support provided by funding
 bodies (like government agencies, foundations, or institutions) to
 researchers or research projects. They are pivotal for enabling
 research by providing necessary resources, and they often influence
 the direction and scope of research activities. It is common for an
 awarded grant to support multiple projects or sub projects to an
 hierarchical. structure of research activities.
- **Organisations**: This encompasses institutions like universities,
 research institutes, and academic societies. Organizations play a
 crucial role in the research ecosystem by providing infrastructure,
 resources, and a collaborative environment for conducting research.
 They are also involved in disseminating research findings and may
 participate in the management of research projects and funding.

<figure>
<img
src="https://cdn-images-1.medium.com/max/960/1*1i25PuJG9X8r4n6tQyrK_g@2x.jpeg" />
<figcaption>Image from
https://blog.apo.org.au/2018/02/27/researchgraph-pilot-project, RDA
Plenary 11, when Research Graph announced a partnership with Australia
Policy Observatory</figcaption>
</figure>

### History

Research Graph formed as a project during the [Research Data Alliance
Working Group on Data Registry
Interoperability](https://www.rd-alliance.org/groups/data-description-registry-interoperability.html).
The initial version of this project was providing the graph schema to a
software platform called [Research Data Switchboard
(RD-Switchboard)](https://github.com/rd-switchboard). This was an
initial attempt by Australian National Data Service (Australia), Dryad
(United States), GESIS (Germany) and other partners to create a platform
for creating links between research data collections across global data
repositories. The graph schema later evolved to Research Graph as a meta
model, and later formed the bases for a series of open source connected
projects. Research Graph Foundation, a not for profit foundation, formed
in Australia in 2018 to coordinate these projects. You can learn more
about the foundation at
[www.researchgraph.org](http://www.researchgraph.org).

### Metadata

Research Graph nodes connected by relationships such as authorship links
between researchers and publications, employment links between
researchers and organisations, and funder links between organisations
and grants.

> In the context of this article, we refer to the node properties and
> structure of the relationships as **metadata**.

<figure>
<img
src="https://cdn-images-1.medium.com/max/658/1*w57fFcZs41aAn-GmL5kfRQ@2x.jpeg" />
<figcaption>Research Graph Schema,
https://researchgraph.org/schema</figcaption>
</figure>

The detail list of properties for nodes and relationships are presented
at [www.researchgraph.org/schema](http://www.researchgraph.org/schema).
However, it is important to highlight five main properties that applies
to all nodes:

- **source**: A string value that shows the domain name of the source
 repository such as crossref.org, or orcid.org.
- **local_id**: A string value that contains the local primary
 identifier of this node at the \*source\*.
- **key**: A string value with the function as a uniquely identifying
 the node across the global network of scholarly works. To achieve
 this, we combine the source [second-level
 domain](https://en.m.wikipedia.org/wiki/Second-level_domain)(e.g.
 crossref) and the local_id.
- **type**: A string that represent the type of the node including
 "researcher","organisation", "publication","dataset","grant".
- **last_updated**: Date and time of the last update for the metadata of
 this node in this research graph.

Beyond this properties, all five node types have properties that further
expand the information about each node. Some properties are optional and
some mandatory. For example, **researcher** nodes has a mandatory
property called "full_name" that would contain the complete name of the
researcher. However, it has several optional nodes such as "first_name",
"last_name", "orcid", "scopus_author_id".

### Persistent Identifiers (PIDs)

A persistent identifier is a long-lasting reference to a digital
resource. ORCID is an example of a persistent identifier for a person.
Learn more about [persistent
identifiers](http://www.ands.org.au/guides/persistent-identifiers-awareness).
DOIs (digital object identifiers) are persistent identifiers for things
or entities such as journal articles and
datasets. [Crossref](https://www.crossref.org) and [DataCite](https://www.datacite.org)
are the main organisations assigning DOIs in the academic ecosystem. We
recommend learning about [the work of DataCite and
Crossref](https://project-thor.readme.io/docs/who-are-datacite-and-crossref)\],
[the organization identifier
project](https://orcid.org/blog/2016/10/31/organization-identifier-project-way-forward), [Project
THOR (Technical and Human infrastructure for Open
Research](https://doi.org/10.5438/6423)),
and [Metadata2020](http://www.metadata2020.org/) (the scholarly
community's group effort to advocate for richer shared metadata).

PIDs are essential components for connecting scholarly networks and
forming a research graph. Most research graph nodes have PIDs as their
main identifier. For metadata from ORCID, such as such as the node that
represents https://orcid.org/0000-0002-4259-9774, the node properties
include

- **source**: "orcid.org"
- **local_id**: "0000--0002--4259--9774"
- **Key**:"orcid/0000--0002--4259--9774"
- **orcid**:"0000--0002--4259--9774"
- **first_name**:"Amir"
- **last_name**: "Aryani"
- **full_name**: "Amir Aryani"
- **scopus_author_id**: "35068996400"

Notice how "key" is formed as the results of concatenating the second
level domain name from source ("orcid") to the "local_id" property. In
this case, we use the orcid value as the "local_id" of the node. In
addition, there is a seperate property called "orcid". While this seems
redundant, it provides consistency for nodes from other sources that
might include ORCID identifier for researchers such as open access
repositories.

<figure>
<img
src="https://cdn-images-1.medium.com/max/1024/1*z7bfYB1kjaCJtz6lPtFxOw.png" />
<figcaption>Neo4j Browser, Screenshot from Australian National PID
Graph, <a
href="https://researchgraph.org/national-graph/">https://researchgraph.org/national-graph/</a></figcaption>
</figure>

### Why Neo4j?

For large research graph data, we need a database to capture and analyse
the graph content. At the time of writing this article, there are
several graph databases such as [ArangoDB](https://arangodb.com/),
[Amazon Neptune](https://aws.amazon.com/neptune), and [Azure Cosmos
DB](https://azure.microsoft.com/en-au/products/cosmos-db). However,
Neo4j seems to be one of the most versatile options as it satisfy the
following requirements:

- Neo4j open source version enables free use and customisation for
 research.
- Neo4j has a large and active community with a lot of education
 materials and active user groups.
- Easy to use Neo4j integration with Python, Java and other programming
 languages makes it very easy to start working with this graph
 database.

The most prominent advantage of Neo4j is the programming environment and
user interface. It is highly accessible to novice programmers due to
excellent integration with Java, Python and other programming languages.
In addition, the graph explorer is easily usable by business analysts.
The query language to interact with a Neo4j database is called Cypher.
You can fin a comprehensive list of Cypher commands and useful tutorials
at

- Getting Started with Cypher:
 <https://neo4j.com/docs/getting-started/appendix/tutorials/guide-cypher-basics/>
- Cypher Cheat Sheet:
 <https://neo4j.com/docs/cypher-cheat-sheet/5/auradb-enterprise>
- Cypher Basics:
 <https://vladbatushkov.medium.com/learn-neo4j-cypher-basics-in-30-minutes-94d68a52544>

### Neo4j Labels

A Neo4j graph is constituent of nodes and edges. Both nodes and edges
have properties and labels. Properties are essential for information
retrieval as it enables us to find query database for particular nodes
and relationships. Labels are key components of the graph that provides
a level of classification. It enables grouping nodes and relationships
and connect them the same way we connect rows in a SQL database in a
form a table, or collect JSON objects in a MongoDB database in a
collection. The main advantage of these labels is indexing. We can index
properties agains labels. Indexes in Neo4j are pairs of \<property,
index\>, and that is essential for large graphs. Grouping nodes by
labels and creating index at this level enables smaller, and
faster indexes.

We have two major labels in a Research Graph Neo4j database:

- Source: these are labels that reference to the source Property such as
 "orcid", "corssref", and "datacite".
- Type: these labels refer to five main node types including
 "publication", "dataset", "researcher", "organisation", and "grant".

Note: Labels and properties are always lowercase, this enables us to
have consistent cypher queries.

### Summary

Research Graph is a heterogeneous network that connects research
outputs, research activities and researchers. This network provides
insights into the digital footprint of research, and we can leverage
Research Graph to analyse science and innovation networks such as
citation networks, collaboration networks, and research communities. In
addition, the Research Graph networks are the pathways to trace and
analyse the impact of research funding and research projects. In this
article, we explored the foundation of the Research Graph meta model,
recommended the minimum metadata and we introduced Neo4j as the best
approach to capture and link Research Graph nodes.

![](https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=0e1ab5df4497){width="1"
height="1"}

### How to use GROBID to extract text from PDF

<figure>
<img
src="https://cdn-images-1.medium.com/max/1024/1*uyD7hkSBZGCOn4wqu2TgMw.png" />
<figcaption>Created using DALL.E on 13 Feb 2024</figcaption>
</figure>

### Author

* Aland Astudillo (ORCID: [0009-0008-8672-3168](https://orcid.org/0009-0008-8672-3168))

GROBID is a powerful and useful tool based on machine learning that can
extract text information from PDF files and other files to a structured
format.

One of the key challenges in knowledge mining from academic articles is
reading the content of PDF files. The ability to efficiently and
accurately read the content of PDF documents using automated tools and
transform this data into a structured model provides many opportunities
for the integration of PDF collections to expert systems, and
AI-assisted data pipelines such as chatbots.

In this article, we will discuss GROBID capabilities, explain how to
install it, and explore how to use it in a Python script or Jupyter
Notebook.

### **What is GROBID?**

GROBID stands for GeneRation of Bibliographic Data and is a machine
learning library for extracting, parsing, and re-structuring raw
documents such as PDF into structured XML/TEI encoded documents. It has
a particular focus on technical and scientific publications. TEI stands
for Text Encoding Initiative (<https://tei-c.org/Guidelines/>). It
defines and documents a markup language for representing the structural,
renditional, and conceptual features of texts.

### How it works?

GROBID is a machine-learning library that extracts information from PDF
files. The goal of GROBID is to help the text mining process,
information extraction, and semantic analysis of scientific publications
by transforming them into machine-friendly, structured, representations
with predictable structure.

Despite the large amount of PDF files and their extensive use among all
fields, it is still a big challenge to process them because they are
encoded in a vast variety of different publisher formats, often
incomplete or inconsistent. In short, PDF files are difficult to exploit
and use.

In a nutshell, GROBID uses a cascade of "sequence labelling models" to
parse a document. Each model from the collection of models is in charge
of identifying a small amount of labels to classify areas and
information in the document. The final models produce 55 labels where
each piece of identified information from the document will be
structured. More information about how this library has been developed
and the machine learning models are in this link:
<https://grobid.readthedocs.io/en/latest/Principles/>

### How to install GROBID

Here we explain the installation of the GROBID tool as a service in a
Linux machine (Ubuntu for example). The same steps apply to a Windows
system using the Windows Subsystem for Linux tool (WSL version 2).

For Linux, in the terminal with super user (sudo) rights create a folder
and once inside, proceed to download the last version of GROBID using
the wget tool. The current version of GROBID is v 0.8.0. After that,
unzip the file:

 > wget https://github.com/kermitt2/grobid/archive/0.8.0.zip
 > unzip 0.8.0.zip

For Windows, you can use Docker containers to install and run the GROBID
service locally. More documentation is available at
<https://grobid.readthedocs.io/en/latest/Run-Grobid/>

Note: At the time of writing this article, GROBID requires JAVA(JDK) and
it works fine with JDK version 1.11 up to version JDK 1.17. Other recent
JDK versions should work correctly. To check your JAVA installation, in
the terminal use the following command:

 > java -version

If you do not have any JAVA installed, proceed to install JAVA using the
following line in the terminal.

 > sudo apt install default-jdk

It is a good practice to check Java after installation to ensure the
right version of Java is installed and ready to operate on your system.

### **How to run GROBID**

Once you downloaded and unzipped the "*grobid"* file, proceed to the
folder and run the grewed file. In the terminal ensure to use "./"
before the command "*grewed".*

 > cd grobid-0.8.0

 > ./grewed run

The first time that you run the service, it will download and install
all the required dependencies and then it will run the service. It will
be completed by printing a message "**:grobid-service:run"** as
presented in the following screenshot.

<figure>
<img
src="https://cdn-images-1.medium.com/max/920/1*gEsGJ87IUzEovXwcFRDCzg.png" />
</figure>

### **How to use GROBID**

After running the GROBID service, you can connect to the created server
in the following direction using your browser:

<http://localhost:8070/>

This follows the default configuration, but you can change the port in
the config file. The image below shows how it looks the first time that
you access the web layout.

<figure>
<img
src="https://cdn-images-1.medium.com/max/1003/1*N5URo-rzuOb_1Cu15yEKEA.png" />
</figure>

The TEI menu will allow you to choose among the different services to
call and to upload a file with the button "Select file". The following
functionalities are available as a service:

- Header extraction and parsing
- References extraction and parsing
- Citation contexts recognition and resolution
- Full-text extraction and structuring
- PDF coordinates
- Parsing of references
- Parsing of names
- Parsing of affiliation and address
- Parsing of dates
- Consolidation and resolution of the extracted bibliographical
 references
- Extraction and parsing of patent and non-patent references in patent
 publications
- Extraction of funders and funding information

Once you upload a PDF document and choose the required service, you can
press the Submit button

<figure>
<img
src="https://cdn-images-1.medium.com/max/995/1*m3zDNnvFesFQuQ7Bv0U3yA.png" />
</figure>

After a few seconds, the web will show you the XML structure of the
output depending on what service you have selected. You can download the
resulting XML structure in an XML file. The image below shows the result
of processing the full-text document for a given paper. For example, in
the image below we used the article *Pan et al., 2024 Unifying Large
Language Models and Knowledge Graphs: A Roadmap. arXiv:2306.08302*
downloaded from arXiv (<https://arxiv.org/abs/2306.08302>)

<figure>
<img
src="https://cdn-images-1.medium.com/max/993/1*xTfpktk_4nH1dOPqoETvkg.png" />
</figure>

### **How to use GROBID Python Client**

The GROBID Python client is a handy tool that can process efficiently
and concurrently a set of PDF files in a given folder using GROBID as a
service without requiring the web option. It includes a command line for
processing PDFs and writing the results in a given folder. It allows us
to use this tool inside Python scripts. To use this client, It assumes
that you are running GROBID service, as we saw in the previous steps.

The repository of this client is here
<https://github.com/kermitt2/grobid_client_python/blob/master/grobid_client/grobid_client.py>

To install the GROBID Python client to use in your scripts or Jupyter
Notebook, in your Python environment, you can use the pip manager in a
given environment:

 !pip install grobid_client_python

If you want to use this client in a script or a Jupyter Notebook, you
must import the GROBID client class. Remember that it assumes that the
GROBID service is already running.

 from grobid_client.grobid_client import GrobidClient

 client = GrobidClient(config_path="./config.json")

Once the client connects to the service it will show you the following
message.

 "GROBID server is up and running"

### **How to analyse a bunch of PDF documents**

After checking the connection, you can call the process method to
analyse a bunch of documents in a folder. The process method has 3
required parameters and other optional parameters. In the following
example we are using the service "**processFulltextDocument**" for
processing all the pdf files in the folder "**resources/test_pdf**" and
get the xml files output on the folder "**resources/test_out/**".
Additionally, we set the consolidate citation option and the TEI
coordinates to "true".

 client.process("processFulltextDocument", 
 "./resources/test_pdf", 
 output="./resources/test_out/", 
 consolidate_citations=True, 
 tei_coordinates=True, 
 force=True)

In the defined folder, the output will be the XML files for each PDF
file with the re-structured information.

### **How to analyse a single PDF file**

For analysing one PDF file at a time, you can use the method
**process_pdf**, with similar parameters as the **process** method:

 service_name = "processFulltextDocument"

 pdf_file = "./pdf_examples/input/0046d83a-edd6-4631-b57c-755cdcce8b7f.pdf"

 rsp = client.process_pdf(service_name, pdf_file, 
 generateIDs=True, 
 consolidate_header=True, 
 consolidate_citations=True, 
 include_raw_citations=True, 
 include_raw_affiliations=True, 
 tei_coordinates=True, 
 segment_sentences=True)

The following text snippet shows the top section of the "rsp" variable
that holds the results from the PDF processing.

 ('./pdf_examples/input/0046d83a-edd6-4631-b57c-755cdcce8b7f.pdf',
 200,
 '<?xml version="1.0" encoding="UTF-8"?> 
 <TEI xml:space="preserve" xmlns="http://www.tei-c.org/ns/1.0" 
 xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" 
 xsi:schemaLocation="http://www.tei-c.org/ns/1.0 https://raw.githubusercontent.com/kermitt2/grobid/master/grobid-home/schemas/xsd/Grobid.xsd" 
 xmlns:xlink="http://www.w3.org/1999/xlink"> 
 <teiHeader xml:lang="en"> 
 <fileDesc> 
 <titleStmt> 
 <title level="a" type="main" xml:id="_nrS3kCw">Multi-contact functional electrical stimulation for hand opening: electrophysiologically driven identification of the optimal stimulation site</title> 
 <funder ref="#_ZTFTHCu"> 
 <orgName type="full">German Federal Ministry for Education and Research [BMBF</orgName> 
 </funder> 
 <funder ref="#_xTkqs2w"> 
 <orgName type="full">German Research Council</orgName> 
 </funder> 
 <funder ref="#_dV4wQd4 #_dVJJnCd"> 
 <orgName type="full">Federal Ministry for Education and Research</orgName> 
 </funder> 
 </titleStmt> 
 <publicationStmt> 
 <publisher>Springer Science and Business Media LLC</publisher> 
 <availability status="unknown">
 Copyright Springer Science and Business Media LLC 
 </availability> 
 <date type="published" when="2016-03-08">2016-03-08</date> 
 </publicationStmt> 
 <sourceDesc> 
 <biblStruct> 
 <analytic> 
 <author> 
 <persName coords="1,56.69,234.15,85.88,9.71"><forename type="first">Cristiano</forename>
 <surname>De Marchis</surname></persName> 
 <email>cristiano.demarchis@uniroma3.it</email> 
 <affiliation key="aff0"> 
 <note type="raw_affiliation"><label>1</label> Division of Functional and Restorative Neurosurgery, Department of Neurosurgery, Eberhard Karls University, Otfried-Mueller-Str.45, 72076 Tübingen, Germany</note> 
 <orgName type="department" key="dep1">Division of Functional and Restorative Neurosurgery</orgName> 
 <orgName type="department" key="dep2">Department of Neurosurgery</orgName> 
 <orgName type="institution">Eberhard Karls University</orgName> 
 <address> 
 <addrLine>Otfried-Mueller-Str.45</addrLine> 
 <postCode>72076</postCode> 
 <settlement>Tübingen</settlement> 
 <country key="DE">Germany</country> 
 </address> 
 </affiliation> 
 </author> 
 …

(Note: The result was truncated)

As you can see, the information is organised hierarchically, using
standardised tags \<tag\> to identify each type of information of the
publication. For instance, the tag *\<titleStmt\>* contains sub-tags
holding the title of the article.

 <title level="a" type="main" xml:id="_nrS3kCw">
 Multi-contact functional electrical stimulation for hand opening: electrophysiologically driven identification of the optimal stimulation site
 </title> 

Other tags contain information about authors, affiliations, and so on.
For example, the identification of the organisation is in the tag
*\<orgName\>*

 <orgName type="full">
 German Federal Ministry for Education and Research [BMBF
 </orgName> 

Information about the authors is inside the tags *\<author\>* containing
details of each author, complete name (\<forename), last name
(*\<surname\>*), email (*\<email\>*), affiliations (*\<affiliation
key="aff0\"\>*), and all other information like the address of the
institution to which the author is affiliated (*\<address\>*), and all
the information contained in the article about the author.

 <author> 
 <persName coords="1,56.69,234.15,85.88,9.71">
 <forename type="first">Cristiano</forename>
 <surname>De Marchis</surname>
 </persName> 
 <email>cristiano.demarchis@uniroma3.it</email> 
 <affiliation key="aff0"> 
 <note type="raw_affiliation"><label>1</label> Division of Functional and Restorative Neurosurgery, Department of Neurosurgery, Eberhard Karls University, Otfried-Mueller-Str.45, 72076 Tübingen, Germany</note> 
 <orgName type="department" key="dep1">Division of Functional and Restorative Neurosurgery</orgName> 
 <orgName type="department" key="dep2">Department of Neurosurgery</orgName> 
 <orgName type="institution">Eberhard Karls University</orgName> 
 <address> 
 <addrLine>Otfried-Mueller-Str.45</addrLine> 
 <postCode>72076</postCode> 
 <settlement>Tübingen</settlement> 
 <country key="DE">Germany</country> 
 </address> 
 </affiliation> 
 </author>

In the same way, we can retrieve information on the citations of the
article in the tag for references. For example, an article in the list
of references has the following structure

 <biblStruct coords="8,319.75,636.26,214.89,6.62;8,319.75,645.27,210.17,6.62;8,319.75,654.28,118.20,6.62" xml:id="b0">
 <analytic>
 <title level="a" type="main">The influence of functional electrical stimulation on hand motor recovery in stroke patients: a review</title>
 <author>
 <persName xmlns="http://www.tei-c.org/ns/1.0">
 <forename type="first">Fanny</forename>
 <surname>Quandt</surname>
 </persName>
 </author>
 <author>
 <persName xmlns="http://www.tei-c.org/ns/1.0">
 <forename type="first">Friedhelm</forename>
 <forename type="middle">C</forename>
 <surname>Hummel</surname>
 </persName>
 </author>
 <idno type="DOI">10.1186/2040-7378-6-9</idno>
 </analytic>
 <monogr>
 <title level="j">Experimental & Translational Stroke Medicine</title>
 <title level="j" type="abbrev">Exp & Trans Stroke Med</title>
 <idno type="ISSNe">2040-7378</idno>
 <imprint>
 <biblScope unit="volume">6</biblScope>
 <biblScope unit="issue">1</biblScope>
 <biblScope unit="page">9</biblScope>
 <date type="published" when="2014-08-21"/>
 <publisher>Springer Science and Business Media LLC</publisher>
 </imprint>
 </monogr>
 </biblStruct>

In which we can identify the title tag (*\<title level="a"
type="main"\>*), authors information (*\<author\>*), DOI (*\<idno
type="DOI"\>*), and the journal details (*\<monogr\>*) of the referenced
article.

Finally, to get each piece of information from this structure for
further analysis, the next steps involves extracting the required
information based on the tags of the XML structure, which can be done
with different tools to match these tags in the text. That said, the
text is ready for mining all the information.

### **Conclusion**

We have explored the important aspects of GROBID. With its simple
installation process user interface, and scripts, we can put GROBID to
work easier than ever. Whether you are facing mountains of PDFs or
struggling to tame unruly bibliographic data, GROBID is your trusty
companion on the path to research efficiency. Download GROBID today and
unlock the door to a world of streamlined workflows, saving time, and
deeper insights from your research endeavours.

**Materials for further reading:**

- <https://grobid.readthedocs.io/en/latest/>
- <https://github.com/kermitt2/grobid_client_python>

![](https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=67df995b16fa){width="1"
height="1"}

<figure>
<img
src="https://cdn-images-1.medium.com/max/1024/1*7Fy1vUav7gAiy9CAWanAvQ.png" />
<figcaption>Created using DALLE on 6 Feb 2024</figcaption>
</figure>

#### Authors:

* Nakul Nambiar (ORCID: [0009-0009-9720-9233](https://orcid.org/0009-0009-9720-9233))
* Zhuochen Wu (ORCID: [0009-0000-5642-5348](https://orcid.org/0009-0000-5642-5348))

Research Graph is a structured representation of research objects that
captures information about entities and the relationships between
Researcher, Organisation, Publication, Grant and Research Data.
Currently, the publications are available as PDF files, and due to the
free-form text, it is difficult to parse a PDF file to extract
structured information. In this article, we will try to create a
research graph from a PDF of a publication by extracting relevant
information from the text and organising it into a graph structure
using OpenAI.

<figure>
<img
src="https://cdn-images-1.medium.com/max/1024/1*40j-_agywNgUsJ_h_kXslA.jpeg" />
<figcaption>Pipeline to create the graph from PDF</figcaption>
</figure>

### OpenAI

In this work, we use OpenAI API and the new Assistant functionality of
GPT (Currently in Beta) to transform PDF documents to a set of
structured JSON files based on Research Graph Schema.

#### Assistants API

The Assistants API allows you to build AI assistants within your
applications. An assistant can answer user questions by using models,
tools, and information per predetermined guidelines. It is a beta API
that is actively under development. Using the assistants API, we can use
the OpenAI-hosted tools like code interpreter and knowledge retrieval.
For this blog post, we will focus on knowledge retrieval.

#### Knowledge Retrieval

Sometimes, we need the AI model to answer queries based on unknown
knowledge, like user-provided documents or sensitive information. We can
augment the model with this information using the assistant APIs
knowledge retrieval tool. We can upload files to the assistant, and it
automatically chunks the documents and creates and stores embeddings to
implement vector search on the data.

#### Example

For our example, we will upload PDF files of publications to the OpenAI
assistant and the knowledge retrieval tool to get a JSON output of the
graph schema for the given publications. The publication we used for
this example can be accessed from the following
[link](https://doi.org/10.5166/jroi-2-1-6).

<figure>
<img
src="https://cdn-images-1.medium.com/max/1024/1*CrAVHSNrRmQiBay81_Bi0Q.png" />
<figcaption>Screenshot from the publication (<a
href="https://doi.org/10.5166/jroi-2-1-6">https://doi.org/10.5166/jroi-2-1-6</a>)</figcaption>
</figure>

#### Step 1

Read the input path where the Publications PDFs are stored and the
output path where the JSON output will be stored.

 import configparser
 config = configparser.ConfigParser()
 config.read('{}/config.ini'.format(current_path))
 input_path = config['DEFAULT']['Input-Path']
 output_path = config['DEFAULT']['Output-Path']
 debug = config['DEFAULT']['Debug']

#### Step 2

Get all the PDF files from the Input path.

 onlyfiles = [f for f in os.listdir(input_path) if os.path.isfile(os.path.join(input_path, f))]

#### Step 3

Then, we need to initialise the assistant to use the Knowledge retrieval
tool. To do this, we need to specify the type of tools in the API for
"retrieval". We also specify the instructions for the assistant and the
OpenAI model to be used.

 my_file_ids = []
 if client.files.list().data==[]:
 for f in onlyfiles:
 file = client.files.create(
 file=open(input_path + f, "rb"),
 purpose='assistants'
 )
 my_file_ids.append(file.id)

 # Add the file to the assistant
 assistant = client.beta.assistants.create(
 instructions = "You are a publication database support chatbot. Use pdf files uploaded to best respond to user queries in JSON.",
 model = "gpt-4-1106-preview",
 tools = [{"type": "retrieval"}],
 # Do not attach all files to the assistant, otherwise, it will mismatch the answers even though specify file ID in query messages.
 # We will attach to each message instead
 )

#### Step 4

Then, we specify the information we need to extract from the
publications files and pass them as user queries to the assistant. After
experimenting with the assistant instructions, we found that requesting
JSON format in each user message generates the most consistent output.

 user_msgs = ["Print the title of this paper in JSON",
 "Print the authors of this paper in JSON",
 "Print the abstract section of the paper in JSON",
 "Print the keywords of the paper in JSON",
 "Print the DOI number of the paper in JSON",
 "Print the author affiliations of the paper in JSON",
 "Print the reference section of the paper in JSON"]

#### Step 5

The next is to pass the queries to the assistant to generate the output.
We need to create an individual thread object for each user query, which
contains the query as a user message. Then, we run the thread and
retrieve the assistant's answer.

 all_results = []

 for i in my_file_ids:
 print('\n#####')
 # the JSON result it can extract and parse, hopefully
 file_result = {}
 
 for q in user_msgs:
 # create thread, user message and run objects for each query
 thread = client.beta.threads.create()
 msg = client.beta.threads.messages.create(
 thread_id=thread.id,
 role="user",
 content=q,
 file_ids=[i] # specify the file/publication we want to extract from
 )
 print('\n',q)
 run = client.beta.threads.runs.create(
 thread_id=thread.id,
 assistant_id=assistant.id,
 additional_instructions="If answer cannot be found, print 'False'" # not very useful at the time of this presentation
 )
 # checking run status by retrieving updated object each time
 while run.status in ["queued",'in_progress']:
 print(run.status) 
 time.sleep(5)
 run = client.beta.threads.runs.retrieve(
 thread_id=thread.id,
 run_id=run.id
 )
 # usually a rate limit error
 if run.status=='failed': logging.info("Run failed: ", run)
 if run.status=='completed':
 print("<Complete>")
 # extract updated message object, this includes user messages
 messages = client.beta.threads.messages.list(
 thread_id=thread.id
 )
 for m in messages:
 if m.role=='assistant':
 value = m.content[0].text.value # get the text response
 if "json" not in value:
 if value=='False':logging.info("No answer found for ", str(q))
 else:
 logging.info("Not JSON output, maybe no answer found in the file or model is outdated: ", str(value))
 else:
 # clean the response and try to parse as json
 value = value.split("```")[1].split('json')[-1].strip()
 try: 
 d = json.loads(value)
 file_result.update(d)
 print(d)
 except Exception as e:
 logging.info(f"Query {q} \nFailed to parse string to JSON: ", str(e))
 print(f"Query {q} \nFailed to parse string to JSON: ", str(e))
 
 all_results.append(file_result)

The JSON output generated for the above publication file is:

 [{'title': 'Dodes (diagnostic nodes) for Guideline Manipulation',
 'authors': [{'name': 'PM Putora',
 'affiliation': 'Department of Radiation-Oncology, Kantonsspital St. Gallen, St. Gallen, Switzerland'},
 {'name': 'M Blattner',
 'affiliation': 'Laboratory for Web Science, Zürich, Switzerland'},
 {'name': 'A Papachristofilou',
 'affiliation': 'Department of Radiation Oncology, University Hospital Basel, Basel, Switzerland'},
 {'name': 'F Mariotti',
 'affiliation': 'Laboratory for Web Science, Zürich, Switzerland'},
 {'name': 'B Paoli',
 'affiliation': 'Laboratory for Web Science, Zürich, Switzerland'},
 {'name': 'L Plasswilma',
 'affiliation': 'Department of Radiation-Oncology, Kantonsspital St. Gallen, St. Gallen, Switzerland'}],
 'Abstract': {'Background': 'Treatment recommendations (guidelines) are commonly represented in text form. Based on parameters (questions) recommendations are defined (answers).',
 'Objectives': 'To improve handling, alternative forms of representation are required.',
 'Methods': 'The concept of Dodes (diagnostic nodes) has been developed. Dodes contain answers and questions. Dodes are based on linked nodes and additionally contain descriptive information and recommendations. Dodes are organized hierarchically into Dode trees. Dode categories must be defined to prevent redundancy.',
 'Results': 'A centralized and neutral Dode database can provide standardization which is a requirement for the comparison of recommendations. Centralized administration of Dode categories can provide information about diagnostic criteria (Dode categories) underutilized in existing recommendations (Dode trees).',
 'Conclusions': 'Representing clinical recommendations in Dode trees improves their manageability handling and updateability.'},
 'Keywords': ['dodes',
 'ontology',
 'semantic web',
 'guidelines',
 'recommendations',
 'linked nodes'],
 'DOI': '10.5166/jroi-2-1-6',
 'references': [{'ref_number': '[1]',
 'authors': 'Mohler J Bahnson RR Boston B et al.',
 'title': 'NCCN clinical practice guidelines in oncology: prostate cancer.',
 'source': 'J Natl Compr Canc Netw.',
 'year': '2010 Feb',
 'volume_issue_pages': '8(2):162-200'},
 {'ref_number': '[2]',
 'authors': 'Heidenreich A Aus G Bolla M et al.',
 'title': 'EAU guidelines on prostate cancer.',
 'source': 'Eur Urol.',
 'year': '2008 Jan',
 'volume_issue_pages': '53(1):68-80',
 'notes': 'Epub 2007 Sep 19. Review.'},
 {'ref_number': '[3]',
 'authors': 'Fairchild A Barnes E Ghosh S et al.',
 'title': 'International patterns of practice in palliative radiotherapy for painful bone metastases: evidence-based practice?',
 'source': 'Int J Radiat Oncol Biol Phys.',
 'year': '2009 Dec 1',
 'volume_issue_pages': '75(5):1501-10',
 'notes': 'Epub 2009 May 21.'},
 {'ref_number': '[4]',
 'authors': 'Lawrentschuk N Daljeet N Ma C et al.',
 'title': "Prostate-specific antigen test result interpretation when combined with risk factors for recommendation of biopsy: a survey of urologist's practice patterns.",
 'source': 'Int Urol Nephrol.',
 'year': '2010 Jun 12',
 'notes': 'Epub ahead of print'},
 {'ref_number': '[5]',
 'authors': 'Parmelli E Papini D Moja L et al.',
 'title': 'Updating clinical recommendations for breast colorectal and lung cancer treatments: an opportunity to improve methodology and clinical relevance.',
 'source': 'Ann Oncol.',
 'year': '2010 Jul 19',
 'notes': 'Epub ahead of print'},
 {'ref_number': '[6]',
 'authors': 'Ahn HS Lee HJ Hahn S et al.',
 'title': 'Evaluation of the Seventh American Joint Committee on Cancer/International Union Against Cancer Classification of gastric adenocarcinoma in comparison with the sixth classification.',
 'source': 'Cancer.',
 'year': '2010 Aug 24',
 'notes': 'Epub ahead of print'},
 {'ref_number': '[7]',
 'authors': 'Rami-Porta R Goldstraw P.',
 'title': 'Strength and weakness of the new TNM classification for lung cancer.',
 'source': 'Eur Respir J.',
 'year': '2010 Aug',
 'volume_issue_pages': '36(2):237-9'},
 {'ref_number': '[8]',
 'authors': 'Sinn HP Helmchen B Wittekind CH.',
 'title': 'TNM classification of breast cancer: Changes and comments on the 7th edition.',
 'source': 'Pathologe.',
 'year': '2010 Aug 15',
 'notes': 'Epub ahead of print'},
 {'ref_number': '[9]',
 'authors': 'Paleri V Mehanna H Wight RG.',
 'title': "TNM classification of malignant tumours 7th edition: what's new for head and neck?",
 'source': 'Clin Otolaryngol.',
 'year': '2010 Aug',
 'volume_issue_pages': '35(4):270-2'},
 {'ref_number': '[10]',
 'authors': 'Guarino N.',
 'title': 'Formal Ontology and Information Systems',
 'source': '1998 IOS Press'},
 {'ref_number': '[11]',
 'authors': 'Uschold M Gruniger M.',
 'title': 'Ontologies: Principles Methods and Applications.',
 'source': 'Knowledge Engineering Review',
 'year': '1996',
 'volume_issue_pages': '11(2)'},
 {'ref_number': '[12]',
 'authors': 'Aho A Garey M Ullman J.',
 'title': 'The Transitive Reduction of a Directed Graph.',
 'source': 'SIAM Journal on Computing',
 'year': '1972',
 'volume_issue_pages': '1(2): 131–137'},
 {'ref_number': '[13]',
 'authors': 'Tai K',
 'title': 'The tree-to-tree correction problem.',
 'source': 'Journal of the Association for Computing Machinery (JACM)',
 'year': '1979',
 'volume_issue_pages': '26(3):422-433'}]}]

#### Step 6

File objects and assistant object needs to be cleaned up because they
cost money in 'retrieval' mode. Also, it is a good coding practice.

 for f in client.files.list().data:
 client.files.delete(f.id)

 # Retrieve and delete running assistants
 my_assistants = client.beta.assistants.list(
 order="desc",
 )
 for a in my_assistants.data: 
 response = client.beta.assistants.delete(a.id)
 print(response)

#### Step 7

The next step is to generate a graph visualisation using the Python
[Networkx](https://networkx.org/documentation/stable/tutorial.html) package.

 import networkx as nx
 import matplotlib.pyplot as plt

 G = nx.DiGraph()
 node_colors = []

 key = "jroi/" + all_results[0]['title']
 G.add_nodes_from([(all_results[0]['title'], {'doi': all_results[0]['DOI'], 'title': all_results[0]['title'], 'source': 'jroi', 'key': key})])
 node_colors.append('#4ba9dc')

 for author in all_results[0]['authors']:
 key = "jroi/" + author['name']
 G.add_nodes_from([(author['name'], {'key': key, 'local_id': author['name'], 'full_name': author['name'], 'source': 'jroi'})])
 G.add_edge(all_results[0]['title'], author['name'])
 node_colors.append('#63cc9e')


 for reference in all_results[0]['references']:
 key = "jroi/" + reference['title']
 G.add_nodes_from([(reference['title'].split('.')[0][:25] + '...', {'title': reference['title'], 'source': 'jroi', 'key': key})])
 G.add_edge(all_results[0]['title'], reference['title'].split('.')[0][:25] + '...')
 node_colors.append('#4ba9dc')

 pos = nx.spring_layout(G)
 labels = nx.get_edge_attributes(G, 'label')
 nx.draw(G, pos, with_labels=True, node_size=1000, node_color=node_colors, font_size=7, font_color='black')
 nx.draw_networkx_edge_labels(G, pos, edge_labels=labels)
 plt.savefig("graph_image.png")
 plt.show()

The graph visualisation is as below:

<figure>
<img
src="https://cdn-images-1.medium.com/max/640/1*-a5y19GUSyQZv63KMXKnZA.png" />
<figcaption>Graph Visualisation generated using Networkx</figcaption>
</figure>

**Note:** Please note that the output structure generated by OpenAI may
be different for different executions. So you may need to update the
above code according to that structure.

### Conclusion

In conclusion, leveraging the GPT API to extract a Research Graph from
PDF publications offers a powerful and efficient solution for
researchers and data analysts. This workflow streamlines the process of
converting PDF publications into structured and accessible Research
Graphs. But we must also pay careful attention to the inconsistencies of
the responses generated by LLM. Over time, accuracy and relevance may be
further enhanced by routinely updating and improving the extraction
model.

![](https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=4655f64337db){width="1"
height="1"}

A novel approach to improving the efficiency of text search in graph
databases utilizing Neo4j, OpenAI, and Typesense.

### Authors

* Nakul Nambiar (ORCID: [0009–0009–9720–9233](https://orcid.org/0009-0009-9720-9233))
* Aishwarya Nambissan (ORCID: [0009–0003–3823–6609](https://orcid.org/0009-0003-3823-6609))

<figure>
<img
src="https://cdn-images-1.medium.com/max/1024/1*3_mdx5SSgImdy1c2RZzl1A.png" />
<figcaption>Created using DALL-E</figcaption>
</figure>

The ability to use cutting-edge tools and frameworks is essential for
staying ahead in the ever-changing field of technology. When it comes to
natural language processing, search capabilities, and graph databases,
**LLM** (Large Language Models), **Typesense**, and **Neo4j** work
together as a powerful trio that offers a cohesive and effective
solution.

As we explore further, we will learn about the unique advantages of
Typesense, Neo4j, and LLM and how their combined powers may open up a
new world of possibilities. This blog is your resource for
comprehending, putting into practice, and optimizing the potential of
this influential trio, whether you're a developer looking for creative
solutions, a data scientist seeking improvements in analytics, or a
business professional trying to maximize information retrieval.

### What is Neo4j and Typesense?

Neo4j is a well-known graph database that specializes in managing
densely linked data. Because of its superior relationship management
capabilities, it is a well-known option for applications with complex
data models. Conversely, Typesense is an extremely quick, open-source
search engine that can quickly search through and retrieve a lot of
text-based data.

While Neo4j is a powerful tool for traversing relationships and querying
graph data, its built-in text search features are not effective for
handling large amounts of text. Here, we will use Typesense to enhance
Neo4j's text search capabilities by using a customized solution for
effective and quick search operations.

### Text Search Challenges in Neo4j

**Memory**: Neo4j relies heavily on memory. Thus, it may consume a lot
of memory for intensive text search queries.

**Lack of Specialised Text Search Features**: Neo4j is a graph database,
and its main function is to handle and store graph-based data. As
compared to other search tools, it lacks some specialized text search
features, which may have a negative impact on the result.

In this article, we are using a Neo4j Graph database that uses this
[schema](https://researchgraph.org/schema) to demonstrate the challenges
for text search and potentially a working solution. The graph includes
five primary objects including research publications, researchers,
organizations, academic grants, and research data collections, capturing
the collaborative network of academic works.

The following is a subset of the schema that contains information
relevant to this article.

<figure>
<img
src="https://cdn-images-1.medium.com/max/1024/1*6l7kfZAin3Bwp_gZdU6U1g.jpeg" />
<figcaption>Publication and grant schema from <a
href="https://researchgraph.org/schema/">Research Graph
Meta Model</a></figcaption>
</figure>

<figure>
<img
src="https://cdn-images-1.medium.com/max/1024/1*lrtN5375h6OAuwtGN9KNYA.jpeg" />
<figcaption>Example of the Research Graph schema</figcaption>
</figure>

To illustrate the difficulty at hand, if we wish to find research grants
and publications associated with 'Artificial Intelligence', we will
encounter the following issues:

\(a\) We will need to find all alternative or relevant terms (e.g. AI,
ML, \...)

\(b\) We will need to have a (computationally) expensive text index
search across Neo4j nodes as Neo4j does not particularly perform well
for text search.

To address this issue we are employing a text-based search engine called
Typesense. But first, let\'s look at the advantages and capabilities of
both tools in our toolbox.

### Advantages of Neo4j

**Flexibility:** The graph model is adaptable and may quickly change to
reflect new relationships and data structures.

**Cypher Query Language:** Neo4j uses cypher query language, which is
expressive, readable, and more suitable for graph databases. It makes
retrieving complex patterns from graph databases easier.

**Performance:** Neo4j's underlying architecture is optimized to query
nodes and relationships in a graph efficiently, thus improving the
performance.

**Scalability:** Neo4j supports horizontal scalability. It lets you add
more machines to handle large datasets when needed.

**Transactional Support:** Neo4j ensures data consistency and integrity
in transactional operations by adhering to the ACID (Atomicity,
Consistency, Isolation, Durability) standards.

### Advantages of Typesense

**Speed**: As a search engine designed just for text queries, Typesense
performs very well in terms of speed. We can offload the text-search
tasks to Typesense and increase the speed of the application.

**Rich Text Search**: Typesense provides various advanced text search
features, typo tolerance, faceted, hybrid, and fuzzy search. Hence, it
can accommodate typos and potential errors in user inputs.

**Scalable**: Typesense can scale horizontally. Thus, it can handle
large datasets and a huge volume of queries.

**Easy Integration**: Typesense has simple APIs and connectors for
different programming languages, making integrating with other platforms
and software accessible.

### Vector Search

Typesense can use various small LLM models to create embeddings for the
text and then use these embeddings to do a nearest neighbor
search (KNN).

An embedding for a document is an alternate representation of data as
floating point values. These embeddings are generated using Machine
Learning models. In principle, two similar documents would have
embeddings "closer" to each other.

For example, using OpenAI, we can convert text to embeddings using
OpenAI APIs as below:

 from openai import OpenAI
 client = OpenAI()

 response = client.embeddings.create(
 input="Your text string goes here",
 model="text-embedding-ada-002"
 )

 print(response.data[0].embedding)

The generated embeddings will be as follows:

 {
 "data": [
 {
 "embedding": [
 -0.006929283495992422,
 -0.005336422007530928,
 ...
 -4.547132266452536e-05,
 -0.024047505110502243
 ],
 "index": 0,
 "object": "embedding"
 }
 ],
 "model": "text-embedding-ada-002",
 "object": "list",
 "usage": {
 "prompt_tokens": 5,
 "total_tokens": 5
 }
 }
 

Typesense converts text in each document using the LLM model mentioned
in the schema.

Here are some common models you can use to generate these document
embeddings:

- [Sentence-BERT](https://www.sbert.net/)
- [E-5](https://medium.com/@hansahettiarachchi/unleashing-the-potential-of-embedding-model-e5-revolutionizing-natural-language-comprehension-3f1516489048)
- [CLIP](https://github.com/openai/CLIP)
- [OpenAI's Text Embeddings
 model](https://platform.openai.com/docs/api-reference/embeddings/create)
- [Google's
 PaLM API](https://developers.generativeai.google/products/makersuite)
- [Google's
 Vertex API](https://cloud.google.com/vertex-ai/docs/generative-ai/embeddings/get-text-embeddings)

You can find the list of built-in models supported by Typesense
[here](https://huggingface.co/typesense/models/tree/main).

For example, below is a schema for a collection of grants that creates
embeddings for the attribute "grant-summary" using the
"all-MiniLM-L12-v2" LLM model for creating the embeddings. The created
embeddings are stored as a new attribute called embedding in the same
document.

 grant_schema = {
 "name": "grant",
 "fields": [
 {"name": "grant-summary", "type": "string"},
 {"name": "flat-field-of-research", "type": "string"},
 {
 "name": "embedding",
 "type": "float[]",
 "embed": {
 "from": [
 "grant-summary"
 ],
 "model_config": {
 "model_name": "ts/all-MiniLM-L12-v2"
 }
 }
 }
 ]
 }

### Demo

We will make use of OpenAI, Typesense, and Neo4j to get relevant data
for a keyword from the Neo4j graph database.

**Resources**: We have a Virtual Machine (VM) with a Neo4j server
running with the grants data embedded on the "grant-summary" attribute.
Typesense is running on the same VM on port 8108. Lastly, we need an
OpenAI API key that can be generated from [OpenAI](https://openai.com/).

In this example, we will search for a keyword and all related related
terms. We will use OpenAI to generate the related words to the keyword.
We will then use these related words to do a semantic search in the
Typesense database to get a list of IDs. Using this list of IDs, we can
then query the Neo4j database to get the graph.

<figure>
<img
src="https://cdn-images-1.medium.com/max/1024/1*5ng10X_cLfH_1o2rvK-cwQ.jpeg" />
<figcaption>Pipeline steps for OpenAI-Typesense-Neo4j integrated
search</figcaption>
</figure>

1. First, we accept the user input, which is the keyword that needs to
 be searched.

2\. Then, this keyword is passed to OpenAI, which generates a set of 5
keywords that are related to the given keyword.

 prompt = "We are searching academic articles for research related to a
 keyword presented by the user. Recommend five alternative terms 
 that find related papers to the given keyword. Only create keywords 
 and no other description in your response. Produce CSV"

 user_prompt = "The keyword I want search for articles is" + search_query_term


 import openai

 completion = openai.ChatCompletion.create(
 model="gpt-4-1106-preview",
 messages=[ 
 {"role": "system", "content": prompt},
 {"role": "user", "content": user_prompt}
 ]
 )

For the keyword "Artificial Intelligence", OpenAI generated the
following output below:

 {
 "role": "assistant",
 "content": "\"machine learning\", \"neural networks\", \"cognitive computing\", \"natural language processing\", \"robotics\""
 }

3\. The generated keywords are then searched in Typesense to get a set
of DOIs related to the keywords.

 import typesense

 client = typesense.Client({
 'nodes': [{
 'host': 'localhost', 
 'port': '8108', 
 'protocol': 'http'
 }],
 'api_key': api_key,
 'connection_timeout_seconds': 300
 })

 search_parameters = {
 'searches': [],
 
 }

 for term in completion.choices[0].message["content"].split(','):
 search_object = {
 'q': term.strip(),
 'query_by': 'embedding',
 'exclude_fields': 'embedding',
 'limit': 2,
 'collection': 'publications'
 }
 search_parameters["searches"].append(search_object)

 common_search_params={}
 data = client.multi_search.perform(search_parameters, common_search_params);
 dois = []
 for search_result in data['results']:
 for doc in search_result['hits']:
 dois.append(doc['document']['DOI'])
 print(doc['document']['DOI'] , ' ' , doc['document']['title'])

4\. DOIs can then be easily queried from the Neo4j database to get the
related data and graph.

<figure>
<img
src="https://cdn-images-1.medium.com/max/1024/1*27ltmtCCpQPnPvkj6uuetw.png" />
<figcaption>Graph for the keyword “Artificial Intelligence”</figcaption>
</figure>

### Conclusion

To sum up, the integration of Neo4j, OpenAI, and Typesense is a
combination that tackles the difficulties related to text search in
graph databases.

With this integration, developers can take advantage of the benefits of
each of the platforms, making text search more effective, scalable, and
feature-rich. For those looking to improve the search capabilities of
their apps, the Neo4j-OpenAI-Typesense combination seems to be a strong
option as businesses struggle with ever-more complicated data models.

![](https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=603718bff21e){width="1"
height="1"}