Australian National University

Vaibhav Khobragade

#### **Exploring the Potential of Temporal Feature-Logic Embedding (TFLEX) in Complex Query Resolution**

<figure>
<img
src="https://cdn-images-1.medium.com/max/1024/0*H6ME71WASxb20ZcS" />
<figcaption>Weaving Time in Knowledge Graphs by ChatGPT
(DALLE2).</figcaption>
</figure>

### Author

· Vaibhav Khobragade (**ORCID:**
[0009--0009--8807--5982](https://orcid.org/0009-0009-8807-5982))

### **Introduction**

Artificial intelligence (AI) and knowledge representation in the field
of temporal knowledge graphs are rapidly gaining interest. Current
challenges include developing models and frameworks that can effectively
form reasoning mechanisms. **Temporal knowledge graphs (TKGs)** refer to
a specific group of knowledge representation systems that besides
concepts, entities, events, and relationships, capture time information
(Li et al., 2021). These graphs give so much useful material in the
processes of forecasting, organizing plans, and decision-making.

TKGs are active databases that not only drive the information but also,
the context of true incidents when they happen and that makes them
crucial in the prediction of forthcoming trends and the understanding of
complex sequences of events.

Innovative techniques termed **Temporal Feature-Logic Embedding
framework (TFLEX)** were introduced, which is purposefully built to deal
with temporal and logical issues especially to handle queries that have
elements of time information, that classic static KGs would fail to
deal with.

***Let's build a fundamental concept before we delve into TFLEX:***

### **What is a Knowledge Graph?**

A knowledge graph is a way of storing information that uses a
graph-structured model. In this model, entities are represented as nodes
and the relationship between them are represented as edges, connected to
the nodes. This structure depicts complex interrelationships
within data.

For example: An entity "Inception" is connected to an entity "Leonardo
DiCaprio" with the relationship "acted in" and to "Christopher Nolan"
entity with the relationship "directed by".

<figure>
<img src="https://cdn-images-1.medium.com/max/940/0*Xa_zd0gENuy-GKlf" />
<figcaption>Figure1: Knowledge graph (entities and relationship
example)</figcaption>
</figure>

This structure makes it easy to understand the connections between
pieces of information and to use them for reasoning, searching, and
knowledge discovery.

### What is a Temporal Knowledge Graph?

A Temporal Knowledge Graph (TKG) enhances traditional static knowledge
graphs by incorporating time data into the structure. This is achieved
by adding a timestamp to the usual triple format, resulting in elements
described as \<head, relation, tail, timestamp\>.

For instance, consider the data point \<Bill Gates, makes a visit,
India, 2024--04--23\>; these records not only the event and the entities
involved but also the exact date it occurred.

### **TFLEX**

Recent research on **Temporal Knowledge Graph Completion (TKGC)** aims
to infer new facts in TKGs that focus on temporal link prediction, which
is simply a single hop. It tries to predict if two nodes that are not
currently connected will form a single direct link in the future based
on the history of the network and its evolution over time. Some TKGC
methods like **timestamp-based transformation, dynamic embedding, and
Markov process** **model** work by performing multi-hop reasoning.
However, none of them could answer logical questions involving
conjunction, negation, and disjunction operation

For instance, find all scientists who have published their research in
the field of Artificial Intelligence (Conjunction), have never received
an award (Negation), or have collaborated with certain universities
before 2020 (Before and Disjunction). In this article, we focused on
multi-hop logical reasoning to answer the temporal complex queries. To
understand this method, we should know what temporal complex
queries are.

### **Temporal Complex Queries**

This type of query for Temporal Knowledge Graphs (TKGs) specifically
includes both entity and temporal elements. These queries are complex
because they require reasoning across multiple connections (multi-hop)
and involve both entities and the times associated with their
relationships. For example, a query might ask which events a person
attended before a certain year and who else was present at those events.
This type of query goes beyond simple fact retrieval, incorporating
logical and temporal operators to extract structured information across
time and entities as discussed above.

In a mathematical way, a temporal complex query can be represented as:

<figure>
<img src="https://cdn-images-1.medium.com/max/940/0*L6jPwbDy21yoABqZ" />
</figure>

The above query will be discussed in more detail in the paragraphs
that follow.

### **Data Generation Process**

Researchers formulated the above temporal complex queries that involve
both entities and their temporal relationships and then applied these
queries to three different popular TKGs to generate three datasets
(ICEWS14, ICEWS05--15, and GDELT-500) for further training and testing
this model.

Multi-hop logical reasoning over knowledge graphs is a complex process
that involves answering queries by traversing multiple relationships (or
"hops") between entities in a knowledge graph. In simpler terms, it's
like solving a puzzle where each piece (or hop) brings you closer to the
complete answer by following a trail of interconnected data points.

<figure>
<img src="https://cdn-images-1.medium.com/max/940/0*BCcPSrLzstsI3hVF" />
<figcaption>Figure 2: An example of a multi-hop temporal complex query
and its processing network is as follows: “Which countries did Xi
Jinping visit but Barack Obama did not visit during François Hollande’s
presidency of France?” Entity sets (blue circles), timestamp sets (green
triangles), entity set projections (blue arrows), and logical operators
(red rectangles) are represented in the computation graph. (Lin et al.,
2022) Article link: <a
href="https://arxiv.org/abs/2205.14307">https://arxiv.org/abs/2205.14307</a></figcaption>
</figure>

1. **Entity and Timestamp Representation**:

- **Entity Nodes (Circles)**: Entities such as "Xi Jinping" and "Barack
 Obama" are represented as blue circles.
- **Timestamp Nodes (Green Triangles)**: The period of François
 Hollande's presidency is represented as a green triangle, indicating
 the specific timeframe relevant to the query.

**2. Entity and Time Set Projections (Arrows)**:

- **Blue Arrows**: These represent entity set projections. For instance,
 an arrow from the entity node "Xi Jinping" to a logical operation
 indicates that the action of visiting countries is being queried
 specifically for Xi Jinping.
- **Green Arrows**: These represent time set projections. They connect
 the timestamp node of Hollande's presidency to other operations,
 applying the time filter to the activities of the entities involved.

**3. Logical Operations (Red Rectangles)**:

- **Visit Operations**: Separate logical operations check which
 countries each leader visited during Hollande's presidency. These
 operations are linked from the entity nodes (Xi Jinping, Barack Obama)
 through blue arrows, indicating entity-focused actions within the
 specified time, linked through green arrows.
- **AND NOT Operation**: This logical operator compares the outputs from
 the Xi and Obama visit operations. It filters out any overlaps, thus
 excluding countries visited by Obama from Xi's list.

**4. Final Output (Node)**:

- The output node at the end of the computation graph provides the
 answer to the query. It lists countries visited by Xi Jinping but not
 by Barack Obama during the defined timeframe. This is achieved after
 processing through the logical AND and NOT operation, which ensures
 only Xi's unique visits are counted.

<figure>
<img src="https://cdn-images-1.medium.com/max/940/0*xEF6vn1R_85HA6xH" />
<figcaption>Figure 3: The computation of time part in temporal operators
Before and After. (Lin et al., 2022) Article link: <a
href="https://arxiv.org/abs/2205.14307">https://arxiv.org/abs/2205.14307</a></figcaption>
</figure>

The computation graph in Figure 3 represents how these time adjustments
are made within the embeddings. It shows that only the time components
are modified in the highlighted colors leaving the entity-related parts
unchanged in faded gray color. It focuses solely on the temporal shift
implied by the query which is important for reasoning over knowledge
that changes over time.

### **Model Prediction in temporal context before and after the event**

<figure>
<img src="https://cdn-images-1.medium.com/max/940/0*gCoU-bQvkz2ubPny" />
<figcaption>Figure 4: Score distributions of Pt,
bPt and aPt, where s is
Citizen (India), r is Criticize or
denounce, o is Sadhu (India),
ground-truth answer of Pt is
2014–03–27. (Lin et al., 2022) Article link: <a
href="https://arxiv.org/abs/2205.14307">https://arxiv.org/abs/2205.14307</a></figcaption>
</figure>

- Pt is projection in time that shows the most likely time for the event
 to have occurred, which is aligned with ground truth date.
- bPt (Before Projection Time): bPt indicates time earlier than the peak
 for "Pt," as highlighted by the 'before' region in the graph.
- aPt(After Projection Time): aPt indicates time earlier than the peak
 for "Pt," as highlighted by the 'after' region in the graph.

<figure>
<img src="https://cdn-images-1.medium.com/max/940/0*UZSHCPLELDoiXMIs" />
<figcaption>Figure 5: Score distributions of Pt,
bPt and aPt, where s is
Citizen (India), r is Use
unconventional violence, o is Chief Secretary
Chandra, ground-truth answer of Pt is
2014–05–11. (Lin et al., 2022) Article link: <a
href="https://arxiv.org/abs/2205.14307">https://arxiv.org/abs/2205.14307</a></figcaption>
</figure>

Figure 4 & 5 uses the x-axis to detail the model's performance across
different types of queries, demonstrating how predictions align with
ground truth data over time. In simple words, the purpose of the above
visualization is to show the model's ability to predict when events
occur relative to a known point in time. ***"Pt"*** is the model's
direct prediction, while ***"bPt" and "aPt"*** show the model's
understanding of the temporal context before and after the event,
respectively.

### **Result**

Take this query example: Philippines denounced or criticized who on
2014--04--01?

<figure>
<img
src="https://cdn-images-1.medium.com/max/1004/0*Ja6buG4h-uwSHuA0" />
<figcaption>Table 1: Intermediate variable assignments and ranks for
example query. Correctness indicates whether the answer belongs to the
ground-truth set of answers.</figcaption>
</figure>

Taking another query example: On 2014--04--04, who consulted the man who
was appealed to or requested by the Head of Government (Latvia) on
2014--08--01?

<figure>
<img src="https://cdn-images-1.medium.com/max/977/0*SCTi1GRsq-phc64R" />
<figcaption>Table 2: Intermediate variable assignments and ranks for
example second query. Correctness indicates whether the answer belongs
to the ground-truth set of answers.</figcaption>
</figure>

In Table 2, the query "On 2014--04--04, who consulted the man who was
appealed to or requested by the Head of Government (Latvia) on
2014--08--01?" was taken and then TFLEX was used to execute a query and
get an answer. The model successfully classified the level of
difficulties of the answer type into Easy and Hard. It is also able to
classify which answer is correct and wrong. The table shows that TFLEX
gives simple answers "François Hollande", "Taavi Rõivas", and hard
answers "Andris Berzins" a higher ranking than incorrect answers "Angela
Merkel" and "Head of Government (Latvia)". This demonstrates how TFLEX
correctly determines the hard answer through sophisticated reasoning and
separates the correct responses from incorrect ones.

### **Industry Applications**

Temporal knowledge graphs (TKGs) have diverse applications:

1. **Recommendation Systems**: TKGs are used to model recommendation
 systems by analyzing users' past behaviours such as liking, buying,
 watching, reviewing, and so on. It can also offer more personalized
 and timely recommendations, such as suggesting products based on
 recent browsing and purchasing trends.
2. **Financial Application** In financial markets, TKGs can help in
 holding behaviours, trading behaviours, and financial events that
 aid in better investment decisions.
3. **Transportation**: In the field of transportation, temporal
 knowledge graphs can be used to improve traffic prediction models
 and enable intelligent transportation systems. (Yuan & Li, 2021)

### **Conclusion**

This post looks at how artificial intelligence is getting better by
using Temporal Knowledge Graphs (TKGs) and a special method called
Temporal Feature-Logic Embedding (TFLEX). TKGs are advanced versions of
knowledge graphs that include time information, which helps in
understanding and predicting complex patterns. TFLEX is designed to deal
with these time-related challenges and improves how we can answer
detailed questions about past and future events. This is useful in areas
like finance, transportation, and personalized recommendations, where
understanding time can make a big difference.

### **References**

- Lin, X., Xu, C., E, H., Su, F., Zhou, G., Hu, T., Li, N., Sun, M., &
 Luo, H. (2022, May 28). TFLEX: Temporal Feature-Logic Embedding
 Framework for Complex Reasoning over Temporal Knowledge Graph.
 arXiv.org. <https://doi.org/10.48550/arXiv.2205.14307>
- Li, M., Tong, P., Li, M., Jin, Z., Huang, J., & Hua, X. S. (2021, May
 18). Traffic Flow Prediction with Vehicle Trajectories. Proceedings of
 the . . . AAAI Conference on Artificial Intelligence.
 <https://doi.org/10.1609/aaai.v35i1.16104>
- Yuan, H., & Li, G. (2021, January 23). A Survey of Traffic Prediction:
 from Spatio-Temporal Data to Intelligent Transportation. Data Science
 and Engineering. <https://doi.org/10.1007/s41019-020-00151-z>

![](https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=0da8483262f9){width="1"
height="1"}

Exploring the Potential of Temporal Feature-Logic Embedding (TFLEX) in Complex Query Resolution

Author · Vaibhav Khobragade (

 ORCID:

0009–0009–8807–5982)

 Introduction

Artificial intelligence (AI) and knowledge representation in the field of temporal knowledge graphs are rapidly gaining interest.

Rogue Scholar Posts

Harnessing Temporal Dynamics: Advanced Reasoning using Temporal Knowledge Graphs