Yunzhong Zhang

**Efficient creation of a stoplight report with data dashboard images**

<figure>
<img
src="https://cdn-images-1.medium.com/max/1000/0*SgLFlOJN2wLSMtg0" />
<figcaption>Image generated by DALL.E</figcaption>
</figure>

### Author:

- Yunzhong Zhang (ORCID:
 [0009--0002--8177--419X](https://orcid.org/0009-0002-8177-419X))

Comparing data dashboards is crucial for understanding trends and
performance differences. Traditionally, this task required manual
effort, which was slow and sometimes inaccurate. Now, thanks to OpenAI's
GPT-4 with Vision (GPT-4V), we are able to automate and improve this
process. This article introduces how to use GPT-4V to compare two data
dashboards quickly and accurately.

Traditionally, to compare two data dashboards and generate a stoplight
report that highlights the differences between them, the process
requires the machine to possess the capability to recognise and compare
text within the images. This involves at least three steps:

1. **Text Extraction:** Initially, data displayed on both dashboards
 must be extracted. This is achieved through Optical Character
 Recognition (OCR), which extracts text from the images.
2. **Data Extraction (Feature Selection):** The next step is to
 identify the key metrics and features presented in both dashboards,
 focusing on the essential data for comparison.
3. **Comparison and Stoplight Report Generation:** With the data
 extracted from the two dashboards, it's possible to compare the
 differences and generate a stoplight report based on these findings.

However, customising the code structure at each step is often necessary.
This is particularly true for comparing charts, graphs, or other visual
elements within the dashboards, which requires an understanding of the
types of visualisations used (e.g., bar charts, line graphs, heatmaps)
and the data they represent.

ChatGPT makes life easy. ChatGPT, developed by OpenAI, can handle a wide
range of natural language processing (NLP) tasks. In September 2023,
OpenAI released the multimodal model GPT-4 with Vision (GPT-4V), which
marries the language processing prowess of GPT-4 with image analysis
capabilities, offering a comprehensive multimodal AI experience. With
GPT-4V, the task of comparing data dashboards is greatly streamlined. By
crafting an appropriate prompt and supplying the dashboard images to
GPT-4V, the model can analyse the data within the images and provide a
detailed stoplight report, making the comparison process more efficient
and user-friendly.

### What is GPT-4V and Stoplight report?

GPT-4 with vision enables users to instruct
[GPT-4](https://openai.com/research/gpt-4) (a Large Multimodal Model) to
analyse image inputs and text prompts provided by the user. With the
ability to interpret and analyse data presented in visual formats, the
GPT-4V is able to monitor the performance, identify trends and patterns,
and enable Real-time Analysis.

A stoplight report uses the colours of a traffic light (red, yellow,
green) to indicate the status or importance of the indicators or
metrics. The colours represent:

- **Green**: Indicates that the metric or indicator is on track,
 performing well, or meeting its target. It signals that no immediate
 action is required.
- **Yellow**: Suggests caution. The metric is somewhat off target, or
 there may be potential issues that need attention. It serves as a
 warning that while not critical, some action may be needed to ensure
 the target is met or to improve performance.
- **Red**: Signifies that the metric or indicator is off target,
 performing poorly, or in a critical state. It alerts users that
 immediate action is necessary to address issues or to get back
 on track.

<figure>
<img
src="https://cdn-images-1.medium.com/max/1000/0*lmV7qfDW92qVeSxj" />
<figcaption>Pipeline of using GPT-4V to generate stoplight.</figcaption>
</figure>

### Example: ARC-ROR Data Dashboard

The following screenshots are from the same dashboard on different
dates. It basically presents the status of the Australian research
council and registry. We will use these two screenshots as examples to
explain how to use LLM to generate stoplight reports for dashboards. We
assume that,

- Percentage of organisations that participated in ARC is very
 important, it is essential to monitor its change
- Number of project, Number of organisation, Number of organisation with
 ROR are less important
- Number of organisation with ROR, Number of organisations by country,
 it is no problem if the changes are not significant

Based upon on these assumption, we are able to define the traffic light:

- **Red**: Percentage of organisations that participated in ARC,
- **Yellow**: Number of project, Number of organisation, Number of
 organisation with ROR
- **Green**: Number of organisation with ROR, Number of organisations
 by country

<figure>
<img
src="https://cdn-images-1.medium.com/max/1000/0*fjIRxhkkeKR4UHa8" />
<figcaption>ARC-ROR 08-Jan-2024</figcaption>
</figure>

<figure>
<img
src="https://cdn-images-1.medium.com/max/1000/0*JkZ-cqF1Y1M10daF" />
<figcaption>ARC-ROR 13-Feb-2024</figcaption>
</figure>

#### Step 0: OpenAi API Preparation

We need OpenAi Api Key to access the GPT-4V.

 import base64
 import requests
 import json
 import configparser
 import os

 # OpenAI API Key
 current_path = os.path.dirname(os.path.realpath(__file__))
 config = configparser.ConfigParser()
 config.read(current_path + '/config.ini')
 api_key = config['DEFAULT']['api_key']

#### Step 1: Define the stoplights

We will assign specific colours to the indices, highlighting their
significance on the dashboards. We also establish thresholds for the
degree of change. If one index' s change is significant, the stoplight
colour of this index would be changed to another colour to indicate the
changes. what

 """
 Red:Percentage of organizations that participated in ARC,
 Yellow:Number of project, Number of organization, Number of organization with ROR
 Green:Number of organization with ROR, Number of organizations by country,

 Note that also take the significant change of the data into consideration,
 - If the change is under 1%, the stoplight should be green
 - If the change is between 1% and 5%, the stoplight should be yellow
 - If the change is over 5%, the stoplight should be red

 The priority of the stoplight is:
 Change > Definition of stoplight for the index

 That is to say, assume an index is defined as green, but it increases 8%(>5%), then the stoplight of this index should be red. 
 """

#### Step 2: Design the response template

We create a response template to provide the format and content range
for the response from chatGPT4-V.

 The response should strictly follow the following json template, also make sure the response is in json format.:
 {
 "Percentage of organisations that participated in ARC": {
 "First image": "20%",
 "Second image": "25%",
 "Stoplight": "red"
 },
 "Number of project": {
 "First image": "100",
 "Second image": "150",
 "Stoplight": "yellow"
 },
 "Number of organisation": {
 "First image": "50",
 "Second image": "75",
 "Stoplight": "yellow"
 },
 "Number of organisation with ROR": {
 "First image": "30",
 "Second image": "45",
 "Stoplight": "green"
 },
 "Number of organisations by country": [
 {
 "Country name": "Australia",
 "First image": "10%",
 "Second image": "12%",
 "Stoplight": "green"
 },
 {
 "Country name": "United States",
 "First image": "15%",
 "Second image": "18%",
 "Stoplight": "green"
 }
 ],
 }

#### Step 3: Encode the images

Base64 encoding ensures that your image data can be safely embedded into
text-based API requests, which is especially important for text-based
data exchange formats like JSON. Also, by directly sending image data,
it avoids security and privacy risks associated with image hosting
issues or URL access permissions.

 def encode_image(image_path):
 with open(image_path, "rb") as image_file:
 return base64.b64encode(image_file.read()).decode('utf-8')


 # Path to the images
 image_path = "./NL0008/nl0008-1.1.0.png"
 image_path_2 = "./NL0008/nl0008-1.0.8.png"
 # Getting the base64 string
 base64_image = encode_image(image_path)
 base64_image_2 = encode_image(image_path_2)

#### Step 4: Call GPT-4V

We pass the images of data dashboards, system prompt which includes
response template and the definition of stoplight, and user prompts to
OpenAI' s gpt-4-vision-preview model. Check
[here](https://platform.openai.com/docs/guides/vision) to know more
about
[gpt-4-vision-preview](https://platform.openai.com/docs/guides/vision).

 headers = {
 "Content-Type": "application/json",
 "Authorization": f"Bearer {api_key}"
 }

 payload = {
 "model": "gpt-4-vision-preview",
 "messages": [
 {
 "role": "system", "content": system_message,
 },
 {
 "role": "user",
 "content": [
 # system prompt
 {
 "type": "text",
 "text": "Can you help me compare the two images and tell me what's difference?"
 },
 {
 "type": "image_url",
 "image_url": {
 "url": f"data:image/jpeg;base64,{base64_image}"
 }
 },
 {
 "type": "image_url",
 "image_url": {
 "url": f"data:image/jpeg;base64,{base64_image_2}"
 }
 }
 ]
 }
 ],
 "max_tokens": 500,
 }
 response = requests.post("https://api.openai.com/v1/chat/completions", headers=headers, json=payload)

#### Output

OpenAI API gives the response following the design of the template and
colour of the stoplight is well assigned, where we can clearly see the
importance and the significant change of the metrics.

 {
 "Percentage of organisations that participated in ARC": {
 "First image": "20%",
 "Second image": "19%",
 "Stoplight": "green"
 },
 "Number of project": {
 "First image": "32238",
 "Second image": "30088",
 "Stoplight": "red"
 },
 "Number of organisation": {
 "First image": "14722",
 "Second image": "14152",
 "Stoplight": "yellow"
 },
 "Number of organisation with ROR": {
 "First image": "2950",
 "Second image": "2663",
 "Stoplight": "red"
 },
 "Number of organisations by country": [
 {
 "Country name": "Australia",
 "First image": "20.8%",
 "Second image": "30.0%",
 "Stoplight": "red"
 },
 {
 "Country name": "United States",
 "First image": "18.9%",
 "Second image": "19.5%",
 "Stoplight": "green"
 }
 ]
 }

We also convert data from JSON format to CSV format for enhanced
visualisation.

 import csv
 import json
 with open('data.json', 'r', encoding='utf-8') as json_file:
 data = json.load(json_file)
 if isinstance(data, str):
 data = json.loads(data)

 # csv writer
 with open('structured_data.csv', 'w', newline='', encoding='utf-8') as file:
 writer = csv.writer(file)

 # column name
 writer.writerow(['Category', 'First image', 'Second image', 'Stoplight'])
 for category, details in data.items():
 if isinstance(details, list):
 for detail in details:
 for key in detail:
 if key not in ['First image', 'Second image', 'Stoplight']:
 key_location = key
 writer.writerow([
 f"{category} - {detail[key_location]}",
 detail['First image'],
 detail['Second image'],
 detail['Stoplight']
 ])
 else:
 writer.writerow([
 category,
 details['First image'],
 details['Second image'],
 details['Stoplight']
 ])

<figure>
<img
src="https://cdn-images-1.medium.com/max/1000/0*YPYkqTRxnOpcSNQk" />
<figcaption>Table output for the information comparison.</figcaption>
</figure>

### Conclusion

GPT-4 Vision excels at interpreting and analysing data displayed in
visual formats, including graphs, charts, and various data
visualisations. It is highly effective in tracking and identifying
differences between two data dashboards, providing valuable insights
into changes and trends.

#### **Sources**

- <https://platform.openai.com/docs/guides/vision>
- <https://gptpluginz.com/gpt-4-vision-api/>

![](https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=f32c25ef8d62){width="1"
height="1"}

Efficient creation of a stoplight report with data dashboard images

Author: Yunzhong Zhang (ORCID: 0009–0002–8177–419X) Comparing data dashboards is crucial for understanding trends and performance differences. Traditionally, this task required manual effort, which was slow and sometimes inaccurate. Now, thanks to OpenAI’s GPT-4 with Vision (GPT-4V), we are able to automate and improve this process.

How to use GPT-4V For Stoplight Report

Australian National University

Amanda Kau

**Unlocking the Power of Questions --- A deep dive into Question
Answering Systems**

<figure>
<img
src="https://cdn-images-1.medium.com/max/1000/0*QP3N3IsXkAfiztnH" />
<figcaption>Source: <a
href="https://unsplash.com/photos/a-pile-of-letters-that-spell-out-the-word-war-MfBnqUOz_qY">https://unsplash.com/photos/a-pile-of-letters-that-spell-out-the-word-war-MfBnqUOz_qY</a></figcaption>
</figure>

### Author:

- **Amanda Kau (ORCID:**
 [**0009--0004--4949--9284**](https://orcid.org/0009-0004-4949-9284)**)**

Virtual assistants have popped up on numerous websites over the years.
These assistants utilise question answering (QA) systems, which are
gaining traction as we continue to automate response generation, thereby
improving QA applications like search engines and chatbots. As the name
implies, QA involves users posing queries to machines which in turn,
provide relevant, well-crafted answers. Active research aims to further
enhance these systems, with one notable avenue being the integration of
knowledge graphs which allow models to understand associations between
entities for added context.

This article will start off with a brief introduction to QA systems. It
will then discuss various subfields of QA, including conversational QA,
open-domain QA, multi-document QA and visual QA. Furthermore, solutions
that have been proposed to incorporate KGs into QA systems will be
reviewed.

### Question Answering Models

There exist many variants of QA models, but they generally fall into two
broad classifications:

1. **Extractive QA** models: These simply extract the relevant answer
 from the given context, which could comprise text, web pages, or
 other forms of data.
2. **Generative QA** models: These craft their own textual responses
 as outputs.

Generative QA models can be further categorised into closed and open
types, which draw upon implicit and explicit knowledge respectively.

In **closed generative QA**, the model generates answers based on the
knowledge it already possesses. For instance, if the generator is a
large language model (LLM), the system relies on information that the
LLM has already been trained on and has internalised into its parameters
to generate an output. This approach has limitations as the LLM can only
memorise a finite amount of data, so it might not be able to recall
supporting evidence for all potential questions. On top of that, the
facts stored within the LLM are frozen in time after training, rendering
the LLM incapable of utilising up-to-date information.

Meanwhile, in **open generative QA**, the model accesses supporting
information from external knowledge bases to aid in response generation.
This resembles extractive QA, but with an additional generative
component that allows the model to formulate its own responses using
facts from documents. External knowledge could take various forms such
as text documents, Wikipedia articles, or even knowledge graphs.

### A Brief Introduction to Knowledge Graphs

<figure>
<img src="https://cdn-images-1.medium.com/max/850/0*ZDv50a2YoMtjgj_y" />
<figcaption>Example of a knowledge graph. Source: RDF 1.1 Primer <a
href="https://www.w3.org/TR/rdf11-primer/">https://www.w3.org/TR/rdf11-primer/</a></figcaption>
</figure>

Knowledge graphs (KGs) are a data structure capable of representing
relationships between entities, as demonstrated in the example image
which depicts "Bob is interested in The Mona Lisa". Nodes (depicted as
circles) represent entities like people and objects, while edges
(depicted as arrows) represent the relationships between the pairs of
nodes. KGs allow for easier visualisation of relationships for analysis
and comparison. For example, it is much easier to understand the graph
example as compared to a lengthy description such as "Bob is a Person.
Bob is born on 14 July 1990. Bob is a friend of Alice..."

### Leveraging Knowledge Graphs for Question Answering

#### Conversational QA

Conversational QA involves answering questions whilst taking into
account the ongoing context of the conversation. Efforts in this field
have been driven by powerful language models like **BERT** and the
**GPT** series, which have been trained on massive amounts of data.
However, the challenge lies in ensuring QA systems can generalise to any
domain and are able to switch between conversational topics without
needing to be retrained for long periods of time on extensive datasets
for every new topic.

KGs have proven to be a promising alternative to machine learning-based
language models in the system called **Converse**, proposed by
Oduro-Afriyie and Jamil (2023). Converse follows a three-stage process:

1. It initiates with a text-to-KG module, where input text is parsed,
 broken down and subject to rule-based analysis. Breakdown of text in
 this way allows for generalisation to any domain, thus circumventing
 the retraining issue associated with machine learning-based models.

<figure>
<img
src="https://cdn-images-1.medium.com/max/1000/0*kT4No5pC6vFgtstS" />
<figcaption>KG constructed for the phrase “It was a beautiful orange
dress”. Source: Oduro-Afriyie &amp; Jamil, 2023. Link: <a
href="https://doi.org/10.1007/978-3-031-42935-4_6">https://doi.org/10.1007/978-3-031-42935-4_6</a></figcaption>
</figure>

2\. Next, the input query is processed in the second module to convert
it into a graph, although using slightly different techniques from the
first module due to the structural differences between questions and
sentences. The graph is subsequently converted into a **SPARQL** query
to extract information from the constructed KG in the first stage.

<figure>
<img src="https://cdn-images-1.medium.com/max/832/0*-3Bn_2UvFSBvwM6U" />
<figcaption>Graph constructed for the question “Who sold the dress that
Alice got?” Source: Oduro-Afriyie &amp; Jamil, 2023. Link: <a
href="https://doi.org/10.1007/978-3-031-42935-4_6">https://doi.org/10.1007/978-3-031-42935-4_6</a></figcaption>
</figure>

3\. Finally, the retrieved results from the SPARQL query are transformed
into a natural language response by arranging the entities (nodes) and
relationships (edges) into coherent sentences.

While this system is capable of answering simple questions, further
developments are required to enhance its performance. Nevertheless, it
provides some insight into the potential of knowledge graphs to execute
QA tasks without the unsustainable requirements for training data
and time.

#### Open-Domain QA

**Open-domain question answering** (**ODQA**) presents a slightly
different challenge as the model is provided with a query in any domain
with no other information. Despite the vast array of potential
questions, the model is expected to return informative answers backed by
facts. Hence, open generative QA models such as the
**Fusion-in-Decoder** (**FiD**) model are commonly employed to fulfil
these tasks. FiD has demonstrated good performance. Yet, two limitations
were noted by Yu et al. (2022) and addressed in their extended
model,** KG-FiD**.

Firstly, FiD failed to capture semantic relationships between text
passages, which are key for better answers. Yu and colleagues opted to
organise knowledge from passages into a KG, the data structure needed to
encapsulate relationships as edges.

Secondly, FiD had to comb through around 100 passages for each question,
resulting in high computational costs. Simply reducing the number of
passages was impractical as it compromised response generation. In
KG-FiD, a **Graph Neural Network** (GNN) was deployed to rank and filter
passages to improve response accuracy and circumvent the high
computational cost.

#### Multi-Document QA

Many works have been dedicated toward performing ODQA, but
**multi-document question answering** (**MD-QA**) has garnered
significantly less attention despite its numerous applications in
academic research, financial and legal inquiries, and more. MD-QA
involves finding relevant information from multiple data sources while
understanding logical associations amongst them. Adding to the
complexity of the problem, documents can contain different forms of
data, including text, figures and tables, all of which the model has to
be able to parse and understand.

<figure>
<img
src="https://cdn-images-1.medium.com/max/1000/0*49kZE3ST7GrAudM4" />
<figcaption>Common types of MD-QA questions. Source: Wang et al., 2023
<a
href="https://doi.org/10.48550/ARXIV.2308.11730">https://doi.org/10.48550/ARXIV.2308.11730</a></figcaption>
</figure>

The figure above illustrates some common MD-QA question types,
classified into bridging questions, comparing questions and structural
questions. Bridging questions require models to reason sequentially,
comparing questions require parallel reasoning across multiple passages,
and structural questions necessitate retrieving contents from various
document structures like specific pages and tables.

Once more, a KG was employed in the proposed **Knowledge Graph
Prompting** (**KGP**) method as it is aptly suited for representing
document structures and their relationships denoting lexical or semantic
similarity. A graph traversal agent based on large language models was
trained to correctly select the relevant document structures necessary
to answer the question.

#### Visual QA

The final field of QA discussed in this article is open-domain
knowledge-based **visual question answering** (**VQA**), whereby the
system is tasked to answer questions about an image with information
that is not present within the image itself. For example, the question
in the figure below, "*Which liquid here comes from a citrus fruit?*",
is presented with the image to the model without specifying which object
is being referred to.

<figure>
<img
src="https://cdn-images-1.medium.com/max/1000/0*14dwFPcpVW2L61Cf" />
<figcaption>Example of the process in KRISP for VQA. Source: Marino et
al., 2020 <a
href="https://doi.org/10.48550/arXiv.2012.11014">https://doi.org/10.48550/arXiv.2012.11014</a></figcaption>
</figure>

One approach known as **KRISP** (**Knowledge Reasoning with Implicit and
Symbolic rePresentations**) was proposed to achieve this task using a
combination of implicit and explicit (or symbolic) knowledge. This is
because it was observed that neither type of knowledge alone was
sufficient. The implicit knowledge stemmed from internalised information
within their multimodal transformer model pre-trained with **BERT**.
Concurrently, explicit knowledge was provided from a KG, which contained
the entities recognised from the image, including places, objects,
attributes and more. Four types of knowledge (trivia, commonsense,
scientific and situational knowledge) from various external sources were
also encompassed in the KG. Implicit and symbolic answer predictions
were made, and the higher scoring answer was selected as the final
output. This process outperformed other methods in VQA and was able to
provide rare responses, hence exhibiting KRISP's generalisability.

### Conclusion

A number of concepts have been discussed in this article, including the
use of Knowledge Graphs in various areas of QA, including conversational
QA, Open-Domain QA, multi-document QA and Visual QA. It is evident that
Knowledge Graphs are a good alternative to presenting and retrieving
data for the task of QA, especially due to their flexibility in
representing any type of object or concept as nodes, and their ability
to capture relationships between nodes. Most importantly, these
relationships are what enhance the performance of retrieval techniques
and enable models to generate more accurate and relevant responses.

#### **Sources**

- Oduro-Afriyie, J., & Jamil, H. (2023). Knowledge Graph Enabled
 Open-Domain Conversational Question Answering. In Flexible Query
 Answering Systems (pp. 63--76). Springer Nature Switzerland.
 <https://doi.org/10.1007/978-3-031-42935-4_6>
- Yu, D., Zhu, C., Fang, Y., Yu, W., Wang, S., Xu, Y., Ren, X., Yang,
 Y., & Zeng, M. (2021). KG-FiD: Infusing Knowledge Graph in
 Fusion-in-Decoder for Open-Domain Question Answering (Version 2).
 arXiv. <https://doi.org/10.48550/ARXIV.2110.04330>
- Wang, Y., Lipka, N., Rossi, R. A., Siu, A., Zhang, R., & Derr, T.
 (2023). Knowledge Graph Prompting for Multi-Document Question
 Answering (Version 3). arXiv.
 <https://doi.org/10.48550/ARXIV.2308.11730>
- Marino, K., Chen, X., Parikh, D., Gupta, A., & Rohrbach, M. (2020).
 KRISP: Integrating Implicit and Symbolic Knowledge for Open-Domain
 Knowledge-Based VQA (Version 1). arXiv.
 <https://doi.org/10.48550/ARXIV.2012.11014>

![](https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=a9d0cd1083d1){width="1"
height="1"}

Unlocking the Power of Questions — A deep dive into Question Answering Systems

Author:

 Amanda Kau (ORCID:


 0009–0004–4949–9284


 )

Virtual assistants have popped up on numerous websites over the years.

Harnessing The Power of Knowledge Graphs in Question Answering

Swinburne University of Technology

Amir Aryani

<figure>
<img
src="https://cdn-images-1.medium.com/max/1024/1*Ajc2vnoQPhCUt2m5FvkfyQ.png" />
</figure>

I asked Generative AI Models about their context window. Their response
was intriguing.

The context window for a large language model (LLM) like OpenAI's GPT
refers to the maximum amount of text the model can consider at any one
time when generating a response. This includes both the prompt provided
by the user and the model's generated text.

In practical terms, the context window limits how much previous dialogue
the model can "remember" during an interaction. If the interaction
exceeds the context window, the model loses access to the earliest parts
of the conversation. This limitation can impact the model's consistency
in long conversations or complex tasks.

> Peering into the Context Window of a Large Language Model is like
> unlocking a secret chamber in the realm of AI: it's the pivotal factor
> that transforms these digital titans from mere code into practical,
> powerful tools.

The question is what LLM models have to say about their context window.
Let\'s talk to each one and see the response. For each one we
consistently sent the same prompt:

***Prompt: What is the size of your Context Window (amount of text you
can consider in a single response)?***

### Google Bards

It seems Google likes to keep the Bard context window a secret. Despite
multiple attempts, I failed to get Bard to tell me its context window.

 BARD Response:

 Unfortunately, I am currently unable to share the exact size of my
 context window. This information is considered confidential by Google
 AI. However, I can assure you that my context window is large enough
 to incorporate a substantial amount of information and provide you
 with comprehensive and informative responses.

 The size of my context window is constantly being evaluated and
 improved upon. As I continue to learn and grow, my capacity to
 process and retain information will increase, allowing me to provide
 even more accurate and helpful responses.

 While I cannot share the specific size, I can tell you that my context
 window is significantly larger than those of most other language
 models. This allows me to access and process a much broader range of
 information, which in turn enables me to generate more creative and
 insightful text formats.

 I hope this explanation is helpful. If you have any further questions,
 please do not hesitate to ask.

### OpenAI ChatGPT 3.5 {#openai-chatgpt-3.5}

It correctly identifies a 4K Context Window.

 ChatGPT 3.5 Response: 

 I can consider a context window of up to 4096 tokens in a single
 response. This means that the text I generate in response to a query
 is based on the previous 4096 tokens of text in the conversation. If
 the conversation exceeds this limit, older parts of the conversation
 may be truncated, which can result in a loss of context. It's
 important to keep the conversation within this token limit to ensure
 coherent and contextually relevant responses.

### ChatGPT 4

It seems ChatGPT 4 has problems identifying the right context window. In
the first attempt, it claims a 2K context window, and in the second
attempt, it reports on a 4K context window.

 ChatGPT 4 first response: 

 The size of my context window, or the amount of text I can consider in
 a single response, is approximately 2048 tokens. A token can be as
 short as one character or as long as one word. This allows me to
 process and respond to fairly large chunks of text, although very
 long documents may exceed this limit and require summarization or
 segmentation for effective analysis and response.


 ChatGPT 4 second response (regenerated):

 The size of my context window, or the amount of text I can consider in
 a single response, is approximately 4096 tokens. This includes both
 the user's input and my own responses. A token can be as small as a
 single character or as large as a word, depending on the complexity
 of the language and syntax. For reference, this typically equates to
 a few pages of text. This limit ensures that while I can engage in
 detailed and extensive conversations, I may need reminders or
 reiterations of previous points in longer discussions.

One parent outcome of this experiment is that at the time of this test
(Dec 2023), the ChatGPT 4 does not operate based on the new 128k (GPT
Turbo) model.

### Open Source Models

Finally, I tested two very popular Open Source models: Mistral AI
and Llama2.

**Mistral AI (7B):**

Mistral failed to identify the token size in the first attempt, so I had
to extend the prompt by giving it a more precise instruction. However,
based on the recent publication (<https://arxiv.org/abs/2310.06825>) by
the Mistral AI team, this model uses a sliding 4K window.

 Intial prompt: What is the size of your Context Window (amount of
 text you can consider in a single response)?

 Mistral AI response:
 As a 7B model, my context window is very large and can consider an
 enormous amount of text in a single response. However, the specific
 size of the context that I use for a particular task will depend on
 the requirements of that task and the amount of data available to me.
 In general, I am able to consider a wide range of textual input and
 incorporate it into my responses in a coherent and meaningful way.

 ---

 Extended prompt: What is the size of your Context Window (amount of
 text you can consider in a single response)? Provide the exact context
 size in tokens.

 Mistral AI response:
 The size of my Context Window depends on the task and input I'm
 processing. For most tasks, it can handle up to 1024 tokens of text.

**LLAMA2 (7B):**

Llama 2 was quite consistent in multiple rounds even with the variations
of the prompt. It refused to provide an exact token size context window.

 Llama 2 (7B) response:

 I'm just an AI, I don't have a fixed context window size in tokens. My
 ability to understand and respond to text is based on the complexity
 and coherence of the input text, rather than a predetermined size.

Overall it seems LLM models have limited capabilities in providing
information about their technical specifications. This can be an issue
related to the limitation of the training data, or particular
alignments.

![](https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=772b1c65b881){width="1"
height="1"}

I asked Generative AI Models about their context window. Their response was intriguing. The context window for a large language model (LLM) like OpenAI’s GPT refers to the maximum amount of text the model can consider at any one time when generating a response. This includes both the prompt provided by the user and the model’s generated text.

Messages de Rogue Scholar

How to use GPT-4V For Stoplight Report

Harnessing The Power of Knowledge Graphs in Question Answering

What is your Context Window?