Australian National University

Vaibhav Khobragade

#### **From Naive to Modular: Tracing the Evolution of Retrieval-Augmented Generation**

<figure>
<img src="https://cdn-images-1.medium.com/max/938/0*IQgdTaKG50F_nbGM" />
<figcaption>Created by DALLE</figcaption>
</figure>

### Author

· Vaibhav Khobragade (**ORCID:**
[0009--0009--8807--5982](https://orcid.org/0009-0009-8807-5982))

### Introduction

Large Language Models (LLMs) have achieved remarkable success. But, they
still face significant limitations, especially in domain-specific or
knowledge-intensive tasks such as question answering, producing
"hallucinations" where the models generate responses that sound
plausible but are actually incorrect when handling queries beyond their
training data or requiring current information.

To overcome these challenges, [Retrieval-Augmented Generation
(RAG)](https://medium.com/@researchgraph/refining-ai-vision-how-retrieval-augmented-generation-transforms-image-captioning-in-large-1a74499a370d)
enhances LLMs by retrieving relevant document chunks from external
knowledge bases through semantic similarity calculation. By referencing
external knowledge, RAG effectively reduces the problem of generating
factually incorrect content. Its integration into LLMs has resulted in
widespread adoption, establishing RAG as a key technology not just in
advancing chatbots, but also in vision language models to support
generating responses for real-world applications.

RAG instance when applied to question-answering tasks follows
this process:

<figure>
<img src="https://cdn-images-1.medium.com/max/940/0*rbHdvUc4YFFKs0SD" />
<figcaption>Figure 1: An example of how the Retrieval-Augmented
Generation (RAG) process is used for answering questions. (Gao et al.,
2023) Article link: <a
href="https://arxiv.org/abs/2312.10997">https://arxiv.org/abs/2312.10997</a></figcaption>
</figure>

The diagram consists of three main steps. First, documents are divided
into smaller parts, converted into vectors, and stored in a database.
Second, the system selects the top k most relevant chunks based on the
question's meaning. Third, the original question and selected chunks are
inputted into a large language model to produce the answer.

<figure>
<img
src="https://cdn-images-1.medium.com/max/1016/0*uEXFltuz-jl29heb" />
<figcaption>Figure 2. Technology tree of RAG research. The stages of
involving RAG mainly include pre-training, fine-tuning, and inference.
(Gao et al., 2023) Article link: <a
href="https://arxiv.org/abs/2312.10997">https://arxiv.org/abs/2312.10997</a></figcaption>
</figure>

RAG technology has rapidly developed in recent years, and the technology
tree summarizing related research is shown above in Figure 2. Initially,
RAGs emerged with Transformer architecture, focusing on enhancing
language models by incorporating additional knowledge through
PreTraining Models (PTM). This early stage was characterized by
foundational work aimed at refining pre-training techniques. The
subsequent arrival of ChatGPT marked a pivotal moment, with LLMs
demonstrating powerful in-context learning (ICL) capabilities. As
research progressed, the enhancement of RAG was no longer limited to the
inference stage but began to be incorporated more with LLM fine-tuning
techniques.

In this article, we review the state-of-the-art RAG method and describe
its evolution through paradigms including Naive RAG, Advanced RAG, and
Modular RAG.

### **Naive RAG**

This is the earliest method of RAG to get prominence shortly after the
advent of ChatGPT. The Naive RAG follows a traditional process that
includes indexing, retrieval, and generation.

Let's understand Naïve RAG with an example.

**Scenario:** Suppose a user asks an LLM, "What are the health benefits
of drinking green tea?"

**Without Naive RAG:** The LLM might generate a response based solely on
its pre-trained knowledge, which could be outdated or incomplete. It
might say, "Green tea is rich in antioxidants and can help in improving
brain function."

**With Naive RAG:**

1\. **Indexing:** It begins by cleaning and extracting raw data about
green tea from various formats like PDF, HTML, Word, and Markdown. This
data is converted into plain text for consistency. To address the
context limitations of language models, the text is divided into
smaller, manageable chunks. These chunks are then turned into vectors
using an embedding model and stored in a vector database. This step is
important for enabling quick and accurate searches in the retrieval
phase and providing up-to-date research and ***articles about green tea
are indexed.***

2\. **Retrieval:** When the RAG system gets a user question, it uses the
same encoding model as before to turn the question into a vector. Then,
it compares this vector to the vectors of the indexed text chunks. The
system picks the top K chunks that are most similar to the question.
These chunks become the extra context for generating the response.
***For example: It retrieves closely related chunks to "health benefits
of green tea."***

3\. **Generation:** The user question and chosen documents are combined
into a clear prompt. The model might answer differently based on
specific task rules. It can use its built-in knowledge or stick only to
the given documents. If it's part of a conversation, previous chat
history can be added to the prompt, helping the model engage better in
multi-turn dialogues. ***For instance, it might retrieve a recent study
stating, "Green tea consumption is linked to reduced risk of heart
disease," which wasn't part of its initial training data.***

<figure>
<img src="https://cdn-images-1.medium.com/max/346/0*uvgeymdjpJec0jk3" />
<figcaption>Figure 3 Naïve RAG Pipeline (Gao et al., 2023) Article link:
<a
href="https://arxiv.org/abs/2312.10997">https://arxiv.org/abs/2312.10997</a></figcaption>
</figure>

However, Naive RAG encounters notable drawbacks:

**Retrieval Challenges --- **that lead to the selection of misaligned or
irrelevant chunks, and the missing of crucial information.

**Generating Challenges --- **the model might struggle with
hallucination and have problems with relevance, toxicity, or bias in its
outputs, which can make the responses less reliable and of lower quality

For instance: The Naive RAG model retrieves relevant documents and
provides them to the LLM. However, the retrieved information may not be
relevant or up to date. ***"Recent studies suggest that new medications
and therapies are being developed to slow the progression of
Alzheimer's."***

### **Advanced RAG**

Advanced RAG introduces specific improvements to overcome the
limitations of Naive RAG. Focusing on enhancing retrieval quality, it
employs pre-retrieval and post-retrieval strategies.

<figure>
<img src="https://cdn-images-1.medium.com/max/375/0*1PrMV9yHEE1eJG0b" />
<figcaption>Figure 4. Advanced RAG Pipeline</figcaption>
</figure>

**Scenario**: Consider a user asking, "What recent advances have been
made in the treatment of Alzheimer's disease?"

**With Advanced RAG:** The following procedures are followed:

**Pre-retrieval process:** The main aim is to improve the indexing
structure and the user's query. Indexing optimization focuses on making
the indexed content better. This includes strategies like improving data
details, organizing indexes better, adding extra information, aligning
things correctly, and using mixed retrieval methods, where information
from external sources is combined with the indexed content for retrieval
purposes. Query optimization aims to make the user's original query
clearer and more suitable for retrieval. Common methods include query
rewriting and query transformation & expansion.

For instance: Using the Alzheimer's example, Advanced RAG use a refined
indexing strategy where documents are not only indexed by content but
also by recency and relevance to medical advancements, using metadata
enhancements. Utilizing enhanced query transformation techniques, the
system precisely retrieves the most current and relevant research papers
discussing the latest treatment breakthroughs, rather than broader
articles on Alzheimer's.

**Post-Retrieval Process:** After retrieving relevant information, it is
essential to effectively combine it with the original query. The main
steps include re-ranking to highlight the most important content and
compressing the context to avoid information overload as retrieved
documents are then re-ranked to prioritize those that specifically
mention recent clinical trials or FDA approvals, ensuring that the most
pertinent information is considered first.

**Context compression:** Redundant or less relevant information is
filtered out to focus on key findings from the most authoritative
sources.

So, the result could be: "The latest breakthrough in Alzheimer's
treatment includes the development of a new drug, Aducanumab, which has
shown potential in reducing cognitive decline in early-stage patients,
according to a 2021 FDA report."

### **Modular RAG**

The modular RAG architecture advances beyond the former two RAG
paradigms, offering enhanced adaptability and versatility. It
incorporates diverse strategies for improving its components, such as
adding a **search module** for similarity searches and refining the
retriever through **fine-tuning**. Modular RAG builds upon the
foundational principles of Advanced and Naive RAG, illustrating a
progression and refinement within the RAG family.

<figure>
<img src="https://cdn-images-1.medium.com/max/701/0*JBAhcxxaIuT8DI0m" />
<figcaption>Figure 5 Modular RAG Pipeline</figcaption>
</figure>

**Modules:** The Modular RAG framework introduces additional specialized
components to enhance retrieval and processing capabilities.

Consider the scenario: ***"What are the latest research findings on the
health benefits of green tea?"***

1. **Search Module:** When the question is posed, this module activates
 and uses LLM-generated queries to search through various databases
 and knowledge graphs specifically for the latest research papers or
 articles on green tea.
2. **RAG-Fusion:** This module takes the initial user query and expands
 it into multiple queries. For instance, it might generate additional
 queries like "recent clinical trials on green tea" and "green tea
 effects on mental health." These expanded queries are then used to
 search for a wide range of relevant information.
3. **Memory Module:** As articles and papers are retrieved, this module
 helps by recalling previously retrieved documents on similar topics
 to improve the relevance and depth of the current search, using the
 system's built-in memory of past queries and their results.
4. **Routing:** This module analyzes the incoming data and decides the
 best path for processing it. For example, it might route the query
 to specific scientific databases for academic papers rather than
 general websites.
5. **Predict Module:** To ensure the retrieved information isn't
 redundant and is relevant, this module generates a synthesized
 context from the retrieved data, focusing only on the most pertinent
 information like key findings or new insights about green tea.
6. **Task Adapter Module:** Tailors RAG to specific tasks by automating
 prompt retrieval for zero-shot inputs and creating task-specific
 retrievers. For example, this module recognizes that the query is
 about health benefits, a specific knowledge domain. It adjusts the
 retrieval process to focus on the most credible medical and
 health-oriented sources and might generate specialized prompts to
 refine further retrieval.

#### **Patterns**

RAG uses various flexible and interchangeable modules to enhance the
performance of LLMs as follows:

1. **Module Interaction and Adjustment:** These are methods in Modular
 RAG that show how different parts (modules) of the system work
 together more dynamically. DSP (Demonstrate-Search-Predict) and
 ITERRETGEN (Iterative Retrieve-Read-Retrieve-Read) use the outputs
 from one module to improve or inform the function of another,
 enhancing the overall synergy and effectiveness. Example Patterns:
 [**DSP**](https://doi.org/10.48550/arXiv.2212.14024) and
 [**ITERRETGEN**](https://doi.org/10.48550/arXiv.2305.15294)
2. **Flexible Orchestration:** These methods demonstrate the
 adaptability of Modular RAG, where the system can adjust its
 retrieval processes based on what's needed for a specific scenario.
 This flexibility means the system isn't stuck to a fixed way of
 retrieving information but can adapt based on the requirements of
 the task at hand. Example Techniques**:**
 [**FLARE**](https://arxiv.org/pdf/2312.10997)
 and [**Self-RAG**](https://doi.org/10.48550/arXiv.2310.11511)
3. **Integration with Other Technologies:** Modular RAG can also easily
 work with other advanced technologies. For example, it can improve
 retrieval accuracy by fine-tuning the retrieval part of the system,
 personalize outputs by fine-tuning the generation process or even
 use reinforcement learning to enhance both processes
 collaboratively. Examples are **Fine-Tuning** and **Reinforcement
 Learning:**

### **Comparison of RAG with other model optimization methods**

<figure>
<img
src="https://cdn-images-1.medium.com/max/1022/0*vkYF69LpphrOyfhC" />
<figcaption>Figure 6 Compares Retrieval-Augmented Generation (RAG) with
other model optimization methods, specifically Fine-tuning and Prompt
Engineering.</figcaption>
</figure>

The diagram, presented as a quadrant chart, illustrates the differences
based on two dimensions: "External Knowledge Required" and "Model
Adaptation Required."

1. **Prompt Engineering**: Requires minimal model modifications and
 external knowledge, leveraging the inherent capabilities of LLMs.
2. **Fine-tuning (FT)**: Involves further training of the model to
 internalize knowledge, making it suitable for tasks that require
 specific structures, styles, or formats.
3. **RAG**: RAG typically requires less training of the model itself
 compared to methods like Fine-tuning. Instead, RAG heavily utilizes
 external, updated knowledge sources to enhance the model's output.
 This allows RAG to provide more accurate and current information by
 integrating the latest data from outside databases and texts,
 without the need for extensive retraining of the model.

### **Future Works**

The key future developments highlighted for RAG are as follows:

**Optimization of Retrieval Efficiency:** Future research can focus on
ways of accelerating the retrieval process more effectively for large
knowledge bases. Optimization of indexing and retrieval of data can be
done to reduce latency and increase the responsiveness to the maximum.

**Multi-Modal RAG:** Take RAG to the next level by making it work with
not only text but other modalities like images, audio, and video. Plenty
of innovations have come through for Multi-Modal RAG already, but more
research in this area will always be helpful.

**Operational and Production Efficiency:** Demonstrate and enhance the
practical deployment of RAG systems within the production environment
with improved management of computational resources, and data security
assured in commercial applications.

### **Conclusion**

This article discusses RAG, a method that enhances LLMs by integrating
external knowledge to improve response accuracy, especially for complex,
real-world applications. RAG reduces factual inaccuracies by accessing
up-to-date databases, supporting various RAG paradigms like Naive,
Advanced, and Modular RAG, each offering specific improvements and
flexibility for task-specific applications.

### **References**

- Gao, Y., Xiong, Y., Gao, X., Jia, K., Pan, J., Bi, Y., Dai, Y., Sun,
 J., Wang, M., & Wang, H. (2023, December 18). *Retrieval-Augmented
 Generation for Large Language Models: A Survey*. arXiv.org.
 <https://arxiv.org/abs/2312.10997>
- Peng, W., Li, G., Jiang, Y., Wang, Z., Ou, D., Zeng, X., Xu, D., Xu,
 T., & Chen, E. (2023, November 7). *Large Language Model based
 Long-tail Query Rewriting in Taobao Search*. arXiv.org.
 <https://arxiv.org/abs/2311.03758>
- Ma, X., Gong, Y., He, P., Zhao, H., & Duan, N. (2023, May 23). *Query
 Rewriting for Retrieval-Augmented Large Language Models*. arXiv.org.
 <https://arxiv.org/abs/2305.14283>

![](https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=186de3e6e354){width="1"
height="1"}

From Naive to Modular: Tracing the Evolution of Retrieval-Augmented Generation

Author · Vaibhav Khobragade (

 ORCID:

0009–0009–8807–5982) Introduction Large Language Models (LLMs) have achieved remarkable success.

Three Paradigms of RAG

Sebastian Achilles

Andreas Herten

1. [MSA Concept](#msa-concept){#markdown-toc-msa-concept}
2. [MSA Software Building
 Blocks](#msa-software-building-blocks){#markdown-toc-msa-software-building-blocks}
3. [Workshop](#workshop){#markdown-toc-workshop}
 1. [Exercises](#exercises){#markdown-toc-exercises}
 1. [1: Hello
 World!](#1-hello-world){#markdown-toc-1-hello-world}
 2. [2. GPU Hello
 World!](#2-gpu-hello-world){#markdown-toc-2-gpu-hello-world}
 3. [3: CPU-GPU Ping
 Pong](#3-cpu-gpu-ping-pong){#markdown-toc-3-cpu-gpu-ping-pong}
 2. [Slides](#slides){#markdown-toc-slides}

On May 29, we held a workshop about using the Modular Supercomputing
Architecture (MSA) together with project partners from ParTec. The
audience were collaborators from two projects funded in the same
[SCALEXA funding
call](https://www.bmbf.de/bmbf/shareddocs/bekanntmachungen/de/2021/08/2021-08-26-Bekanntmachung-SCALEXA.html)
by BMBF: [MExMeMo](https://gauss-allianz.de/de/project/title/MExMeMo)
and [IF(CES)^2^](https://gauss-allianz.de/de/project/title/IFCES2).

## MSA Concept

[![Slide from MSA Workshop slidedeck showing JSC\'s Supecomputer
strategy (page
6)](/assets/img/posts/2023-06-28-scal-msa-workshop/slides-p6.tn.png "JSC's supercomputer history; link to PDF page"){srcset="/assets/img/posts/2023-06-28-scal-msa-workshop/slides-p6.tn.png 1x, /assets/img/posts/2023-06-28-scal-msa-workshop/slides-p6.tn.2x.png 2x, /assets/img/posts/2023-06-28-scal-msa-workshop/slides-p6.tn.4x.png 4x"}](/assets/slides/posts/2023-06-28-scal-msa-workshop/aherten-sachilles--scalexa-msa-workshop-slides.pdf#page=6)While
CPU-based system are versatile and can be used for various workloads,
accelerated systems (think GPUs, FPGAs, AI chips) can be quite
efficient, but only suited for a sub-set of workloads. The MSA concept
foresees a combination of HPC systems with distinct hardware (*modules*)
into one combined *super*-system, usable for execution by a program.
That enables heterogeneous workloads, in which parts map to the
best-suited modules. The MSA concept was developed to production-grade
in the [EU-funded DEEP projects](https://www.deep-projects.eu/), of
which the [DEEP
system](https://www.fz-juelich.de/de/ias/jsc/systeme/prototypsysteme/deep-system)
is still being used to develop further features. By now, also many
production systems are built-up in an MSA fashion, like
[JURECA](https://www.fz-juelich.de/en/ias/jsc/systems/supercomputers/jureca)
and
[JUWELS](https://www.fz-juelich.de/en/ias/jsc/systems/supercomputers/juwels)
at JSC, but also for example [Leonardo at
CINECA](https://leonardo-supercomputer.cineca.eu/).

[![Slide from MSA Workshop slidedeck showing, amongst other things, the
JUWELS network
topology.](/assets/img/posts/2023-06-28-scal-msa-workshop/slides-p8.tn.png "JUWELS Modular Systems; link to PDF page"){srcset="/assets/img/posts/2023-06-28-scal-msa-workshop/slides-p8.tn.png 1x, /assets/img/posts/2023-06-28-scal-msa-workshop/slides-p8.tn.2x.png 2x, /assets/img/posts/2023-06-28-scal-msa-workshop/slides-p8.tn.4x.png 4x"}](/assets/slides/posts/2023-06-28-scal-msa-workshop/aherten-sachilles--scalexa-msa-workshop-slides.pdf#page=28)In
MExMeMo, the project we are involved in, we want to use MSA for a
heterogeneous multi-scale physics simulation: two application
cooperatively simulate a soft matter process -- one takes over the large
scales using CPUs, and the [other
simulates](https://gitlab.com/InnocentBug/SOMA) certain parts very
detailed and verifies the large-scale model on GPUs.

## MSA Software Building Blocks

[![Slide from MSA Workshop slidedeck with an example srun
call.](/assets/img/posts/2023-06-28-scal-msa-workshop/slides-p27.tn.png "An example heterogeneous job; link to PDF page"){srcset="/assets/img/posts/2023-06-28-scal-msa-workshop/slides-p27.tn.png 1x, /assets/img/posts/2023-06-28-scal-msa-workshop/slides-p27.tn.2x.png 2x, /assets/img/posts/2023-06-28-scal-msa-workshop/slides-p27.tn.4x.png 4x"}](/assets/slides/posts/2023-06-28-scal-msa-workshop/aherten-sachilles--scalexa-msa-workshop-slides.pdf#page=27)From
a software perspective, MSA utilizes many pre-existing components. We
usually use the heterogeneous job features of Slurm to launch HPC jobs
crossing module boundaries; we use MPI to communicate with the
components of the job. In addition, it can be useful to employ tools
like [`xenv`{.language-plaintext
.highlighter-rouge}](https://apps.fz-juelich.de/jsc/hps/jureca/modular-jobs.html),
[`env`{.language-plaintext
.highlighter-rouge}](https://man7.org/linux/man-pages/man1/env.1.html),
or even [`renv`{.language-plaintext
.highlighter-rouge}](https://github.com/FZJ-JSC/renv) to manipulate the
module-specific environments in an *ad-hoc fashion*. Through the setup,
application parts or even entirely disparate application, can be
executed on the respective modules and exchange messages with each
other, solving a scientific problem collaboratively. Advanced MSA
features are implemented in
[ParaStationMPI](https://github.com/ParaStation/psmpi), like MPI
collectives which take module affinity under consideration.

## Workshop

Just like this blogpost, the workshop started with a theoretical
introduction to the topic. In slides, ParTec shared details about the
MSA concept and usage with Slurm ^[1](#fn:partecslides){.footnote
rel="footnote"}^. Following that, we took over to present our MSA setup
at JSC -- globally and the specific software environment. Find the
slides embedded at the end!

The second part of the workshop were actual hands-on exercises in *fill
in the blank* style. Augmented with `TODO`{.language-plaintext
.highlighter-rouge} comments and descriptions in readmes, the
participants were steered to solving MSA software challenges of
increasing complexity.

### Exercises

We have [released the exercises material on
Zenodo](https://doi.org/10.5281/zenodo.8083130), but you can also find
it in [our JSC Gitlab
repository](https://gitlab.jsc.fz-juelich.de/herten1/msa-hello-world).
In the following, a quick run through the three exercises.

#### 1: Hello World!

The [first
exercises](https://gitlab.jsc.fz-juelich.de/herten1/msa-hello-world/-/tree/main/01-MSA-hello-world)
is a simple MPI *hello world* program which prints the hostname of the
node it is executed on. See the following (incomplete) listing for a
sketch:

::: {.language-c++ .highlighter-rouge}
::: {.highlight}
``` highlight
MPI_Init(NULL, NULL);
MPI_Comm_size(MPI_COMM_WORLD, &world_size);
MPI_Get_processor_name(processor_name, &name_len);
printf("Hello world from processor %s, rank %d out of %d processors\n",
 processor_name, world_rank, world_size);
```
:::
:::

The source code is not different from a standard MPI hello world.
However, this task should teach how to launch an MPI application to two
modules. That is why the focus is on the job script. Using [Slurm's
heterogeneous job
support](https://slurm.schedmd.com/heterogeneous_jobs.html) enables
specifying requirements for each component of a job using the
`:`{.language-plaintext .highlighter-rouge} (*colon*) syntax. In
addition, we are using `xenv`{.language-plaintext .highlighter-rouge} to
load required modules, which might be different for the individual
components.

An example batch jobs definition reads:

::: {.language-bash .highlighter-rouge}
::: {.highlight}
``` highlight
#!/bin/bash -x
#SBATCH --nodes=1
#SBATCH --ntasks-per-node=1
#SBATCH --partition=cpu
#SBATCH hetjob
#SBATCH --nodes=1
#SBATCH --ntasks-per-node=1
#SBATCH --partition=gpu

srun \
 xenv -P -L GCC -L ParaStationMPI ./mpi_hello_world.cpu.out : \
 xenv -P -L GCC -L ParaStationMPI ./mpi_hello_world.gpu.out
```
:::
:::

Executing the script with the example application on the Cluster and
Booster modules of JUWELS, the two different machines can be easily
identified:

::: {.language-bash .highlighter-rouge}
::: {.highlight}
``` highlight
$ cat slurm-out.7577116 
Hello world from processor jwc00n024.juwels, rank 0 out of 2 processors
Hello world from processor jwb0001.juwels, rank 1 out of 2 processors
```
:::
:::

Of course we are not limited to only two job components in MSA, but we
could add as many as needed by the modular application!

Now that we have learned how we can launch an MSA job, we can continue
and sprinkle a little GPU on top.

#### 2. GPU Hello World!

In the [second
exercise](https://gitlab.jsc.fz-juelich.de/herten1/msa-hello-world/-/tree/main/02-MSA-hello-world-gpu),
a message is sent from a CPU node directly to a GPU buffer on a
GPU-equipped node:

::: {.language-c++ .highlighter-rouge}
::: {.highlight}
``` highlight
cudaMalloc((void**)&d_payload, 6);
cudaMemcpy(d_payload, payload, 6, cudaMemcpyHostToDevice);

hello<<<1, 1>>>(d_payload); // contains a `printf()`

if (rank == 1)
 MPI_Recv(d_payload, 6, MPI_CHAR, 0, tag, MPI_COMM_WORLD, &stat);

hello<<<1, 1>>>(d_payload);
```
:::
:::

The job script from task 1 needs to be slightly extended by using
GPU-specific modules for the second job component:

::: {.language-bash .highlighter-rouge}
::: {.highlight}
``` highlight
srun \
 xenv -P -L GCC -L ParaStationMPI ./hello.cpu.out : \
 xenv -P -L GCC -L ParaStationMPI -L MPI-settings/CUDA ./hello.gpu.out
```
:::
:::

The out is then showing the combined message from both parts:

::: {.language-bash .highlighter-rouge}
::: {.highlight}
``` highlight
$ cat slurm-out.7916615 
hello world!
```
:::
:::

#### 3: CPU-GPU Ping Pong

Finally, the [third
exercise](https://gitlab.jsc.fz-juelich.de/herten1/msa-hello-world/-/tree/main/03-MSA-ping-pong)
not only sends a single "world" to a GPU buffer, but a continuous
exchange between CPU buffer and GPU buffer is implemented across the
module boundaries (i.e. JUWELS Cluster ↔ JUWELS Booster). The message
size is increased step-wise and the bandwidth printed:

::: {.language-bash .highlighter-rouge}
::: {.highlight}
``` highlight
Transfer size (B): 8, Time (s): 0.000002580, Bandwidth (GB/s): 0.002888259
Transfer size (B): 16, Time (s): 0.000002594, Bandwidth (GB/s): 0.005745580
...
Transfer size (B): 1073741824, Time (s): 0.086975408, Bandwidth (GB/s): 11.497502804
```
:::
:::

### Slides

The slides were presented at the beginning of the workshop and were
[accompanied by slides from
ParTec](/assets/slides/posts/2023-06-28-scal-msa-workshop/partec-msa-workshop-mexmemo_ifces2.pdf)
focusing on the perspective from a *MSA solutions provider*; the ParTec
team introduced the benefits of the MSA concepts and details about the
MSA features of ParaStationMPI.

[Download our
slides](/assets/slides/posts/2023-06-28-scal-msa-workshop/aherten-sachilles--scalexa-msa-workshop-slides.pdf)
or browse through them here:

Your browser is unable to display the PDF. Instead, please [download
it](/assets/slides/posts/2023-06-28-scal-msa-workshop/aherten-sachilles--scalexa-msa-workshop-slides.pdf).

::: {.footnotes role="doc-endnotes"}
1. ::: {#fn:partecslides}
 ParTec allowed us to upload the slides as well, please [find them
 here](/assets/slides/posts/2023-06-28-scal-msa-workshop/partec-msa-workshop-mexmemo_ifces2.pdf). [↩](#fnref:partecslides){.reversefootnote
 role="doc-backlink"}
 :::
:::

MSA Concept MSA Software Building Blocks Workshop Exercises 1: Hello World! 2. GPU Hello World! 3: CPU-GPU Ping Pong Slides  On May 29, we held a workshop about using the Modular Supercomputing Architecture (MSA) together with project partners from ParTec.

Messaggi di Rogue Scholar

Three Paradigms of RAG

MSA Introduction Workshop