Andreas Herten

[![](/assets/img/posts/2024-06-12-p3hpc-compat/table.png){srcset="/assets/img/posts/2024-06-12-p3hpc-compat/table.png 1x, /assets/img/posts/2024-06-12-p3hpc-compat/table.2x.png 2x"}](/models/)

In [November of 2022](/2022/11/02/gpu-vendor-model-compat.html), I
created a table comparing GPU programming models and their support on
GPUs of the three vendors (AMD, Intel, NVIDIA) for a talk. The audience
liked it, so I beefed it up a little and posted it in this very blog.

People [**still** liked
it](https://scholar.google.de/scholar?oi=bibs&hl=de&cites=8160066363790021249),
so I took opportunity of some downtime to update and significantly
extend it, and make it into a paper for the [SC23 Workshop on
*Performance, Portability and Productivity in
HPC*](https://p3hpc.org/workshop/2023/) ([P3HPC](https://p3hpc.org/)).
In November 2023, [the paper was published as part of the SC23 Workshop
Proceedings](https://dl.acm.org/doi/10.1145/3624062.3624178?cid=99659740494).
Because it's closed access, you can access it freely if you come from
this blog^[1](#fn:authorizer){.footnote rel="footnote"}^; alternatively,
check out [the
preprint](https://arxiv.org/abs/2309.05445)^[2](#fn:perfectionism){.footnote
rel="footnote"}^.  
I also created a dedicated [*Models page* here on the blog](/models/),
which has the content in HTML form -- and also includes the link to the
paper.

I've changed the following things, compared to the original blog post:

1.  Update all entries and check for additions, deprecations, and other
    developments; extend the descriptions
2.  Describe more properly the methods and categories
3.  Add references for each entry, usually referring to the tool or
    specific aspect discussed
4.  Add discussions and a conclusion
5.  Keep the [GitHub repo](https://github.com/AndiH/gpu-lang-compat) as
    the single source of truth, but extend it to generate the PDF paper
    and to include references in the HTML version (through some horrible
    Pandoc trickery^[3](#fn:pandoctrickery){.footnote rel="footnote"}^)

As the paper is by no means *finished* but a mere snapshot of the status
quo in November 2023, my mini-presentation at the P3HPC workshop
(embedded below) asked for help from the community to keep the content
up-to-date; in the form of contributions to the [GitHub
repository](https://github.com/AndiH/gpu-lang-compat).  
So, **please help** [keep the content
up-to-date](https://github.com/AndiH/gpu-lang-compat/issues)! All
included updates will be incorporated immediately into the [*Models*
page](/models/), the new home of the table. In case you ever forget, the
table and repository are also linked [from the P3HPC workshop
website](https://p3hpc.org/resources/).

Your browser is unable to display the PDF. Instead, please [download
it](/assets/slides/posts/2024-06-12-p3hpc-compat/aherten-p3phpc-mcmm.pdf).

::: {.footnotes role="doc-endnotes"}
1.  ::: {#fn:authorizer}
    ACM gives authors the opportunity to add two referring domains to an
    *accept list*, which displays the paper despite closed access. Not
    great, but better than
    nothing. [↩](#fnref:authorizer){.reversefootnote
    role="doc-backlink"}
    :::

2.  ::: {#fn:perfectionism}
    While I have you here, let me talk about TeX. I've created the PDF
    following the SC23 ACM template guidelines. The most elaborate
    TeXing is some TikZ sprinkled here and there. For the literature, I
    used BibLaTeX, as per ACM's guidelines. Now, first, arXiv has issues
    with the two-column layout and stumbles over hyphenating words for
    line breaks. I could fix most, but not all. Then, for the actual
    publishing, turns out thath the publisher can not even accept
    BibLaTeX, but needs BibTeX. Because I use a lot of URLs and the
    appropriate fields in the bib file, I needed to rework this
    extensively -- last-minute and with a less pleasing look. So while
    the arXiv version still contains BibLaTeX (but has hyphenation
    issues), the published version has BibTeX (but no hyphenation
    issues). [A locally generated pre-preprint with proper hyphenation
    and bibliography can be found here for
    reference](/assets/misc/posts/2024-06-12-p3hpc-compat/aherten-p3hpc-gpu-compat.pdf).
    Phew. [↩](#fnref:perfectionism){.reversefootnote
    role="doc-backlink"}
    :::

3.  ::: {#fn:pandoctrickery}
    References can be referred to by multiple entries, of course, so
    just adding a reference behind each explaining entry would create a
    lot of duplications. To resolve that, I basically concatenate all
    references in a long, Markdown-formatted string and convert it to
    HTML with Pandoc, letting Pandoc resolve the dependencies. I then
    parse the HTML string again to extract the references for each table
    entry in HTML form and add them to the
    output. [↩](#fnref:pandoctrickery){.reversefootnote
    role="doc-backlink"}
    :::
:::


In November of 2022, I created a table comparing GPU programming models and their  support on GPUs of the three vendors (AMD, Intel, NVIDIA) for a talk. The audience liked it, so I beefed it up a little and posted it in this very blog.

Paper: Many Cores, Many Models – GPU Programming Model vs. Vendor Compatibility Overview

Chelsea John

The [Supercomputing Conference 2023](https://sc23.supercomputing.org/)
took place in Denver, Colorado, from November 12th to 17th. For the
*Women in HPC* workshop, we submitted a
[paper](https://zenodo.org/doi/10.5281/zenodo.10116241), which focused
on benchmarking different accelerators for AI. The paper was accepted
and I was invited to hold a [lightning
talk](https://sc23.conference-program.com/presentation/?id=ws_whpc106&sess=sess442)
to show the work, spun off our [OpenGPT-X](https://opengpt-x.de/en/)
project.

Given the multitude of AI accelerators available in the market, our
objective was to start establishing a framework for benchmarking these
accelerators systematically. The currently selected benchmarks enable a
evaluation of performance and energy efficiency in two key AI domains:
Computer Vision and Natural Language Processing (NLP). As an initial
step in assessing hardware capacities, we evaluated two benchmarks using
resources available in the [JURECA
DC](https://doi.org/10.17815/jlsrf-7-182) supercomputer, specifically
the [JURECA DC Evaluation
Platform](https://apps.fz-juelich.de/jsc/hps/jureca/evaluation-platform-overview.html).
We evaluated

- NVIDIA A100 GPUs (40 GB memory, SXM),
- NVIDIA H100 GPUs (80 GB, PCIe),
- AMD MI250 GPUs (64 GB), and
- Graphcore GC200 IPUs (IPU-M2000 POD-4, ≈260 GB).

## ResNet Benchmark

The first benchmark employs TensorFlow for a
[ResNet-50](https://en.wikipedia.org/wiki/Residual_neural_network)
benchmark, providing insights into the overall machine learning capacity
of the hardware. The second benchmark, derived from the OpenGPT-X fork
of [Megatron-LM](https://github.com/NVIDIA/Megatron-LM.git), offers
insights into large language model (LLM) training. A [fork by our
colleagues from Helmholtz
AI](https://github.com/HelmholtzAI-FZJ/tf_cnn_benchmarks.git) of the
ResNet-50 benchmark was executed on the NVIDIA and AMD GPUs. For
Graphcore IPUs, a [device-optimized
version](https://github.com/graphcore/examples.git) provided by the
vendor was used, as a naïve TensorFlow setup is not compatible with the
IPU architecture.

The results for the ResNet-50 benchmark are displayed as heat maps in
the following, with the number of devices on the *y axis* and the global
batch size on the *x axis*. The color map depicts the throughput
measured in images/sec. The throughput scales in relation to the global
batch size and the number of devices used. As the ResNet-50 model fits
into a single device for all the tested hardware, it implies that the
degree of data parallelism matches the number of devices employed.

[![](/assets/img/posts/2023-12-12-sc-novel-architecture-exploration/heatmap-comparison.svg)](/assets/img/posts/2023-12-12-sc-novel-architecture-exploration/heatmap-comparison.svg)

By considering the NVIDIA A100 GPU as a reference point, several
insights can be inferred. Specifically, upon comparing the last row of
the heatmaps for the NVIDIA A100 and H100, a nearly ≈ 1.4 − 2× increase
in throughput is observed. This increase aligns with expectations from
the latest generation of hardware. As seen from the last five columns of
all the heatmaps, NVIDIA GPUs appear to deliver the highest throughput
for larger batch sizes \>128.

Conversely, upon examining Graphcore, the initial four columns,
corresponding to batch sizes below 256, demonstrate superior performance
relative to other accelerators. This observation suggests that the
unique memory architecture of Graphcore IPUs, with SRAM distributed into
an organized set of small independent memory units, contributes to
increased in-processor memory. This enhancement is particularly
beneficial with the MIMD (Multiple Instructions, Multiple Data)
architecture, especially when batches fit into the in-process memory.

Regarding the AMD MI250, we've faced challenges in reaching the GPUs'
peak performance and intend to investigate it further.

## NLP Benchmark

Shifting our focus to the second benchmark, we trained an 800 million
parameter GPT model using the OpenGPT-X fork of Megatron-LM on a single
node (4 GPUs) of NVIDIA A100 and H100 GPUs. The following bar graph
illustrates the compute performance per GPU on the y axis across
different batch sizes for data parallelism of 4. Similar to the ResNet
results, we observed a 1.5× increase in performance.

[![](/assets/img/posts/2023-12-12-sc-novel-architecture-exploration/LLM_tflops_nvidia.svg){width="66%"}](/assets/img/posts/2023-12-12-sc-novel-architecture-exploration/LLM_tflops_nvidia.svg)

We also provide insight into energy efficiency, an aspect of growing
importance in modern computing. To study energy consumption, we train
the model from above for 1 h to achieve representative results. The
total energy consumed per device in a node is calculated using power
values logged with `nvidia-smi`{.language-plaintext .highlighter-rouge}.
H100 GPUs consume less energy than A100 GPUs, with an 18% decrease for a
global batch size of 16. We've added an *energy efficiency* to the
figure, dividing performance (in FLOP/s) by consumed energy (in Wh).

# Conclusion

In conclusion, from our results we observe that the IPU architecture
excels when dealing with small batch sizes that can efficiently fit into
in-processor memory, showcasing remarkable performance gains in these
scenarios. Conversely, the GPU architecture demonstrates its superiority
when it comes to accommodating large batch sizes and scaling, where its
parallel processing capabilities shine. The H100 is expectedly providing
better performance compared to A100, especially if energy is considered
as well.

*OpenGPT-X is funded by the Federal Ministry for Economic Affairs and
Climate Action (BMWK) of Germany for the period 2022-2024.*


The Supercomputing Conference 2023 took place in Denver, Colorado, from November 12th to 17th. For the
<em>
 Women in HPC
</em>
workshop, we submitted a paper, which focused on benchmarking different accelerators for AI. The paper was accepted and I was invited to hold a lightning talk to show the work, spun off our OpenGPT-X project.

SC23 WHPC Workshop Paper: OpenGPT-X – Novel Architecture Exploration

Royal Library of Belgium

Sven Lieber

Hi everyone,

today's summer-break edition is for anyone who likes free and open
knowledge. There is a short video that explains the intriguing new
**Wikifunctions** project. Besides that I will talk about my experiences
of creating a Wikidata page about a company, hopefully it inspires you
to also contribute to free and open knowledge. Warning, it can be
addicting ;-)

::: {.subscription-widget-wrap attrs="{\"url\":\"https://fairdata.substack.com/subscribe?\",\"text\":\"Subscribe\",\"language\":\"en\"}" component-name="SubscribeWidgetToDOM"}
::: {.subscription-widget .show-subscribe}
::: {.preamble}
Thanks for reading FAIR Data Digest! Subscribe for free to receive new
posts and support my work.
:::

::: {.fake-input-wrapper}
::: {.fake-input}
:::

::: {.fake-button}
:::
:::
:::
:::

🎥 Anyone who has ever cooked knows that **only knowing the ingredients
is not sufficient for success**. The recipe to follow, thus the steps to
perform is of great importance to create a delicious meal. In computer
science you have functions which you give an input to receive an output.
As with a recipe, **what the function does is important knowledge!**
Don\'t miss the *Wikifunctions in 7 minutes* video that explains how the
**Wikimedia foundation** is creating a **catalog of multilingual
functions** to share such important knowledge.

🌐 I was updating my Wikidata entry and noticed that a company I've
previously worked for was not yet on Wikidata. Check out how I used
**existing databases, guidelines and the web archive** to create FAIR
data about my former employer *science + computing ag*
([Q121307911](https://www.wikidata.org/wiki/Q121307911)).

<div>

------------------------------------------------------------------------

</div>

# 🎥 Videos not to miss: Wikifunctions in 7 minutes {#videos-not-to-miss-wikifunctions-in-7-minutes}

<div>

------------------------------------------------------------------------

</div>

Could you bake delicious pancakes only with ingredients and without the
recipe of your grandma? In this case maybe yes, but for many things the
recipe, or more general the steps to perform are important knowledge. In
technical terms you can speak of *functions*, and the **Wikimedia
foundation is trying to give them also a dedicated home such that they
become FAIR!**

The following 7 minutes video by [Denny
Vrandečić](https://twitter.com/vrandezo), one of the founders of
Wikidata, will tell you all about the new exiting project Wikifunctions.

Not part of the video, but I'm also curious to see how Wikifunctions
will **integrate with** **other existing catalogs of functions**. [Ben
De Meester](https://work.de-meester.org/), a former colleague of mine,
created the Function Ontology (🦄 FnO) and a
[hub](https://fno.io/hub/search) to search for functions. [DOI:
10.1016/j.future.2019.10.006](https://doi.org/10.1016/j.future.2019.10.006).

::: {#youtube2-bHy63VOp0RQ .youtube-wrap attrs="{\"videoId\":\"bHy63VOp0RQ\",\"startTime\":null,\"endTime\":null}" component-name="Youtube2ToDOM"}
::: {.youtube-inner}
::: {.iframe}
::: {#player}
:::

::: {.player-unavailable}
# Ein Fehler ist aufgetreten. {#ein-fehler-ist-aufgetreten. .message}

::: {.submessage}
[Sieh dir dieses Video auf www.youtube.com
an](https://www.youtube.com/watch?v=bHy63VOp0RQ){target="_blank"} oder
aktiviere JavaScript, falls es in deinem Browser deaktiviert sein
sollte.
:::
:::
:::
:::
:::

<div>

------------------------------------------------------------------------

</div>

# 🌐 Wikidata Spotlight: company s+c {#wikidata-spotlight-company-sc}

<div>

------------------------------------------------------------------------

</div>

Last week I noticed that an IT company for which I've worked for a few
years was not yet linked to my Wikidata page. This information is on my
LinkedIn page, but **not yet as structured open data on Wikidata and I
wanted to change that!**

First things first: I used the **Wikidata search** to check if the
company already exists on Wikidata. If yes I could have just created a
link. Unfortunately there was not yet a Wikidata item. Thus I've created
one with a label and a description to start with. There are
[guidelines](https://www.wikidata.org/wiki/Help:Description) about what
a good description should look like.

I also **looked up the company in different databases to obtain
third-party identifiers**: the [Virtual International Authority
File](https://viaf.org/) (VIAF) and the [International Standard Name
Identifier](https://isni.org/) (ISNI). Additionally, I've looked up the
companies identifier in the Google Knowledge Graph via the
[kalicube](https://kalicube.pro/tools/knowledge-graph-explorer) website.

Ideally, **claims on Wikidata should be supported with references**.
Specific information about the registration of the company I found in
the *German Commercial register (Handelsregister)*. Since 2020 they
provide a web interface to search for registered companies.
Unfortunately there are **no unique links or identifiers**. So instead
of providing a precise URL as a reference, I could only indicate
***that*** the information is stated in the register, but not
***where*** (see image below).

::: {.captioned-image-container}
<figure>
<a
href="https://substackcdn.com/image/fetch/f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fb868535c-beae-402c-abaa-f3436fcb57c5_804x237.png"
class="image-link image2" target="_blank"
data-component-name="Image2ToDOM"></a>
<div class="image2-inset">
<img
src="https://substackcdn.com/image/fetch/w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fb868535c-beae-402c-abaa-f3436fcb57c5_804x237.png"
class="sizing-normal"
data-attrs="{&quot;src&quot;:&quot;https://substack-post-media.s3.amazonaws.com/public/images/b868535c-beae-402c-abaa-f3436fcb57c5_804x237.png&quot;,&quot;srcNoWatermark&quot;:null,&quot;fullscreen&quot;:null,&quot;imageSize&quot;:null,&quot;height&quot;:237,&quot;width&quot;:804,&quot;resizeWidth&quot;:null,&quot;bytes&quot;:22296,&quot;alt&quot;:null,&quot;title&quot;:null,&quot;type&quot;:&quot;image/png&quot;,&quot;href&quot;:null,&quot;belowTheFold&quot;:true,&quot;topImage&quot;:false,&quot;internalRedirect&quot;:null}"
srcset="https://substackcdn.com/image/fetch/w_424,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fb868535c-beae-402c-abaa-f3436fcb57c5_804x237.png 424w, https://substackcdn.com/image/fetch/w_848,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fb868535c-beae-402c-abaa-f3436fcb57c5_804x237.png 848w, https://substackcdn.com/image/fetch/w_1272,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fb868535c-beae-402c-abaa-f3436fcb57c5_804x237.png 1272w, https://substackcdn.com/image/fetch/w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fb868535c-beae-402c-abaa-f3436fcb57c5_804x237.png 1456w"
sizes="100vw" loading="lazy" width="804" height="237" />
<div>

</div>
</div>
<figcaption>Backing up a claim by using a reference: if the source would
provide a unique identifier and URL this would be even
better.</figcaption>
</figure>
:::

## Finding references and other links. {#finding-references-and-other-links.}

Nowadays, the company is part of another company. Via the [web
archive](https://archive.org/) I checked older news items from the
company to check for noteworthy links. Apparently the company
participated in the European project KnowARC
([Q2602664](https://www.wikidata.org/wiki/Q2602664)). I found that
project on Wikidata with minimal information (only a label and subclass
of statement). So, by looking up the project on the European CORDIS
service, I could complete the Wikidata item of the project, including a
link to the company item I just created =\> science + computing ag
([Q121307911](https://www.wikidata.org/wiki/Q121307911)).

There are actually many tutorials on how to do things on Wikidata. Go
check them out and contribute as well to open and free knowlegde.

[Check out the Wikidata
tutorials](https://www.wikidata.org/wiki/Wikidata:Tours){.button
.primary}

<div>

------------------------------------------------------------------------

</div>

That's it for this week of the FAIR Data Digest. I hope you found the
content interesting. Don't forget to share or subscribe. See you next
week!

Sven

::: {.subscription-widget-wrap attrs="{\"url\":\"https://fairdata.substack.com/subscribe?\",\"text\":\"Subscribe\",\"language\":\"en\"}" component-name="SubscribeWidgetToDOM"}
::: {.subscription-widget .show-subscribe}
::: {.preamble}
Thanks for reading FAIR Data Digest! Subscribe for free to receive new
posts and support my work.
:::

::: {.fake-input-wrapper}
::: {.fake-input}
:::

::: {.fake-button}
:::
:::
:::
:::


Hi everyone,  today’s summer-break edition is for anyone who likes free and open knowledge. There is a short video that explains the intriguing new
<strong>
 Wikifunctions
</strong>
project. Besides that I will talk about my experiences of creating a Wikidata page about a company, hopefully it inspires you to also contribute to free and open knowledge.

Rogue Scholar Beiträge

Paper: Many Cores, Many Models – GPU Programming Model vs. Vendor Compatibility Overview

SC23 WHPC Workshop Paper: OpenGPT-X – Novel Architecture Exploration

FAIR Data Digest #11