Australian National University

Qingqin Fang

#### An improvement architecture superior to the Transformer, proposed by Meta

<figure>
<img src="https://cdn-images-1.medium.com/max/852/0*pFztXZpCGW0GplEE" />
<figcaption>Image Generated by Canvas</figcaption>
</figure>

### Author

· Qingqin Fang (**ORCID:**
[0009--0003--5348--4264](https://orcid.org/0009-0003-5348-4264))

### Introduction

Recently, researchers from Meta and the University of Southern
California have introduced a model called Megalodon. They claim that
this model can expand the context window of language models to handle
millions of tokens without overwhelming your memory.

In previous studies on long texts, the focus was mainly on enhancing
performance through optimizations of the Transformer. However, lab data
from recent experiments show that Megalodon outperforms Transformer
models of similar size when processing large volumes of text, and many
researchers are beginning to view it as the potential successor to the
Transformer.

Transformer architecture is generally limited by its high complexity and
its weak ability to handle very long inputs. While there are simpler
solutions that try to address these issues, they usually don't perform
as well in terms of training speed or accuracy on practical tasks. The
emergence of Megalodon is specifically aimed at addressing the challenge
of handling infinite context, which makes it better suited for tasks
where understanding extensive context is crucial.

This post delves into Megalodon.

### Long Context Window

In long-text situations, "**context window**" indicates the number of
tokens a model can handle at one time. Expanding this window enables LLM
to engage in longer conversations, process a wider range of documents,
and learn more information.

In machine learning models, the **attention** mechanism is a special
structure that enhances the weights of certain parts of the input data
while reducing the weights of other parts. This mechanism automatically
learns and calculates the contribution of input data to the output data,
allowing the model to focus on different parts of the input with varying
levels of importance during task execution. This concept is somewhat
similar to human attention.

The diagram below shows the core parts and steps of the **QKV
attention** mechanism, an acronym for Query, Key, and Value, commonly
used in **self-attention** mechanisms, particularly in Transformer
models. Initially, input data is linearly transformed into three
segments: Query (yellow), Key (pink), and Value (green). Each Query is
then dot-multiplied with all Keys to calculate their similarity scores,
which are normalized using the Softmax function. This process assigns
attention weights to each Key relative to each Query, ultimately
producing the final output based on these weights.

<figure>
<img
src="https://cdn-images-1.medium.com/max/1024/0*0CGhyOD6zPwz2Nl3" />
<figcaption>Transformer-Attention Flow (scaled-dot-product). Link: <a
href="https://scholar.harvard.edu/sites/scholar.harvard.edu/files/binxuw/files/mlfs_tutorial_nlp_transformer_ssl_updated.pdf">From
Transformer to LLM: Architecture, Training and Usage</a></figcaption>
</figure>

However, self-attention mechanism compares each element in the input
sequence with all others using Scaled Dot-Product Attention. This means
that if the input size is doubled, the memory and computation time
needed to process the input will quadruple. Expanding the Transformer's
context window comes with significant costs. The challenge with
Transformers lies in their "**quadratic complexity**".

Meanwhile, Transformers also face issues with "**the lack of
position-related inductive bias**", which can impact their performance.
Unlike Recurrent Neural Networks (RNNs) or Convolutional Neural Networks
(CNNs), which naturally process data sequentially or have the ability to
recognize spatial structures, transformers rely on **position
embeddings** to incorporate information about the order of tokens. This
method involves adding positional information to the word embedding
vectors of each token, allowing the same word to have different
representations depending on its position in the sentence.

This artificially creates a position-related bias, but such a bias may
not be as effective as the natural positional sensitivity found in other
models. When dealing with tasks where positional context is crucial, for
example, the two *"I"* are located at different positions in the
sentence "*I think, therefore I am",* it should have the same output
representations, but the two I's don\'t. This can sometimes lead to
inefficiencies or require additional mechanisms to address the issue.

In 2022, the **Moving Average Equipped Gated Attention**(MEGA) model was
introduced as a significant enhancement to the traditional Transformer
architecture, particularly improving its performance in handling long
sequence data. [This
article](https://medium.com/@_init_/how-self-attention-with-relative-position-representations-works-28173b8c245a)
provides a more detailed explanation.

MEGA incorporates the **Exponential Moving Average** (EMA) into the
self-attention mechanism. Moving averages, which is a classic approach
for sequential data modelling, widely used in time series data to smooth
out short-term fluctuations and highlight long-term trends or cycles is
calculated for each element's Query and Key, smoothing data and updating
weights. These Queries and Keys reflect not only the current input
features but also the influence of historical data, enabling more
effective capture and utilization of long-distance contextual
information. EMA facilitates the flow of information across different
time steps, improving the model's understanding of long-range
dependencies within sequences.

Moreover, by dividing the computation of the attention mechanism into
smaller segments, MEGA substantially reduces computational complexity
from O(n²) to **approximately O(n)**. The use of EMA allows the MEGA
model to maintain a more coherent flow of information between local and
overall context, helping to prevent the loss of information during the
processing across blocks. This restructuring makes it possible to handle
longer data sequences more efficiently while preserving the integrity of
the information processed.

<figure>
<img
src="https://cdn-images-1.medium.com/max/1024/0*IhNa1lHvSi8YNuXh" />
<figcaption>Mega — model architecture, Source: Xuezhe Ma et al., 2022.
Link: <a
href="https://arxiv.org/pdf/2209.10655">https://arxiv.org/pdf/2209.10655</a></figcaption>
</figure>

### Megalodon

Building on the MEGA architecture, Megalodon has implemented a series of
enhancements and added several new technical components:

Firstly, Megalodon introduces the **Complex Exponential Moving Average
(CEMA)** component, a novel technique that extends the [Multidimensional
Damped Exponential Moving Average](https://arxiv.org/abs/2312.02549)
(EMA) from MEGA into the complex domain. The previous MEGA model was
based on real numbers, and Megalodon extends it to include imaginary
numbers, hence the term "complex domain" here. This enhancement boosts
the model's capability to process complex data, thereby improving
performance and stability.

Secondly, normalization is a crucial step in Transformer models,
involving adjusting data to a distribution with a mean of 0 and a
variance of 1. This reduces variations in data distribution during model
training, thus helping the model learn faster. The commonly used
normalization methods are:

- **Layer Normalization** adjusts the outputs by calculating the mean
  and standard deviation of all neuron outputs within a layer. This
  method effectively stabilizes the hidden states in recurrent networks
  but does not account for batch and sequential data properties,
  resulting in poor management of dependencies between time steps.
- **Group Normalization** normalizes neuron outputs by grouping them.
  This method is not affected by batch size and maintains good stability
  across a wide range of batch sizes. However, its performance heavily
  depends on how the channels are grouped, and it may inadvertently leak
  future information in autoregressive models due to normalization
  across different time steps.

To address these issues, researchers have proposed a new, more complex
method called **Timestep Normalization**. This technique extends group
normalization to autoregressive sequence modelling, allowing for
independent normalization at each time step. This enables the model to
effectively adjust step-by-step in sequence processing, ensuring that
dependencies between time steps do not compromise model performance.
This method helps the model handle sequential data more accurately,
improving processing effectiveness and efficiency.

<figure>
<img
src="https://cdn-images-1.medium.com/max/1024/0*V5PbIbNUwDAash4D" />
<figcaption>Normalization methods. The elements in blue or yellow are
the regions to compute means and variances. Source: Xuezhe Ma et al.,
2024. Link: <a
href="https://arxiv.org/pdf/2404.08801">https://arxiv.org/pdf/2404.08801</a></figcaption>
</figure>

Additionally, Megalodon has introduced a **normalized attention
mechanism**. This involves normalizing the queries (Query) and keys
(Key) before calculating the attention scores, which helps stabilize the
scores and addresses issues caused by inconsistent sizing, improving the
model's overall reliability and training stability.

These adjustments are achieved by enhancing common normalization
techniques. By breaking down the input sequence into parts, Megalodon
simplifies the computational process and memory usage during model
training and inference, making it more efficient. This allows Megalodon
to handle large amounts of data more effectively without using excessive
resources.

### Experiment and its Results

Researchers trained a 7-billion-parameter Megalodon model on the same 2
trillion pre-training data as used with the Llama2 model, allowing for a
completely fair comparison with Llama2--7B. The final results are shown
in the diagram below.

<figure>
<img
src="https://cdn-images-1.medium.com/max/1024/0*AJmNTzXPJVCQRkK8" />
<figcaption>Processed tokens during training for MEGALODON-7B, LLAMA2–7B
and LLAMA2–13B. Source: Xuezhe Ma et al., 2024. Link: <a
href="https://arxiv.org/pdf/2404.08801">https://arxiv.org/pdf/2404.08801</a></figcaption>
</figure>

It is evident from the direct comparison with Llama 2 that Megalodon not
only trains more efficiently but also surpasses the Transformer in
accuracy when handling tasks with 7 billion parameters and 2 trillion
training tokens. Specifically, Megalodon's training loss was 1.70, which
is between Llama2--7B (1.75) and 13B (1.67).

At the same time, researchers used the same computational resources to
run experiments with LLAMA 2--7B and MEGALODON-7B, comparing their
training speeds at different context lengths.

<figure>
<img src="https://cdn-images-1.medium.com/max/836/0*BGeSHoAz8e3dL6jV" />
<figcaption>Average word/token per second for MEGALODON-7B and
LLAMA2–7B. Source: Xuezhe Ma et al., 2024. Link: <a
href="https://arxiv.org/pdf/2404.08801">https://arxiv.org/pdf/2404.08801</a></figcaption>
</figure>

At a 4K context length, MEGALODON-7B was slightly slower (by about 6%)
than LLAMA 2--7B, due to the introduction of CEMA and timestep
normalization. However, when the context length was expanded to 32K,
MEGALODON-7B was significantly faster by 32% compared to LLAMA 2--7B,
demonstrating MEGALODON's computational efficiency in long-context
pre-training.

For further experimental details, please refer to the [full
paper](https://arxiv.org/pdf/2404.08801). Additionally, [the code for
this project has been made open
source](https://github.com/XuezheMax/megalodon), making it accessible to
a broader audience. The computational cost of running the 7-billion
parameter model is optimized to be feasible on personal computers,
enhancing its accessibility for individual researchers and developers.

### Future Thinking

So, can the Transformer be replaced by these new architectures?

As of now, the Transformer architecture still holds a leading position.
While Meta is exploring new architectures like Megalodon, it is also
actively improving its Transformer models and has recently released the
latest version of its open-source
LLM, [Llama-3](https://ollama.com/library/llama3).

One challenge that competitors to Transformers face is the need for
specialized hardware and software tools. There is already a large
library and ecosystem of tools available for training, fine-tuning, and
customizing Transformer models, suitable for various applications and
hardware configurations. Researchers have also developed optimized
software codes to enhance the performance of Transformer LLMs on
memory-constrained devices. Alternatives still need to keep up with
these advancements.

Meanwhile, other researchers are modifying the Transformer architecture
to reduce its memory and computational demands. For instance, Google
researchers recently published a paper on
[Infini-attention](https://arxiv.org/abs/2404.07143), aimed at providing
Transformers with an infinite context window without increasing memory
and computational complexity.

While new architectures like Megalodon bring exciting improvements,
particularly in handling longer sequences and reducing computational
demands, the traditional Transformer model still holds a dominant
position in the AI field due to its widespread application and
continuous development.

In the future, these new models could work alongside or gradually
integrate with the traditional Transformer framework to more effectively
address specific challenges. Whether these models will completely
replace Transformers, or rather complement them, depends on further
advancements and broader adoption of these innovative approaches in
practical applications.

Looking ahead, observing how these models evolve and are utilized across
various domains might reshape our understanding and implementation of
neural network architectures in large-scale artificial intelligence
systems. The journey from traditional Transformers to models like
Megalodon marks a dynamic and transformative phase in machine learning,
promising to enhance the capabilities and impact of AI technologies
globally.

### **References**

- Xuezhe Ma et al. (2024). Megalodon: Efficient LLM Pretraining and
  Inference with Unlimited Context Length. arXiv:2404.08801 \[cs.LG\].
  DOI: <https://arxiv.org/pdf/2404.08801>
- Xuezhe Ma et al. (2024). Mega: Moving average equipped gated
  attention. arXiv:2209.10655 \[cs.LG\]. DOI:
  <https://doi.org/10.48550/arXiv.2209.10655>

![](https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=b4c7322602ba){width="1"
height="1"}


An improvement architecture superior to the Transformer, proposed by Meta  Author   · Qingqin Fang (
<strong>
 ORCID:
</strong>
0009–0003–5348–4264) Introduction   Recently, researchers from Meta and the University of Southern California have introduced a model called Megalodon. They claim that this model can expand the context window of language models to handle millions of tokens without overwhelming your memory.

The longer the context, the better? Unlimited Context Length in Megalodon

Western University of Health Sciences

Matt Wedel

[![](https://svpow.files.wordpress.com/2022/11/bored-meg-tooth-1-lingual.jpg?w=480){.size-large
.wp-image-20442 .aligncenter loading="lazy" attachment-id="20442"
permalink="http://svpow.com/2022/11/09/shark-week-4-megalodon-tooth-colonized-by-boring-clams/bored-meg-tooth-1-lingual/"
orig-file="https://svpow.files.wordpress.com/2022/11/bored-meg-tooth-1-lingual.jpg"
orig-size="2250,3000" comments-opened="1"
image-meta="{\"aperture\":\"0\",\"credit\":\"\",\"camera\":\"\",\"caption\":\"\",\"created_timestamp\":\"0\",\"copyright\":\"\",\"focal_length\":\"0\",\"iso\":\"0\",\"shutter_speed\":\"0\",\"title\":\"\",\"orientation\":\"0\"}"
image-title="Bored meg tooth 1 – lingual" image-description=""
image-caption=""
medium-file="https://svpow.files.wordpress.com/2022/11/bored-meg-tooth-1-lingual.jpg?w=225"
large-file="https://svpow.files.wordpress.com/2022/11/bored-meg-tooth-1-lingual.jpg?w=768"
width="480" height="640"
srcset="https://svpow.files.wordpress.com/2022/11/bored-meg-tooth-1-lingual.jpg?w=480 480w, https://svpow.files.wordpress.com/2022/11/bored-meg-tooth-1-lingual.jpg?w=960 960w, https://svpow.files.wordpress.com/2022/11/bored-meg-tooth-1-lingual.jpg?w=113 113w, https://svpow.files.wordpress.com/2022/11/bored-meg-tooth-1-lingual.jpg?w=225 225w, https://svpow.files.wordpress.com/2022/11/bored-meg-tooth-1-lingual.jpg?w=768 768w"
sizes="(max-width: 480px) 100vw, 480px"}](https://svpow.files.wordpress.com/2022/11/bored-meg-tooth-1-lingual.jpg)

Alert readers probably noticed that I titled the first post in this
series "[Matt's first megalodon
tooth](https://svpow.com/2022/11/05/matts-first-megalodon-tooth/)",
implying that there would be other megalodon teeth to follow. Here's my
second one.  
  
At first glance, this is a pretty jacked-up megalodon tooth. It is
pocked with circular and ovoid craters, and has a big fat hole drilled
right through it. Hardly collector grade! And in fact that's what first
caught my attention about this tooth --- it's a 6-incher that was being
offered for an enticingly low price. But I got even more excited when I
clicked past the thumbnail image on the sale site and saw precisely how
this tooth was damaged. This is not random, senseless taphonomic battery
([ahem](https://svpow.com/2019/09/16/matt-bought-an-iguanodon-pelvis-or-did-he/));
this tooth was colonized by a bunch of [boring
clams](https://en.wikipedia.org/wiki/Pholad_borings).

[![](https://svpow.files.wordpress.com/2022/11/bored-meg-tooth-2-labial.jpg?w=480){.size-large
.wp-image-20446 .aligncenter loading="lazy" attachment-id="20446"
permalink="http://svpow.com/2022/11/09/shark-week-4-megalodon-tooth-colonized-by-boring-clams/bored-meg-tooth-2-labial/"
orig-file="https://svpow.files.wordpress.com/2022/11/bored-meg-tooth-2-labial.jpg"
orig-size="2550,3400" comments-opened="1"
image-meta="{\"aperture\":\"0\",\"credit\":\"\",\"camera\":\"\",\"caption\":\"\",\"created_timestamp\":\"0\",\"copyright\":\"\",\"focal_length\":\"0\",\"iso\":\"0\",\"shutter_speed\":\"0\",\"title\":\"\",\"orientation\":\"0\"}"
image-title="Bored meg tooth 2 – labial" image-description=""
image-caption=""
medium-file="https://svpow.files.wordpress.com/2022/11/bored-meg-tooth-2-labial.jpg?w=225"
large-file="https://svpow.files.wordpress.com/2022/11/bored-meg-tooth-2-labial.jpg?w=768"
width="480" height="640"
srcset="https://svpow.files.wordpress.com/2022/11/bored-meg-tooth-2-labial.jpg?w=480 480w, https://svpow.files.wordpress.com/2022/11/bored-meg-tooth-2-labial.jpg?w=960 960w, https://svpow.files.wordpress.com/2022/11/bored-meg-tooth-2-labial.jpg?w=113 113w, https://svpow.files.wordpress.com/2022/11/bored-meg-tooth-2-labial.jpg?w=225 225w, https://svpow.files.wordpress.com/2022/11/bored-meg-tooth-2-labial.jpg?w=768 768w"
sizes="(max-width: 480px) 100vw, 480px"}](https://svpow.files.wordpress.com/2022/11/bored-meg-tooth-2-labial.jpg)  
Like [Adam Savage](https://www.youtube.com/watch?v=rEg-ZNB3qyI) --- and,
I suspect, most collectors-of-things --- I am fascinated by objects and
the stories that they tell. And this tooth tells several stories. First,
it's a huge tooth from a huge shark, a truly vast, multi-ton animal
heavier than a *T. rex* and longer than my house. Second, it's a fossil
that's millions of years old, evidence of an extinct species from a
vanished ecology, one where gigantic sharks and [macroraptorial sperm
whales](https://en.wikipedia.org/wiki/Livyatan) hunted small baleen
whales, early seals and sea lions, and manatees and sea cows. And third,
it's a relic of another, entirely different ecology, one in which this
shed tooth sank to the sea floor and was colonized by a host of smaller
organisms, including most obviously hole-boring clams. In effect, this
one tooth was a miniature reef, supporting multiple species of
invertebrates. The traces left by those invertebrates are themselves
ichnofossils, so this tooth is a body fossil with ichnofossils dug out
of it. It's [turtles all the way
down](https://en.wikipedia.org/wiki/Turtles_all_the_way_down)!

[![](https://svpow.files.wordpress.com/2022/11/bored-meg-tooth-3-distal-margin-with-hole.jpg?w=480){.size-large
.wp-image-20444 .aligncenter loading="lazy" attachment-id="20444"
permalink="http://svpow.com/2022/11/09/shark-week-4-megalodon-tooth-colonized-by-boring-clams/bored-meg-tooth-3-distal-margin-with-hole/"
orig-file="https://svpow.files.wordpress.com/2022/11/bored-meg-tooth-3-distal-margin-with-hole.jpg"
orig-size="3200,2400" comments-opened="1"
image-meta="{\"aperture\":\"0\",\"credit\":\"\",\"camera\":\"\",\"caption\":\"\",\"created_timestamp\":\"0\",\"copyright\":\"\",\"focal_length\":\"0\",\"iso\":\"0\",\"shutter_speed\":\"0\",\"title\":\"\",\"orientation\":\"0\"}"
image-title="Bored meg tooth 3 – distal margin with hole"
image-description="" image-caption=""
medium-file="https://svpow.files.wordpress.com/2022/11/bored-meg-tooth-3-distal-margin-with-hole.jpg?w=300"
large-file="https://svpow.files.wordpress.com/2022/11/bored-meg-tooth-3-distal-margin-with-hole.jpg?w=1024"
width="480" height="360"
srcset="https://svpow.files.wordpress.com/2022/11/bored-meg-tooth-3-distal-margin-with-hole.jpg?w=480 480w, https://svpow.files.wordpress.com/2022/11/bored-meg-tooth-3-distal-margin-with-hole.jpg?w=960 960w, https://svpow.files.wordpress.com/2022/11/bored-meg-tooth-3-distal-margin-with-hole.jpg?w=150 150w, https://svpow.files.wordpress.com/2022/11/bored-meg-tooth-3-distal-margin-with-hole.jpg?w=300 300w, https://svpow.files.wordpress.com/2022/11/bored-meg-tooth-3-distal-margin-with-hole.jpg?w=768 768w"
sizes="(max-width: 480px) 100vw, 480px"}](https://svpow.files.wordpress.com/2022/11/bored-meg-tooth-3-distal-margin-with-hole.jpg)  
Can we figure out what any of those invertebrates were? Just a few years
ago that would have been a challenging task for a non-specialist, but
fortunately in 2019 Harry Maisch and colleagues published a really cool
paper, "Macroborings in *Otodus megalodon* and *Otodus chubutensis*
shark teeth from the submerged shelf of Onslow Bay, North Carolina, USA:
implications for processes of lag deposit formation". That paper is very
well illustrated, and the figures basically serve as a field guide for
anyone who wants to identify similar traces in rocks or teeth of
equivalent age. I will take up that sword in a future post.

[![](https://svpow.files.wordpress.com/2022/11/bored-and-complete-megalodon-teeth-lingual.jpg?w=480){.size-large
.wp-image-20454 .aligncenter loading="lazy" attachment-id="20454"
permalink="http://svpow.com/2022/11/09/shark-week-4-megalodon-tooth-colonized-by-boring-clams/bored-and-complete-megalodon-teeth-lingual/"
orig-file="https://svpow.files.wordpress.com/2022/11/bored-and-complete-megalodon-teeth-lingual.jpg"
orig-size="3000,2000" comments-opened="1"
image-meta="{\"aperture\":\"0\",\"credit\":\"\",\"camera\":\"\",\"caption\":\"\",\"created_timestamp\":\"0\",\"copyright\":\"\",\"focal_length\":\"0\",\"iso\":\"0\",\"shutter_speed\":\"0\",\"title\":\"\",\"orientation\":\"0\"}"
image-title="Bored and complete megalodon teeth – lingual"
image-description="" image-caption=""
medium-file="https://svpow.files.wordpress.com/2022/11/bored-and-complete-megalodon-teeth-lingual.jpg?w=300"
large-file="https://svpow.files.wordpress.com/2022/11/bored-and-complete-megalodon-teeth-lingual.jpg?w=1024"
width="480" height="320"
srcset="https://svpow.files.wordpress.com/2022/11/bored-and-complete-megalodon-teeth-lingual.jpg?w=480 480w, https://svpow.files.wordpress.com/2022/11/bored-and-complete-megalodon-teeth-lingual.jpg?w=960 960w, https://svpow.files.wordpress.com/2022/11/bored-and-complete-megalodon-teeth-lingual.jpg?w=150 150w, https://svpow.files.wordpress.com/2022/11/bored-and-complete-megalodon-teeth-lingual.jpg?w=300 300w, https://svpow.files.wordpress.com/2022/11/bored-and-complete-megalodon-teeth-lingual.jpg?w=768 768w"
sizes="(max-width: 480px) 100vw, 480px"}](https://svpow.files.wordpress.com/2022/11/bored-and-complete-megalodon-teeth-lingual.jpg)

Incidentally, this is now the biggest tooth in my little collection,
just slightly --- but noticeably --- bigger than my first megalodon
tooth: 157mm on the long side, vs 155mm, and 112mm max root width, vs
107mm.

[![](https://svpow.files.wordpress.com/2022/11/bored-meg-tooth-4-phaser.jpg?w=480){.size-large
.wp-image-20448 .aligncenter loading="lazy" attachment-id="20448"
permalink="http://svpow.com/2022/11/09/shark-week-4-megalodon-tooth-colonized-by-boring-clams/bored-meg-tooth-4-phaser/"
orig-file="https://svpow.files.wordpress.com/2022/11/bored-meg-tooth-4-phaser.jpg"
orig-size="3200,2400" comments-opened="1"
image-meta="{\"aperture\":\"0\",\"credit\":\"\",\"camera\":\"\",\"caption\":\"\",\"created_timestamp\":\"0\",\"copyright\":\"\",\"focal_length\":\"0\",\"iso\":\"0\",\"shutter_speed\":\"0\",\"title\":\"\",\"orientation\":\"0\"}"
image-title="Bored meg tooth 4 – phaser" image-description=""
image-caption=""
medium-file="https://svpow.files.wordpress.com/2022/11/bored-meg-tooth-4-phaser.jpg?w=300"
large-file="https://svpow.files.wordpress.com/2022/11/bored-meg-tooth-4-phaser.jpg?w=1024"
width="480" height="360"
srcset="https://svpow.files.wordpress.com/2022/11/bored-meg-tooth-4-phaser.jpg?w=480 480w, https://svpow.files.wordpress.com/2022/11/bored-meg-tooth-4-phaser.jpg?w=960 960w, https://svpow.files.wordpress.com/2022/11/bored-meg-tooth-4-phaser.jpg?w=150 150w, https://svpow.files.wordpress.com/2022/11/bored-meg-tooth-4-phaser.jpg?w=300 300w, https://svpow.files.wordpress.com/2022/11/bored-meg-tooth-4-phaser.jpg?w=768 768w"
sizes="(max-width: 480px) 100vw, 480px"}](https://svpow.files.wordpress.com/2022/11/bored-meg-tooth-4-phaser.jpg)

Bonus goofy observation: I strongly suspect that no other megalodon
tooth in the world beats this one in simulating a Star Trek phaser.

## Reference

[Maisch IV, H.M., Becker, M.A. and Chamberlain Jr, J.A. 2020.
Macroborings in *Otodus megalodon* and *Otodus chubutensis* shark teeth
from the submerged shelf of Onslow Bay, North Carolina, USA:
implications for processes of lag deposit formation. Ichnos 27(2):
122-141.](https://www.researchgate.net/profile/John-Jr-3/publication/338011863_Macroborings_in_Otodus_megalodon_and_Otodus_chubutensis_Shark_Teeth_from_the_Submerged_Shelf_of_Onslow_Bay_North_Carolina_USA_Implications_for_Processes_of_Lag_Deposit_Formation/links/5dfa2f2c299bf10bc36379aa/Macroborings-in-Otodus-megalodon-and-Otodus-chubutensis-Shark-Teeth-from-the-Submerged-Shelf-of-Onslow-Bay-North-Carolina-USA-Implications-for-Processes-of-Lag-Deposit-Formation.pdf)


{.size-large .wp-image-20442 .aligncenter loading=“lazy” attachment-id=“20442” permalink=“http://svpow.com/2022/11/09/shark-week-4-megalodon-tooth-colonized-by-boring-clams/bored-meg-tooth-1-lingual/” orig-file=“https://svpow.files.wordpress.com/2022/11/bored-meg-tooth-1-lingual.jpg” orig-size=“2250,3000” comments-opened=“1”

Shark week 4: megalodon tooth colonized by boring clams

Something cool came in the mail today: a fossil tooth of a great white shark, Carcharodon carcharias. The root is a bit eroded, but the enamel-covered crown is in great shape, and it’s almost exactly the same size as my cast tooth from a modern great white. I got this for a couple of reasons.

::: {#attachment_20405 .wp-caption .aligncenter shortcode="caption" style="width: 1034px"}
[![](https://svpow.files.wordpress.com/2022/11/fossil-great-white-tooth-1-labial-side-in-hand.jpg?w=480){.wp-image-20405
.size-large aria-describedby="caption-attachment-20405" loading="lazy"
attachment-id="20405"
permalink="http://svpow.com/2022/11/06/shark-week-2-great-white-teeth-redux/fossil-great-white-tooth-1-labial-side-in-hand/"
orig-file="https://svpow.files.wordpress.com/2022/11/fossil-great-white-tooth-1-labial-side-in-hand.jpg"
orig-size="3000,2250" comments-opened="1"
image-meta="{\"aperture\":\"0\",\"credit\":\"\",\"camera\":\"\",\"caption\":\"\",\"created_timestamp\":\"0\",\"copyright\":\"\",\"focal_length\":\"0\",\"iso\":\"0\",\"shutter_speed\":\"0\",\"title\":\"\",\"orientation\":\"0\"}"
image-title="Fossil great white tooth 1 – labial side in hand"
image-description="" image-caption=""
medium-file="https://svpow.files.wordpress.com/2022/11/fossil-great-white-tooth-1-labial-side-in-hand.jpg?w=300"
large-file="https://svpow.files.wordpress.com/2022/11/fossil-great-white-tooth-1-labial-side-in-hand.jpg?w=1024"
width="480" height="360"
srcset="https://svpow.files.wordpress.com/2022/11/fossil-great-white-tooth-1-labial-side-in-hand.jpg?w=480 480w, https://svpow.files.wordpress.com/2022/11/fossil-great-white-tooth-1-labial-side-in-hand.jpg?w=960 960w, https://svpow.files.wordpress.com/2022/11/fossil-great-white-tooth-1-labial-side-in-hand.jpg?w=150 150w, https://svpow.files.wordpress.com/2022/11/fossil-great-white-tooth-1-labial-side-in-hand.jpg?w=300 300w, https://svpow.files.wordpress.com/2022/11/fossil-great-white-tooth-1-labial-side-in-hand.jpg?w=768 768w"
sizes="(max-width: 480px) 100vw, 480px"}](https://svpow.files.wordpress.com/2022/11/fossil-great-white-tooth-1-labial-side-in-hand.jpg)

Cast (white) and fossil (gray) great white shark teeth, lingual (tongue)
sides.
:::

Something cool came in the mail today: a fossil tooth of a great white
shark, *Carcharodon carcharias*. The root is a bit eroded, but the
enamel-covered crown is in great shape, and it's almost exactly the same
size as my cast tooth from a modern great white.

::: {#attachment_20406 .wp-caption .aligncenter shortcode="caption" style="width: 1034px"}
[![](https://svpow.files.wordpress.com/2022/11/fossil-great-white-tooth-2-lingual-side-in-hand.jpg?w=480){.wp-image-20406
.size-large aria-describedby="caption-attachment-20406" loading="lazy"
attachment-id="20406"
permalink="http://svpow.com/2022/11/06/shark-week-2-great-white-teeth-redux/fossil-great-white-tooth-2-lingual-side-in-hand/"
orig-file="https://svpow.files.wordpress.com/2022/11/fossil-great-white-tooth-2-lingual-side-in-hand.jpg"
orig-size="3000,2250" comments-opened="1"
image-meta="{\"aperture\":\"0\",\"credit\":\"\",\"camera\":\"\",\"caption\":\"\",\"created_timestamp\":\"0\",\"copyright\":\"\",\"focal_length\":\"0\",\"iso\":\"0\",\"shutter_speed\":\"0\",\"title\":\"\",\"orientation\":\"0\"}"
image-title="Fossil great white tooth 2 – lingual side in hand"
image-description="" image-caption=""
medium-file="https://svpow.files.wordpress.com/2022/11/fossil-great-white-tooth-2-lingual-side-in-hand.jpg?w=300"
large-file="https://svpow.files.wordpress.com/2022/11/fossil-great-white-tooth-2-lingual-side-in-hand.jpg?w=1024"
width="480" height="360"
srcset="https://svpow.files.wordpress.com/2022/11/fossil-great-white-tooth-2-lingual-side-in-hand.jpg?w=480 480w, https://svpow.files.wordpress.com/2022/11/fossil-great-white-tooth-2-lingual-side-in-hand.jpg?w=960 960w, https://svpow.files.wordpress.com/2022/11/fossil-great-white-tooth-2-lingual-side-in-hand.jpg?w=150 150w, https://svpow.files.wordpress.com/2022/11/fossil-great-white-tooth-2-lingual-side-in-hand.jpg?w=300 300w, https://svpow.files.wordpress.com/2022/11/fossil-great-white-tooth-2-lingual-side-in-hand.jpg?w=768 768w"
sizes="(max-width: 480px) 100vw, 480px"}](https://svpow.files.wordpress.com/2022/11/fossil-great-white-tooth-2-lingual-side-in-hand.jpg)

The labial (outer or lip-facing) sides of the same teeth.
:::

I got this for a couple of reasons. One, I wanted a real great white
shark tooth to show people alongside my megalodon tooth (for which see
the [previous
post](https://svpow.com/2022/11/05/matts-first-megalodon-tooth/)).
Extant great whites are quite rightly protected, and their teeth are
outside my price range when they are available at all. Fortunately there
are zillions of fossil great white teeth to be had.

Also, the cast great white tooth has been kind of a disappointment. It's
so white that it's actually a letdown, visually. Tactilely it's great,
with all kinds of subtle features on the crown especially, but those
features are almost impossible to see or photograph. In the photo above,
you can make out some of the long, smooth wrinkles in the enamel of the
cast tooth, but the median ridge, which is dead obvious on the fossil
tooth, only shows up under very low-angle lighting on the cast. The
fossil tooth is just a more interesting and more informative specimen,
material and origin aside. Now that I have it, I might try either
staining or painting the cast tooth, to see if I can rehabilitate it as
a visual object.

[![](https://svpow.files.wordpress.com/2022/11/fossil-great-white-tooth-3-lateral-view.jpg?w=480){.size-large
.wp-image-20407 .aligncenter loading="lazy" attachment-id="20407"
permalink="http://svpow.com/2022/11/06/shark-week-2-great-white-teeth-redux/fossil-great-white-tooth-3-lateral-view/"
orig-file="https://svpow.files.wordpress.com/2022/11/fossil-great-white-tooth-3-lateral-view.jpg"
orig-size="2500,1875" comments-opened="1"
image-meta="{\"aperture\":\"0\",\"credit\":\"\",\"camera\":\"\",\"caption\":\"\",\"created_timestamp\":\"0\",\"copyright\":\"\",\"focal_length\":\"0\",\"iso\":\"0\",\"shutter_speed\":\"0\",\"title\":\"\",\"orientation\":\"0\"}"
image-title="Fossil great white tooth 3 – lateral view"
image-description="" image-caption=""
medium-file="https://svpow.files.wordpress.com/2022/11/fossil-great-white-tooth-3-lateral-view.jpg?w=300"
large-file="https://svpow.files.wordpress.com/2022/11/fossil-great-white-tooth-3-lateral-view.jpg?w=1024"
width="480" height="360"
srcset="https://svpow.files.wordpress.com/2022/11/fossil-great-white-tooth-3-lateral-view.jpg?w=480 480w, https://svpow.files.wordpress.com/2022/11/fossil-great-white-tooth-3-lateral-view.jpg?w=960 960w, https://svpow.files.wordpress.com/2022/11/fossil-great-white-tooth-3-lateral-view.jpg?w=150 150w, https://svpow.files.wordpress.com/2022/11/fossil-great-white-tooth-3-lateral-view.jpg?w=300 300w, https://svpow.files.wordpress.com/2022/11/fossil-great-white-tooth-3-lateral-view.jpg?w=768 768w"
sizes="(max-width: 480px) 100vw, 480px"}](https://svpow.files.wordpress.com/2022/11/fossil-great-white-tooth-3-lateral-view.jpg)

This fossil tooth is also noticeably thicker than the cast tooth. I
don't know if that's serial, individual, population, or evolutionary
variation. In the [last
post](https://svpow.com/2022/11/05/matts-first-megalodon-tooth/) I
contrasted the proportional thinness of the cast tooth with the
robustness of the megalodon tooth; this fossil tooth might fare a little
better if subjected to the same comparison. I should have thought to do
that when I was taking these photos.

[![](https://svpow.files.wordpress.com/2022/11/fossil-great-white-tooth-4-megalodon-comparison.jpg?w=480){.size-large
.wp-image-20408 .aligncenter loading="lazy" attachment-id="20408"
permalink="http://svpow.com/2022/11/06/shark-week-2-great-white-teeth-redux/fossil-great-white-tooth-4-megalodon-comparison/"
orig-file="https://svpow.files.wordpress.com/2022/11/fossil-great-white-tooth-4-megalodon-comparison.jpg"
orig-size="3200,2400" comments-opened="1"
image-meta="{\"aperture\":\"0\",\"credit\":\"\",\"camera\":\"\",\"caption\":\"\",\"created_timestamp\":\"0\",\"copyright\":\"\",\"focal_length\":\"0\",\"iso\":\"0\",\"shutter_speed\":\"0\",\"title\":\"\",\"orientation\":\"0\"}"
image-title="Fossil great white tooth 4 – megalodon comparison"
image-description="" image-caption=""
medium-file="https://svpow.files.wordpress.com/2022/11/fossil-great-white-tooth-4-megalodon-comparison.jpg?w=300"
large-file="https://svpow.files.wordpress.com/2022/11/fossil-great-white-tooth-4-megalodon-comparison.jpg?w=1024"
width="480" height="360"
srcset="https://svpow.files.wordpress.com/2022/11/fossil-great-white-tooth-4-megalodon-comparison.jpg?w=480 480w, https://svpow.files.wordpress.com/2022/11/fossil-great-white-tooth-4-megalodon-comparison.jpg?w=960 960w, https://svpow.files.wordpress.com/2022/11/fossil-great-white-tooth-4-megalodon-comparison.jpg?w=150 150w, https://svpow.files.wordpress.com/2022/11/fossil-great-white-tooth-4-megalodon-comparison.jpg?w=300 300w, https://svpow.files.wordpress.com/2022/11/fossil-great-white-tooth-4-megalodon-comparison.jpg?w=768 768w"
sizes="(max-width: 480px) 100vw, 480px"}](https://svpow.files.wordpress.com/2022/11/fossil-great-white-tooth-4-megalodon-comparison.jpg)

Speaking of comparisons, here's megalodon to remind everyone who's boss.
There's no scale bar here, but the cast great white tooth is 65mm from
the tip of the crown to the tip of the longer root, and the meg tooth is
155mm between the same points.

Now I have a gleam in my eye of assembling a couple of sets of fossil
teeth: one to illustrate the evolution of the modern great white from
its less-serrated ancestors, like [this
diagram](https://en.wikipedia.org/wiki/Great_white_shark#/media/File:Evolution_of_the_great_white_shark.jpg)
from the great white Wikipedia page, and one to illustrate the evolution
of megalodon from its side-cusped ancestors, like [this
diagram](https://en.wikipedia.org/wiki/File:Otodus_evolution.webp) from
the megalodon page --- presuming that current hypotheses for the two
lineages are accurate. If I ever get either set done, I'm sure I'll yap
about it here.


{.wp-image-20405 .size-large aria-describedby=“caption-attachment-20405” loading=“lazy” attachment-id=“20405” permalink=“http://svpow.com/2022/11/06/shark-week-2-great-white-teeth-redux/fossil-great-white-tooth-1-labial-side-in-hand/” orig-file=“https://svpow.files.wordpress.com/2022/11/fossil-great-white-tooth-1-labial-side-in-hand.jpg” orig-size=“3000,2250” comments-opened=“1”

Shark week 2: great white teeth redux

1140.fm

Body, jaw, and dentition lengths of macrophagous lamniform sharks, and body size evolution in Lamniformes with special reference to ‘off-the-scale’ gigantism of the megatooth shark, Otodus megalodon

Messages de Rogue Scholar

The longer the context, the better? Unlimited Context Length in Megalodon

Shark week 4: megalodon tooth colonized by boring clams

Shark week 2: great white teeth redux

Matt’s first megalodon tooth