University of Glasgow

Roderic Page

I keep returning to the problem of viewing large graphs and trees, which
means my hard drive has accumulated lots of failed prototypes. Inspired
by some recent discussions on comparing taxonomic classifications I
decided to package one of these (wildly incomplete) prototypes up so
that I can document the idea and put the code somewhere safe.

> Very cool, thanks for sharing this\-- the tree diff is similar to what
> J Rees has been cooking up lately with his \'cl diff\' tool. I\'ll tag
> [@beckettws](https://twitter.com/beckettws?ref_src=twsrc%5Etfw) in
> here too so he can see potential crossover. The goal is autogenerate
> diffs like this as 1st step to mapping taxo name-to concept
>
> --- Nate Upham (@n8_upham) [December 28,
> 2021](https://twitter.com/n8_upham/status/1475834371131289608?ref_src=twsrc%5Etfw)

## Google Maps-like viewer

::: {.separator style="clear: both;"}
[![](https://blogger.googleusercontent.com/img/a/AVvXsEiGDVVKGmwRi0gn-kEtpxBCUEdK-WCLBSpmkALAUDHGeTPkTSICSYVgnsj5N7zUeUfQALfFFHJJCsfeFvRULKbmqxLz51rW5hp_11dVXh-FHrnlRA7RJTA7I82l7sERF5jAjlah0LyEheVayO9nAfHTGZDuw5rnCe9iEO3dHQmA8_5AFIlJJg=s400){border="0"
width="400" original-height="448"
original-width="500"}](https://blogger.googleusercontent.com/img/a/AVvXsEiGDVVKGmwRi0gn-kEtpxBCUEdK-WCLBSpmkALAUDHGeTPkTSICSYVgnsj5N7zUeUfQALfFFHJJCsfeFvRULKbmqxLz51rW5hp_11dVXh-FHrnlRA7RJTA7I82l7sERF5jAjlah0LyEheVayO9nAfHTGZDuw5rnCe9iEO3dHQmA8_5AFIlJJg=s500){style="display: block; padding: 1em 0; text-align: center; "}
:::

I\'ve created a simple viewer that uses a tiled map viewer (like Google
Maps) to display a large graph. The idea is to draw the entire graph
scaled to a 256 x 256 pixel tile. The graph is stored in a database that
supports geospatial queries, which means the queries to retrieve the
individual tiles need to display the graph at different levels of
resolution are simply bounding box queries to a database. I realise that
this description is cryptic at best. The GitHub repository
<https://github.com/rdmpage/gml-viewer> has more details and the code
itself. There\'s a lot to do, especially adding support for labels(!)
which presents some interesting challenges ([levels of
detail](https://en.wikipedia.org/wiki/Level_of_detail_(computer_graphics))
and
[generalization](https://en.wikipedia.org/wiki/Cartographic_generalization)).
The code doesn\'t do any layout of the graph itself, instead I\'ve used
the [yEd](https://www.yworks.com/products/yed) tool to compute the x,y
coordinates of the graph.

Since this exercise was inspired by a discussion of the [ASM Mammal
Diversity Database](https://www.mammaldiversity.org), the graph I\'ve
used for the demonstration above is the ASM classification of extant
mammals. I guess I need to solve the labelling issue fairly quickly!


I keep returning to the problem of viewing large graphs and trees, which means my hard drive has accumulated lots of failed prototypes. Inspired by some recent discussions on comparing taxonomic classifications I decided to package one of these (wildly incomplete) prototypes up so that I can document the idea and put the code somewhere safe.

Large graph viewer experiments

Playing a bit more with the Wikipedia mammal data, there are some
interesting patterns to note. The first is that rank the mammal pages by
size (here defined as the number of characters in the source for the
page) and plot size against rank then we get a graph that looks very
much like a [power law](http://en.wikipedia.org/wiki/Power_law):  

::: {style="text-align:center;"}
![pow1.png](http://lh6.ggpht.com/_Gct8lVAxKqQ/Sp1VLmmuD3I/AAAAAAAAAhs/sTPFaUVH-qI/pow1.png?imgmax=800){border="0"
width="400" height="442"}
:::

  
There are a few large pages on mammals (these are on the left), and lots
of small pages (the long tail on the right). If we do a log-log plot we
get this:  

::: {style="text-align:center;"}
![pow2.png](http://lh4.ggpht.com/_Gct8lVAxKqQ/Sp1Vlr7gbLI/AAAAAAAAAhw/QnvN6YU1tWY/pow2.png?imgmax=800){border="0"
width="400" height="442"}
:::

  
The straight line is characteristic of a power law. The dip at the far
right reflects the fact that Wikipedia pages have a minimum size (for
example, they must include a Taxobox). Now, this is a bit crude (I
should probably look at \"Power-law distributions in empirical data\"
[arXiv:0706.1062v2](http://arxiv.org/abs/0706.1062) before getting too
carried away), but power laws are characteristic of the link structure
of the web (a few big sites with huge numbers of links, huge numbers of
sites with few links), and indeed of at least parts of Wikipedia, such
as the Gene Wiki project (see
[doi:10.1371/journal.pbio.0060175](http://dx.doi.org/10.1371/journal.pbio.0060175)).  
  
In this context, the diagrams are showing that even if mammals are
\"charismatic megafauna\", most of them aren\'t that charismatic.
Wikipedia mammal pages are mostly small. This raises the question of
whether the high frequency in which Wikipedia mammal pages appeared in
the top of Google searches might be attributed to those large pages on
(presumably) charismatic mammals. If this were the case, then we\'d
expect that small pages wouldn\'t rank highly in Google searches. So, I
plotted page size against Google search rank for the Wikipedia mammal
pages:  

::: {style="text-align:center;"}
![sizexrank.png](http://lh5.ggpht.com/_Gct8lVAxKqQ/Sp1aMFsnTCI/AAAAAAAAAh0/ZRaCOqA5wQ4/sizexrank.png?imgmax=800){border="0"
width="400" height="451"}
:::

  
This is a [box plot](http://en.wikipedia.org/wiki/Box_and_whisker_plot),
where the grey boxes represent 50% of the distribution of page size (the
horizontal black line is the median), and extreme values are shown as
circles. Note that \"0\" is the highest rank (i.e., the first hit in
Google), and 9 is the lowest.  
  
While, not surprisingly, most large Wikipedia pages do well in Google
searches, and rarely are large pages low down the rankings, my sense is
that small pages can have any rank, from top (0) to bottom (9). If page
size (i.e., which is a crude measure of the effort put into editing a
Wikipedia page) is a measure of \"charisma\" (contributors are more
likely to edit pages on animals that lots of people know about), then
charisma isn\'t a great predictor of where you come in Google\'s search
results. It\'s not about size, it\'s about being in Wikipedia.  


Playing a bit more with the Wikipedia mammal data, there are some interesting patterns to note.

Wikipedia mammals and the power law

::: {style="float:right"}
:::

  
  
Continuing the saga of making sense of the mammal classification in
Wikipedia, I\'ve done a quick comparison with the Mammal Species of the
World (third edition) classification. MSW is the default taxonomic
reference used by [WikiProject
Mammals](http://en.wikipedia.org/wiki/Wikipedia:WikiProject_Mammals#Taxonomy).
I downloaded the MSW taxonomy [as a CSV
file](http://www.bucknell.edu/MSW3/) (warning, it\'s big), and wrote a
script to pull out the classification as a [GML
file](http://bioguid.info/demos/msw/msw.gml) (my preferred graph
format).  
  
Based on some earlier work with Gabriel Valiente, I wrote a simple
program that takes two trees and highlights the nodes in common to the
two trees. I then input into this program the MSW tree, and the largest
component of the [graph of Wikipedia
mammals](http://iphylo.blogspot.com/2009/08/mammal-tree-from-wikipedia.html).
The MSW tree has 13582 nodes, the Wikipedia tree has 6287. Note that
Wikipedia has more taxa than these 6287 nodes suggest, but they aren\'t
connected to the largest tree (often due to intermediate nodes in the
classification lacking a page in Wikipedia). The two trees have 4935
nodes in common (again, this number will be a little low, there are some
weird taxon names due to problems parsing Wikipedia).  
  
**MSW versus Wikipedia**  
Below is a the MSW classification with taxa in Wikipedia shown in red.  

::: {style="text-align:center;"}
![w-msw.jpg](http://lh4.ggpht.com/_Gct8lVAxKqQ/SpwEcr5t4bI/AAAAAAAAAhY/p3bXj9Xdh-k/w-msw.jpg?imgmax=800){border="0"
width="425" height="478"}
:::

  
  
\[Larger scale view
[here](http://bioguid.info/demos/msw/w/w-msw.html)\]  
  
The impression given is that most Wikipedia mammal pages are in MSW,
with some notable exceptions, including higher level taxa such as
[Afrotheria](http://en.wikipedia.org/wiki/Afrotheria), and extinct taxa
such as the
[Multituberculata](http://en.wikipedia.org/wiki/Multituberculata). Some
extant taxa are missing due to synonymy. For example, Wikipedia gives
the scientific name of [Anthony\'s
pipistrelle](http://en.wikipedia.org/wiki/Anthony%27s_pipistrelle) as
*Pipistrellus anthonyi*, whereas MSW has it as *Hypsugo anthonyi*.  

::: {style="font-size:12px"}
As an aside, Wikipedia pages often get muddled about parentheses around
taxonomic author names. The authority is in parentheses if the current
genus is not the original genus the species was placed. Hence,
*Pipistrellus anthonyi* (Tate, 1942) should actually be *Pipistrellus
anthonyi* Tate, 1942, as Tate originally described this taxon as a
species of *Pipistrellus* (see
[hdl:2246/1783](hdl.handle.net/2246/1783)). However, the name *Hypsugo
anthonyi* (Tate, 1942) does need parentheses.
:::

  
  
Some Wikipedia taxa also postdate the publication of MSW, such as
[Philander deltae](http://en.wikipedia.org/wiki/Philander_deltae) (see
[doi:10.1644/05-MAMM-A-065R2.1](http://dx.doi.org/doi:10.1644/05-MAMM-A-065R2.1)).  
  
  
**Wikipedia versus MSW**  
When we do the reverse comparison we see something rather different.  
  

::: {style="text-align:center;"}
![msw-w.jpg](http://lh3.ggpht.com/_Gct8lVAxKqQ/SpwFHQuJEZI/AAAAAAAAAhc/-0Pln00c3OQ/msw-w.jpg?imgmax=800){border="0"
width="425" height="438"}
:::

  
  
\[Larger scale view
[here](http://bioguid.info/demos/msw/m/msw-w.html)\]  
  
This is the MSW tree, coloured red where the MSW taxon has a page in
Wikipedia. There are big gaps, some of which are due to those pages
being in another component (in other words, many \"missing\" taxa do
have pages in Wikipedia, they are just not properly linked to the bigger
tree). MSW is also rich in subspecies, which tend to lack their own
pages in Wikipedia (possibly a good thing in the cases of taxa such as
pocket gophers).  
  
It would be nice to make these comparisons automatic, and develop tools
so that managing taxonomy in Wikipedia could be made easier.  


Continuing the saga of making sense of the mammal classification in Wikipedia, I've done a quick comparison with the Mammal Species of the World (third edition) classification. MSW is the default taxonomic reference used by WikiProject Mammals.

Comparing Wikipedia and Mammal Species of the World classifications

Following on from my previous post about [visualising the mammalian
classification in
Wikipedia](http://iphylo.blogspot.com/2009/08/visualising-wikipedia-classification-of.html),
I\'ve extracted the largest component from the graph for all mammal taxa
in Wikipedia, and it is a tree. This wasn\'t apparent in the previous
diagram, where the component appeared as a big ball due to the layout
algorithm used.  

::: {style="text-align:center;"}
![tree.jpg](http://lh5.ggpht.com/_Gct8lVAxKqQ/Spl5YxJ8zTI/AAAAAAAAAhQ/NWQnlsJy92o/tree.jpg?imgmax=800){border="0"
width="425" height="500"}
:::

  
  
What this suggests is that Wikipedia contributors are quite capable of
generating trees, it\'s just that not all the bits of the tree are
connected (hence all the components in the [previous
post](http://iphylo.blogspot.com/2009/08/visualising-wikipedia-classification-of.html).  
  
As Cyndy Parr suggested in her comments, it would be useful to compare
the Wikipedia-derived tree with other trees, say from Mammal species of
the World or ITIS.


Following on from my previous post about visualising the mammalian classification in Wikipedia, I've extracted the largest component from the graph for all mammal taxa in Wikipedia, and it is a tree. This wasn't apparent in the previous diagram, where the component appeared as a big ball due to the layout algorithm used.

Rogue Scholar Beiträge

Large graph viewer experiments

Wikipedia mammals and the power law

Comparing Wikipedia and Mammal Species of the World classifications

Mammal tree from Wikipedia