Juan Carlos Graciosa

Introduction and benchmarking

The convection of the Earth’s mantle is usually modelled as an incompressible process, referred to as the Boussinesq approximation. However, in the Earth’s mantle, the pressure increase associated with depth also increases the density due to self-compression (King et al. 2010). In some applications, this compressibility may be non-negligible and modelling it may be desirable.

Compressible convection in cartesian coordinates with Underworld3

Romain Beucher

Conda packages are now available for most of the underworldcode suite of software


 (Underworld, UWGeodynamics, Lavavu, Stripy) from the geo-down-under conda channel.

Let’s face it. Users don’t want to spend time compiling code on their machine. It is time consuming, requires some specific skills and creates unnecessary hurdles in a workflow.

Here comes Conda...

Haibin Yang

A Lagrangian integration point finite element method for large deformation modeling of viscoelastic geomaterials

Abstract   Numerical models have become an indispensable tool for understanding and predicting the flow of ice sheets and glaciers. Here we present the full-Stokes software package Underworld to the glaciological community. The code is already well established in simulating complex geodynamic systems. Advantages for glaciology are that it provides a full-Stokes solution for elastic–viscous–plastic materials and includes mechanical anisotropy.

ISMIP-HOM benchmark experiments using Underworld

Louis Moresi

Meghan S Miller, Louis Moresi, Research School of Earth Sciences, Australian National University


 A new study by an international team of scientists has found lockdown measures to stop the spread of COVID-19 led to a 50 per cent reduction in seismic noise observed around the world.

It is the largest reduction in human-generated noise ever observed globally.

COVID-19 lockdown leads to seismic noise “quiet period”

Meghan S. Miller,

 Australian National University

and Louis Moresi,

 Australian National University


 
 How we built a simple dashboard using Github actions with open source software and openly available (FAIR) data.
 

We recently wrote an article in The Conversation that shows how the Australian Seismometers in schools network registers the pulse of Australian life through changes in the seismic noise

Australian Seismometers in Schools - Noise monitoring dashboard

Julian Giordani

The Underworld team would like to congratulate Bénédicte Cenki-Tok and co-authors at the University of Sydney and the University of Montpellier on their recent publication,  B. Cenki-Tok, P.F. Rey, D. Arcay;

Underworld publication news: Crustal thickness anomalies in stable continents.

Meghan S. Miller,

 Australian National University

and Louis Moresi,

 Australian National University

Our responses to the COVID-19 pandemic have dramatically changed human activity all over the world. People are working from home, schools are closed in many places, travel is restricted, and in some cases only essential shops and businesses are open. Scientists see signs of these changes wherever they look.

Australian cities are quiet during lockdown. Earthquake scientists are making the most of it

John Mansour

A new paper by John Mansour and others has just been published in the Journal of Open Source Software. Mansour, J., Giordani, J., Moresi, L., Beucher, R., Kaluza, O.,  Velic,  M., Farrington, R., Quenette, S., Beall, A., 2020, Underworld2:  Python  Geodynamics Modelling for Desktop, HPC and Cloud, Journal of Open

JOSS publication - Underworld 2

Rohan Byrne

As the old truism states, the best way to learn is by doing. This is terribly unhelpful since, really, you can't 'do' anything

 until

you learn. This is a Catch-22 that every grad student encounters; doubly so when it comes to anything involving code, where daily life provides very little in the way of intuition. Unlike other geodynamic modelling software,

 Underworld

is designed to be approached with code.

Getting started: 60 seconds to Underworld

Rebecca Farrington

Viscoelastic materials exhibit the properies of both solids and liquids, with deformation rates dependent on both the viscous stress and elastic stress rate.

Viscoelasticity in Underworld2

Making your research reproducible means that you provide the entire workflow from data, through software and post-processing freely available. Not only can somebody repeat your experiments and verify them, they can build upon them. In lab-based disciplines, there are many further challenges, but in research that is predominantly based on data processing, this ought to be an achievable goal.

Alaska Moho Model (Reproducible research with containers)

Zenodo is a repository for immutable versions of software that are provided with a persistent DOI for the purposes of citation and reproducibility.   Underworld can be cited via a zenodo DOI. There is a master DOI for all releases (10.5281/zenodo.1436039) and releases after 2.6.0 are automatically given a DOI _under the master_.

Cite Underworld from Zenodo

(they don't make them like they used to ... )

Cratons are anomalously-strong regions of the continents that have largely resisted tectonic forces for billions of years. How such strong zones could be forged in a hot, low-viscosity, low stress,  early-Earth has been a long-standing puzzle for geologists.

Craton Formation and the Onset of Plate Tectonics

Using physical units and how to appropriately scale a model is a top question users ask when beginning with Underworld. The equations Underworld solves are stated in a physically correct form, they remain valid as long as every material constant, geometry, time, etc., are expressed in the same system.  There is no explicit concept of units no scaling in Underworld, the choice is entirely left to the user.

Using physical units in Underworld

(aka the Underworld renovation project)

Underworld is our parallel, particle-in-cell, finite element geodynamics code [1]. For the past year or so, the Underworld team has been working on a refurbished user interface. We've known for a long time that it needed to be done but we finally bit the bullet.

Underworld 2

In a recent paper [1], we used Underworld models to examine subduction congestion associated with the ingestion of a continental ribbon. The SE Australian geological record turned out to be a wonderful place to study this process. Here is a short summary of the work for a relatively non-technical audience that we put together and some additional figures which I prepared.

The Dynamics of Continental Accretion

The Underworld Community has been supporting usage of Docker containers for developing and running Underworld powered scientific models.

Developing Underworld using Python Virtual Environments

The new generation of Apple Mac comes with the new Apple Silicon (M1) chip which has an Arm architecture (as opposed to the older generation that had i386 Intel processor). This brings all manner of troubles and requirements for the development of codes. Here I detail how to install Underworld on Mac OSX Big Sur 11.5.2 (Apple Silicon) using Python 3.9  Important Note: The following guide uses the CMake branch of Underworld.

How to install Underworld on Mac OSX (Apple Silicon M1)

How we added an auto-updating set of citations to underworld in our publications webpage.

We need to curate all the publications that we can find that use the underworld geodynamics code and provide this information on our website. To avoid needless repetition, we take advantage of the fact that nearly all the information we require is online and automatically construct the bibliography from an online public zotero library.

An automated (zotero) bibliography in a webpage

By the way, before you read this post, catch up with how we use docker by reading part 1  Docker allows us to distribute pre-built applications which are hosted in a virtual machine and are therefore platform independent.

Underworld and Docker (part 2)

How does Underworld scale on a HPC? In this post we showcase how Underworld 2.11 scales across two of Australia's premiere HPC systems. Gadi - https://nci.org.au/our-systems/hpc-systems Magnus - https://pawsey.org.au/systems/magnus/  The reference model chosen for this scaling showcase is a extended 3D stokes flow: Analytic Solution SolDB3D . Q1P0 elements were used and a fixed solver iteration count for solving the saddle point problem.

Underworld 2.11 Scaling

One of the most popular cloud services for python / jupyter - based codes is binder (www.mybinder.org) which can transform an online repository into a running instance in the cloud with little effort on the part of the owner of that repository to set it up and no effort at all for the end user.

Underworld's  lightweight cloud for online classrooms.

Modelling the relative time-scales of the Rayleigh-Taylor Instability and delamination, using Underworld

With Adam Beall, Cardiff University. Why model sub-continental gravitational instabilities? Within the plate tectonics framework, continents are generally considered to have a much lower density than the asthenosphere below and therefore avoid the kind of recycling that the oceanic crust and lithosphere undergoes.

Modelling Drips and Delamination with Underworld

In honour of Hans Mühlhaus' 70th birthday this month, here are some shear band simulations made with Underworld. We are investigating the role of dilatancy in the geometry of the shear bands for a box of material when a small trapdoor is opened. The extent to which large-scale deformation is needed to release material through the trapdoor depends on how much the material dilates when shear bands form.

Shear Bands with Dilatancy modelled with Underworld

Underworld2 is a python-friendly version of the Underworld geodynamics code which provides a programmable and flexible front end to all the functionality of the code running in a parallel HPC environment. This gives signficant advantages to the user, with access to the power of python libraries for setup of complex problems, analysis at runtime, problem steering, and coupling of multiple problems.

Underworld2 Workshop at CIG 2016 Meeting

While huge improvements in usability have been achieved in Underworld2, the installation process is still unfortunately as painful as ever. This difficulty is in large part due to Underworld's numerous dependencies, and also the multiple platforms we try to support. Compounding this, the legacy of a individual user's machine often conspires against success, rarely in obvious ways. Simply, compiling is not fun. Enter docker.

Underworld and Docker (part 1)

https://doi.org/10.59350/t7ghx-8f823 The following is compatible with Ubuntu 20.04 under

 
 Windows
 

10/11 WSL 2 (

 
 Windows
 

Subsystem for Linux). PETSc, the Portable, Extensible Toolkit for Scientific Computation is the main dependency required for building Underworld. In the following, I will describe my current workflow for configuring and installing PETSc on Linux.

Configuring and Installing PETSc for Underworld

Version control systems and online-communities such as Github have made contributing to the development of opensource software incredibly easy. That is assuming that people get over their initial fear of sharing their code!  In this short blog post I go through how to submit a pull request to one of our repository. Do not fear and join us for some fun!

Getting started with Pull requests

Version 2.8 of Underworld has been released recently. As with all major releases, this release brings numerous new features, enhancements and bug fixes. A summary of changes may be found within the usual CHANGES.md file. As is also usually the case, numerous API changes have been necessary or warranted.

Underworld Release&nbsp;2.8

Computational approaches to studying non-linear dynamics of the crust and mantle

Numerical investigation of 2D convection with extremely large viscosity variations

The accuracy of finite element solutions of Stokes's flow with strongly varying viscosity

Role of temperature‐dependent viscosity and surface plates in spherical shell models of mantle convection

We mark January 17, 1994 (the day better known for the Northridge Earthquake) as the birthday of the CITCOM family of codes. I was shaken awake early in the morning, and spent the rest of the day nervously waiting for the next aftershock while I checked that the code was really solving all the test problems quickly and accurately.

30 Years of Citcom, Ellipsis and Underworld

Yeah, one repo, one vision! In an effort to simplify maintenance and compatibility between Underworld and UWGeodynamics, we have decided to merge the codes into a single  repository. Starting with version 2.13, UWGeodynamics will now live under Underworld. All UWGeodynamics functionalities and workflows will remain available to the users.

Folding UWGeodynamics into Underworld

Neng Lu

Free surface in geodynamics simulations

Geodynamic simulations increasingly rely on models with a true free surface to investigate questions of tectonic deformation, mantle convection, and coupling of surface processes and lithosphere dynamics. Historically, most mantle convection simulations have been performed with free-slip boundary conditions at the surface.

Free surface in Underworld

To date

 weak scaling

tests have been run on two of the largest computers in Australia:

 Gadi

(NCI) and

 Magnus

(Pawsey). Here we present the results of those tests and discuss:

 Gadi: Weak scaling - SolDB3D Q1


 Gadi: Weak scaling - SolDB3D Q2


 Magnus: Weak scaling - SolDB3D Q2 - v2.10 vs v2.9

Underworld's

 Gadi

installation is setup as a

Scaling in Underworld

Issues with intra-element discontinuities in PIC-FEM   The classical finite element method (FEM) has been widely used to simulate diverse problems in engineering. Unlike most engineering problems, geological simulations are dominated by emergence of geometrical structures due to the non-linear processes involved.

Stress recovery in Underworld

As mentioned, packages can be installed directly from our conda channel. You may however want to build your own package at some point in the future or you may want to fix some of ours by submitting a Pull Request on github (You are more welcome to do so...)  The following goes through the steps of building a conda package for Badlands which contains Python and Fortran code. This is relatively advanced but not really difficult.

How to build Conda packages?

Underworld 2.10 has been released! Available via

 docker

(recommended),

 pip

and

 source code

. See here for more information on each install process. For a

 quick

taste of Underworld try our binder cloud resource, a temporary "virtual sandbox" to explore the release and example models. Click the button below. This new release includes the following: Enhancements Using UWGeodynamics-2.10.1, see here.

Underworld 2.10

Ben Mather

We've been busy creating the next major release of Stripy. To refresh your memory, Stripy is a Python tool for triangulating scattered points either in Cartesian coordinates or on the sphere. It wraps a bunch of Fortran codes in a neat, object-oriented Python interface that can be used for many geographical applications. What's new?

 Spline tension

- a lot of data transformations in Stripy are underpinned by cubic splines (e.g.

Stripy 2.0 released

The Underworld 2.9 release is available from Github, as a docker container and via zenodo (doi:10.5281/zenodo.3964957) it is also available through pip install for the first time (see below). Underworld paper in the Journal of Open Source Software   The paper of record for underworld 2 has been published. It does tie to the Underworld 2.9 release but is sufficiently broad to cite for all 2.x releases.

Underworld 2.9

Underworld Geodynamics Community

The Underworld Community has been supporting usage of Docker containers for developing and running Underworld powered scientific models. They provide a good option to control the running environment (dependencies).

Docker container can be used for developing the code base but creating a suitable development workflow has proven challenging so far and I have opted for a more simple solution involving native builds of PETSc and Python Virtual Environment.

In the following I describe how I install Underworld (v2.12) in a Python virtual environment. This is by no way perfect and others may have a different approach, I always welcome suggestions!

## PETSc

The main dependency that we use is PETSc (the Portable, Extensible Toolkit for Scientific Computation). PETSc installation is not complicated but can be time-consuming depending on how many External Packages (Solvers etc.) are required. I suggest reading my other blogpost to learn what\'s needed and how to install PETSc on your machine.

PETSc tends to evolve quite fast and we thus need to make sure that we keep up to date with the changes introduced with each new version. We also need to be able to test for some sort of back-compatibility.\
My solution to this problem has been to compile different versions and configuration of PETSc on my machine. They are all located in the `/opt` directory and I can switch easily between versions as needed.

Here is a snapshot of my `/opt/` directory:

``` bash
3.13.6  3.14.6  3.15.5  3.16.1
```

## Python Virtual Environment

Python Virtual Environment provide an easy way to control your python packages stack. It is not the only way to achieve this, some other tools such as Conda and Module files can do the same thing and actually extend the idea to other non-python-based applications.\
So here again all roads lead to Rome and you just need to find what works for you.

To use virtual environments you first need to install the appropriate package on your machine. Note that it is good practice to use the package provided by your LINUX distribution.

On Arch Linux we use:

``` bash
sudo pacman -Syy
sudo pacman -S python-virtualenv
```

on Ubuntu

``` bash
sudo apt-get update
sudo apt-get install -y python3-venv
```

Virtual Environment are created using:

``` bash
python -m venv --name my_underworld_env
```

The virtual environment is then activated by sourcing the activate script as follow:

``` bash
source my_underworld_env/bin/activate
```

## Handling PETSc

The `activate` script that you source when activating a Python virtual environment contains a set of instructions that set the `PATH` and  `PYTHONHOME` environment variables.\
The trick to handle PETSc is to also let the `activate` script to manage the `PETSC_DIR` environment variable. The `PETSC_DIR` variable should point to the PETSc version you want to use with the Python virtual environment.

In the following I am modifying the script so that it creates:

-   PETSC_DIR (that points to the PETSc built I want to use)
-   PYTHONPATH (points to `$PETSC_DIR/lib` )
-   PATH (points to `$PETSC_DIR/bin` where I have `mpirun` etc.)

I also modify the `deactivate` function so that variables are reset when we deactivate the virtual environment.

``` file
# This file must be used with "source bin/activate" *from bash*
# you cannot run it directly

deactivate () {
    # reset old environment variables
    if [ -n "${_OLD_VIRTUAL_PATH:-}" ] ; then
        PATH="${_OLD_VIRTUAL_PATH:-}"
        export PATH
        unset _OLD_VIRTUAL_PATH
    fi
    if [ -n "${_OLD_VIRTUAL_PYTHONHOME:-}" ] ; then
        PYTHONHOME="${_OLD_VIRTUAL_PYTHONHOME:-}"
        export PYTHONHOME
        unset _OLD_VIRTUAL_PYTHONHOME
    fi
    
    if [ -n "${_OLD_PYTHONPATH:-}" ] ; then
        PYTHONPATH="${_OLD_PYTHONPATH:-}"
        export PYTHONPATH
        unset _OLD_PYTHONPATH
    fi
    if [ -n "${_OLD_PETSC_DIR:-}" ] ; then
        PETSC_DIR="${_OLD_PETSC_DIR:-}"
        export PETSC_DIR
        unset _OLD_PETSC_DIR
    fi

    # This should detect bash and zsh, which have a hash command that must
    # be called to get it to forget past commands.  Without forgetting
    # past commands the $PATH changes we made may not be respected
    if [ -n "${BASH:-}" -o -n "${ZSH_VERSION:-}" ] ; then
        hash -r 2> /dev/null
    fi

    if [ -n "${_OLD_VIRTUAL_PS1:-}" ] ; then
        PS1="${_OLD_VIRTUAL_PS1:-}"
        export PS1
        unset _OLD_VIRTUAL_PS1
    fi

    unset VIRTUAL_ENV
    if [ ! "${1:-}" = "nondestructive" ] ; then
    # Self destruct!
        unset -f deactivate
    fi
}

# unset irrelevant variables
deactivate nondestructive

VIRTUAL_ENV="/home/romain/my_underworld_venv"
export VIRTUAL_ENV

_OLD_VIRTUAL_PATH="$PATH"
PATH="/opt/petsc/3.16.1/bin:$VIRTUAL_ENV/bin:$PATH"
export PATH

# unset PYTHONHOME if set
# this will fail if PYTHONHOME is set to the empty string (which is bad anyway)
# could use `if (set -u; : $PYTHONHOME) ;` in bash
if [ -n "${PYTHONHOME:-}" ] ; then
    _OLD_VIRTUAL_PYTHONHOME="${PYTHONHOME:-}"
    unset PYTHONHOME
fi

if [ -z "${VIRTUAL_ENV_DISABLE_PROMPT:-}" ] ; then
    _OLD_VIRTUAL_PS1="${PS1:-}"
    PS1="(uw2_venv) ${PS1:-}"
    export PS1
fi

# This should detect bash and zsh, which have a hash command that must
# be called to get it to forget past commands.  Without forgetting
# past commands the $PATH changes we made may not be respected
if [ -n "${BASH:-}" -o -n "${ZSH_VERSION:-}" ] ; then
    hash -r 2> /dev/null
fi

_OLD_PYTHONPATH="$PYTHONPATH"
PYTHONPATH=/opt/petsc/3.16.1/lib
export PYTHONPATH

_OLD_PETSC_DIR="$PETSC_DIR"
PETSC_DIR=/opt/petsc/3.16.1 
export PETSC_DIR
```

Now sourcing the activate script sets `PETSC_DIR` `PATH` and `PYTHONPATH` to the appropriate values. Running `deactivate` takes us back to normal. We are ready to install Underworld and the python packages we need!

## 

### Installing Underworld in development mode

Underworld requires some building dependencies.

**Ubuntu:**

``` bash
apt-get update -qq 
apt-get install \
        build-essential \
        pkg-config \
        python3-dev \
        cmake \
        make \
        swig \
        libxml2-dev \
        g++
```

### Installing h5py

After activating the Python virtual Environment so that the install process uses the appropriate version og HDF5 and H5PCC

``` bash
source my_underworld_venv/bin/activate

CC=h5pcc HDF5_MPI="ON" HDF5_DIR=${PETSC_DIR} pip3 install --no-cache-dir --no-binary=h5py git+https://github.com/h5py/h5py@master
```

Get the current development branch of Underworld from GitHub

``` bash
source my_underworld_venv/bin/activate

git clone git@github.com:underworldcode/underworld2.git
cd underworld2
git checkout development
```

Then build Underworld

``` bash
# From the underworld directory
pip install -vvv -e .
```

You should now have a Python virtual environment with a working version of Underworld

# Conclusion

Python Virtual Environment can be useful for Underworld development. You can easily configure them so that they  point to a specific version of PETSc on your machine.\
This make possible to test the code against different versions of PETSc, run tests and try different optimization options or configurations of the software stack.
