In the next months, the new NSF-funded Petascale Computing System Frontera will open for early testing at the Texas Advanced Computing Center (TACC). Frontera will be approximately twice as powerful as the current XSEDE flagship Stampede2. It will provide a “leadership-class” computing resource to enable much larger and more complex research challenges than previously possible and push the boundaries of currently available numerical methods.
NSF has awarded CIG 0.27 million node hours to participate in Frontera’s early testing phase from May 2019 to January 2020, which will be used to benchmark the performance of existing CIG software and further its scalability using new numerical methods in preparation for wider community use. Specifically, we will focus on testing and developing the CIG codes ASPECT and Calypso for mantle and core dynamics, respectively. In addition to research on numerical methods to reduce scalability bottlenecks applicable to a range of CIG codes, we will also conduct “proof-of-concept” simulations that target outstanding geophysical questions that were intractable with the current generation of HPC systems.
Using Frontera’s resources and ASPECT’s adaptive mesh-refinement features will allow it for the first time to examine how 3D mantle convection patterns are influenced by a variable and evolving grain size, which can introduce significant additional viscosity variations within the convecting mantle and alter the dynamics of upwellings and downwellings (Figure 1).
Using Frontera’s resources for the study of dynamos, we will conduct scaling tests with Calypso ideally up to the 450,000 cores available. Crucial for this upscaling will be work on a new hybrid OpenMP/MPI parallelization scheme for both simulation and visualization. As the example science application, we will develop and test parameterizations of small scale turbulence effects on the large scale magnetic field, which can then be utilized in other smaller scale models (Figure 2).
As a community organization, CIG supports the testing and benchmarking of all its developed and supported codes. We encourage project maintainers who are interested in assessing the performance and scalability of their software on Frontera to contact CIG for a reasonable allocation of compute time.
Contributed by Juliane Dannberg, Rene Gasmoeller, Hiro Matsui, and John Naliboff
Dannberg, J., Eilon, Z., Faul, U., Gassmöller, R., Moulik, P. and Myhill, R., 2017. The importance of grain size to mantle dynamics and seismological observations. Geochemistry, Geophysics, Geosystems, 18(8), pp.3034-3061.
Figure 1. Effects of an evolving grain-size evolution on subduction zones and mantle plumes in existing 2D studies (Dannberg et al., 2017). Frontera’s resources will allow to investigate this process in 3D at the necessary resolution (< 5km).
Figure 2. 3-D line integral convolution (LIC) image of magnetic field line at intense magnetic field viewing from mid latitude (left) and stream line at the southern hemisphere looking from the north pole (right) obtained by geodynamo simulation. Frontera's resources will allow increasing the truncation degree of the spherical harmonics to L = 512 to resolve finer structures.