The following instructions assume that you have installed SPECFEM3D and familiarized yourself with how you will run the package based on your computer configuration, as detailed in the SPECFEM3D User manual (Chapter 2 provides installation help). Additionally, we will make use of an external, hexahedral mesher CUBIT. Please make sure you have these packages installed on your system.

The example is distributed with the package under the examples/ directory. However, you might need to edit these example scripts slightly to launch them on your system.

Waterlayered halfspace

This is a step-by-step tutorial how to create a mesh for a layered halfspace with a water layer on top, export it into a SPECFEM3D file format and run the mesh partitioning and database generation.

Meshing

In the following, we will run a python script within CUBIT to create the needed mesh files for the partitioner.

1. change your working directory to the example folder:

   []$ cd SPECFEM3D/examples/waterlayered_halfspace
   

2. start the graphical user interface (GUI) of CUBIT:

   [waterlayered_halfspace]$ claro
   

   
3. run the python file “ block_mesh.py” :

   use the (Play Journal File) button and open the file "waterlayer_mesh_boundary_fig8.py"
   

4. in case the script executed successfully, it should create you three volumes with a refined mesh for the top layer, a triplication layer and a coarser mesh layer at the bottom: (Image(cubit_waterlayered.jp) failed - File not found) check that the script created a new folder “MESH/” containing the files:

   [waterlayered_halfspace]$ ls -1 MESH/
   absorbing_surface_file_bottom
   absorbing_surface_file_xmax
   absorbing_surface_file_xmin
   absorbing_surface_file_ymax
   absorbing_surface_file_ymin
   free_surface_file
   materials_file
   mesh_file
   nodes_coords_file
   nummaterial_velocity_file
   

   you can also check if the export was successful by examining the output in the Command line window of CUBIT. The sensitive part should look like
5. optionally, you could modify the material properties of the model, either in CUBIT or in the newly created material file:
   • changing the block properties in CUBIT

the following table applies for acoustic material properties:

the following table applies for elastic material properties:

directly modifying the file nummaterial_velocity_file in the MESH/ directory:

1   1          1028.000000          1480.000000             0.000000             0.000000  0
2   2          3200.000000          7500.000000          4300.000000          9000.000000  0
2   3          3200.000000          7500.000000          4300.000000          9000.000000  0

this file defines the material properties. the format is:

Domain_id	Material_id	Density	P-wave speed	S-wave speed	Quality factor	Anisotropy_flag
1 = acoustic	unique material	density	wave speed in m/s	wave speed in m/s	Quality factor	flag for anisotropy model
2 = elastic	identifier	in kg/m3	(ignored for acoustic materials)		0 = no attenuation	0 = no anisotropy

Setting up example folder for simulations

We will set up the example folder for simulation runs:

* databases directory: create a directory in_out_files/DATABASES_MPI/ into which you will put the mesh partitions:

[]$ cd SPECFEM3D/examples/waterlayered_halfspace
[waterlayered_halfspace]$ mkdir -p in_out_files/DATABASES_MPI

* parameter files: make sure you have the parameter files in a local directory in_data_files/ for the example:

these files should already be provided in the example folder.

* executables: compile the executables in the root directory:

[]$ cd SPECFEM3D/
[SPECFEM3D]$ make

in case the compilation was successful, it will create the executables xdecompose_mesh_SCOTCH, xgenerate_databases and xspecfem3D in the bin/ directory

and create a local directory bin/ to link the executables from the root directory:

[]$ cd SPECFEM3D/examples/waterlayered_halfspace
[waterlayered_halfspace]$ mkdir -p bin    
[waterlayered_halfspace]$ cd bin/
[bin]$ ln -s ../../../bin/xdecompose_mesh_SCOTCH
[bin]$ ln -s ../../../bin/xgenerate_databases
[bin]$ ln -s ../../../bin/xspecfem3D

All these steps and the following decomposition, database generation and solver run are put in a process.sh bash script file in the example folder. You can simply run the script:

[waterlayered_halfspace]$ ./process.sh

to do the setup and following steps for you. Please modify and adapt the script to your needs.

Mesh partitioning

In this example, we will partition the mesh for 4 CPU cores.

run the mesh partitioner:

[waterlayered_halfspace]$ ./bin/xdecompose_mesh_SCOTCH 4 MESH/ in_out_files/DATABASES_MPI/

check that this created the mesh partitions:

[waterlayered_halfspace]$ ls -1 in_out_files/DATABASES_MPI/
proc000000_Database
proc000001_Database
proc000002_Database
proc000003_Database 

You are done.

Database generation

Next, you will need to create the mesh databases.

1. in case you can run parallel programs on your desktop (needs an MPI installation), you can run the executable like:

   [waterlayered_halfspace]$ cd bin/
   [bin]$ mpirun -np 4 ./xgenerate_databases
   

   otherwise, you will need to modify and adapt one of the cluster scripts provided in the SPECFEM3D/utils/Cluster/ directory.
2. check the output file in_out_files/OUTPUT_FILES/output_mesher.txt to see if the databases were generated successfully.

   The output file contains the suggested time step for your mesh:
   Verification of simulation parameters 
   ...
   Minimum period resolved =   1.632425 
   **Maximum suggested time step =   5.1801954E-03**
   

Forward simulation

To run a forward simulation, do the following:

1. make sure, you have the parameter files in the directory in_data_files/. Most parameters in the Par_file should be set before running the database generation. The following may be changed after running xgenerate_databases:

   # forward or adjoint simulation
   SIMULATION_TYPE                 = 1   # 1 = forward, 2 = adjoint, 3 = both simultaneously
   NOISE_TOMOGRAPHY                = 0   # 0 = earthquake simulation,  1/2/3 = three steps in noise simulation
   SAVE_FORWARD                    = .false.  
   # time step parameters
   NSTEP                           = 4500
   DT                              = 0.005
   # absorbing boundary conditions for a regional simulation
   ABSORBING_CONDITIONS            = .true.
   # save AVS or OpenDX movies
   MOVIE_SURFACE                   = .false.
   MOVIE_VOLUME                    = .false.
   NTSTEP_BETWEEN_FRAMES           = 200
   CREATE_SHAKEMAP                 = .false.
   SAVE_DISPLACEMENT               = .false.
   USE_HIGHRES_FOR_MOVIES          = .false.
   HDUR_MOVIE                      = 0.0
   # interval at which we output time step info and max of norm of displacement
   NTSTEP_BETWEEN_OUTPUT_INFO      = 500
   # interval in time steps for writing of seismograms
   NTSTEP_BETWEEN_OUTPUT_SEISMOS   = 10000
   # interval in time steps for reading adjoint traces
   NTSTEP_BETWEEN_READ_ADJSRC      = 0      # 0 = read the whole adjoint sources at the same time
   # print source time function
   PRINT_SOURCE_TIME_FUNCTION      = .false.
   

2. in case you can run parallel programs on your desktop (needs an MPI installation), you can run the executable like:

   [waterlayered_halfspace]$ cd bin/
   [bin]$ mpirun -np 4 ./xspecfem3D
   

   note, this example should take about 1 hour 45 minutes to finish the simulation.
3. check the output file output_solver.txt in the output directory in_out_files/OUTPUT_FILES/ to see if the forward simulation was successfully finishing.

   the seismograms will look like this, using gnuplot:
   gnuplot> plot "X10.DB.BXZ.semd" w l,"X20.DB.BXZ.semd" w l,"X30.DB.BXZ.semd" w l
   

   

Created on 02 Mar 2022, Last modified on 02 Mar 2022