[CIG-SEISMO] Specfem Cartesian 2.1 Example issues

Tue Aug 4 02:26:24 PDT 2015

Dear Steve,

unfortunately, i can’t test it on a CentOS machine, but i once encountered the same problem on another system. this is likely a problem with your mpi installation. try to see first if you can run the command ‘mpirun’  with some simple test executable (attached here is such a test-file test_mpi.f90):

you can compile the test by:
> mpif90 test_mpi.f90

then try running it with:
>  mpirun -np 4 ./a.out

which should give you:
"
 hello from process:            0
 hello from process:            1
 hello from process:            2
 hello from process:            3
“

in your case, i would suspect that it complains that you’re attempting to run as root. see if you need to use the command ‘mpiexec’ instead of ‘mpirun’. this might be the easy solution. otherwise you will need to figure out how to do a proper OpenMPI installation on your system to use SPECFEM3D in parallel.

good luck,
daniel

________________________________

This message and its contents including attachments are intended solely for the original recipient. If you are not the intended recipient or have received this message in error, please notify me immediately and delete this message from your computer system. Any unauthorized use or distribution is prohibited. Please consider the environment before printing this email.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: test_mpi.f90
Type: application/octet-stream
Size: 447 bytes
Desc: not available
URL: <http://lists.geodynamics.org/pipermail/cig-seismo/attachments/20150804/bcab167a/attachment.obj>
-------------- next part --------------

> On Aug 3, 2015, at 5:30 AM, 胡喆骞 <huzheqian at gmail.com> wrote:
> 
> 
> Dear experts:
> 
> 
> 
> I used Specfem3D 2.0 before. 
> 
> 
> 
> There are some issues when I use specfem Cartesian version 2.1. When I run example/homogeneous_halfspace following READEM like that below:
> 
> [root at CentOS homogeneous_halfspace]# which mpirun
> 
> /opt/openmpi/bin/mpirun
> 
> [root at CentOS homogeneous_halfspace]# which gout
> 
> /app/20140625/specfem3d/src/decompose_mesh/scotch_5.1.12b/bin/gout
> 
> [root at CentOS homogeneous_halfspace]# ./run_this_example.sh ./run_this_example.sh >& run_this_example.sh.log
> 
> Then ,the log show :
> 
> running example: Sat Aug  1 17:41:10 CST 2015
> 
>  
> (will take about 15 minutes)
> 
>  
>  
>    setting up example...
> 
>  
> ## ---------------------------- ##
> 
> ## setting up compilation flags ##
> 
> ## ---------------------------- ##
> 
> checking build system type... x86_64-unknown-linux-gnu
> 
> checking host system type... x86_64-unknown-linux-gnu
> 
> checking for gfortran... gfortran
> 
> checking whether the Fortran compiler works... yes
> 
> checking for Fortran compiler default output file name... a.out
> 
> checking for suffix of executables...
> 
> checking whether we are cross compiling... no
> 
> checking for suffix of object files... o
> 
> checking whether we are using the GNU Fortran compiler... yes
> 
> checking whether gfortran accepts -g... yes
> 
> configure: running /bin/sh ./flags.guess
> 
> checking how to get verbose linking output from gfortran... -v
> 
> checking for Fortran libraries of gfortran...  -L/usr/lib/gcc/x86_64-redhat-linux/4.4.7 -L/usr/lib/gcc/x86_64-redhat-linux/4.4.7/../../../../lib64 -L/lib/../lib64 -L/usr/lib/../lib64 -L/usr/lib/gcc/x86_64-redhat-linux/4.4.7/../../.. -lgfortranbegin -lgfortran -lm
> 
> checking for gcc... gcc
> 
> checking whether we are using the GNU C compiler... yes
> 
> checking whether gcc accepts -g... yes
> 
> checking for gcc option to accept ISO C89... none needed
> 
> checking for dummy main to link with Fortran libraries... none
> 
> checking for Fortran name-mangling scheme... lower case, underscore, no extra underscore
> 
> checking for Fortran flag to compile .f90 files... none
> 
> checking for Fortran flag needed to accept free-form source... none
> 
> checking for gcc... (cached) gcc
> 
> checking whether we are using the GNU C compiler... (cached) yes
> 
> checking whether gcc accepts -g... (cached) yes
> 
> checking for gcc option to accept ISO C89... (cached) none needed
> 
> checking how to run the C preprocessor... gcc -E
> 
> checking for grep that handles long lines and -e... /bin/grep
> 
> checking for egrep... /bin/grep -E
> 
> checking for ANSI C header files... yes
> 
> checking for sys/types.h... yes
> 
> checking for sys/stat.h... yes
> 
> checking for stdlib.h... yes
> 
> checking for string.h... yes
> 
> checking for memory.h... yes
> 
> checking for strings.h... yes
> 
> checking for inttypes.h... yes
> 
> checking for stdint.h... yes
> 
> checking for unistd.h... yes
> 
> checking emmintrin.h usability... yes
> 
> checking emmintrin.h presence... yes
> 
> checking for emmintrin.h... yes
> 
> checking xmmintrin.h usability... yes
> 
> checking xmmintrin.h presence... yes
> 
> checking for xmmintrin.h... yes
> 
> ## ------ ##
> 
> ## SCOTCH ##
> 
> ## ------ ##
> 
> checking for scotchfarchinit  in -lscotch... no
> 
> checking whether Scotch is usable... no, using bundled scotch instead
> 
> checking for flex... flex
> 
> checking lex output file root... lex.yy
> 
> checking lex library... none needed
> 
> checking whether yytext is a pointer... no
> 
> checking for bison... bison -y
> 
> checking for the pthreads library -lpthreads... no
> 
> checking whether pthreads work without any flags... no
> 
> checking whether pthreads work with -Kthread... no
> 
> checking whether pthreads work with -kthread... no
> 
> checking for the pthreads library -llthread... no
> 
> checking whether pthreads work with -pthread... yes
> 
> checking for joinable pthread attribute... PTHREAD_CREATE_JOINABLE
> 
> checking if more special flags are required for pthreads... no
> 
> pthread found
> 
> ## ----------------- ##
> 
> ## module extensions ##
> 
> ## ----------------- ##
> 
> checking Fortran 90 module extension... mod
> 
> checking Fortran 90 module inclusion flag... -I
> 
> checking Fortran 90 module output flag... -J
> 
> ## --- ##
> 
> ## MPI ##
> 
> ## --- ##
> 
> checking for g++... g++
> 
> checking whether we are using the GNU C++ compiler... yes
> 
> checking whether g++ accepts -g... yes
> 
> checking how to run the C++ preprocessor... g++ -E
> 
> checking mpi.h usability... yes
> 
> checking mpi.h presence... yes
> 
> checking for mpi.h... yes
> 
> ## ----------------------------------- ##
> 
> ## setting up default simulation setup ##
> 
> ## ----------------------------------- ##
> 
> configure: creating ./config.status
> 
> config.status: creating Makefile
> 
> config.status: creating setup/constants.h
> 
> config.status: creating setup/precision.h
> 
> config.status: creating ./src/decompose_mesh/scotch/src/Makefile.inc
> 
> config.status: creating setup/config.h
> 
> config.status: setup/config.h is unchanged
> 
> config.status: executing bin commands
> 
> config.status: executing obj commands
> 
> config.status: executing DATA commands
> 
> config.status: executing OUTPUT_FILES commands
> 
> config.status: executing DATABASES_MPI commands
> 
>  
> ## --------------- ##
> 
> ## Specfem3D 2.1.0 ##
> 
> ## --------------- ##
> 
>  
> ./configure has completed and set up a default configuration to build.
> 
>  
> You may wish to modify the following files before running a simulation:
> 
>   DATA/Par_file           Set parameters affecting the simulation.
> 
>   DATA/CMTSOLUTION        Set the source parameters before running the solver.
> 
>   DATA/STATIONS           Set the receiver stations before running the solver.
> 
>  
> cleaning by CLEAN not defined
> 
> rm -f  ./obj/combine_surf_data.aux.o ./obj/combine_vol_data.aux.o ./obj/convolve_source_timefunction.aux.o ./obj/create_movie_shakemap_AVS_DX_GMT.aux.o ./obj/model_update.aux.o ./obj/save_external_bin_m_up.aux.o ./obj/smooth_vol_data.aux.o ./obj/sum_kernels.aux.o  ./obj/specfem3D_par.spec.o ./obj/pml_par.spec.o ./obj/model_update.aux.o ./obj/initialize_simulation.spec.o ./obj/read_mesh_databases.spec.o ./obj/save_external_bin_m_up.aux.o  ./obj/specfem3D_gpu_cuda_method_stubs.cudacc.o ./obj/specfem3D_adios_stubs.spec_noadios.o ./obj/sum_kernels.aux.o  ./obj/smooth_vol_data.aux.o  ./obj/combine_surf_data.aux.o  ./obj/combine_vol_data.aux.o ./obj/combine_vol_data_impl.aux.o  ./obj/combine_vol_data_adios_stubs.aux_noadios.o ./obj/create_movie_shakemap_AVS_DX_GMT.aux.o  ./obj/combine_vol_data_mod.mod ./obj/combine_vtk.mod  ./obj/sum_par.mod ./obj/combine_vol_data_adios_mod.mod ./bin/xcombine_surf_data ./bin/xcombine_vol_data ./bin/xconvolve_source_timefunction ./bin/xcreate_movie_shakemap_AVS_DX_GMT ./bin/xmodel_update ./bin/xsmooth_vol_data ./bin/xsum_kernels   ./obj/check_mesh_quality_CUBIT_Abaqus.check.o ./obj/convert_skewness_to_angle.check.o ./obj/multiply_CUBIT_Abaqus_mesh_by_1000.check.o   ./bin/xcheck_mesh_quality_CUBIT_Abaqus ./bin/xconvert_skewness_to_angle ./bin/xmultiply_CUBIT_Abaqus_mesh_by_1000   ./obj/check_fields_cuda.cuda.o ./obj/compute_add_sources_acoustic_cuda.cuda.o ./obj/compute_add_sources_viscoelastic_cuda.cuda.o ./obj/compute_coupling_cuda.cuda.o ./obj/compute_forces_acoustic_cuda.cuda.o ./obj/compute_forces_viscoelastic_cuda.cuda.o ./obj/compute_kernels_cuda.cuda.o ./obj/compute_stacey_acoustic_cuda.cuda.o ./obj/compute_stacey_viscoelastic_cuda.cuda.o ./obj/initialize_cuda.cuda.o ./obj/noise_tomography_cuda.cuda.o ./obj/prepare_mesh_constants_cuda.cuda.o ./obj/save_and_compare_cpu_vs_gpu.cudacc.o ./obj/transfer_fields_cuda.cuda.o ./obj/update_displacement_cuda.cuda.o ./obj/write_seismograms_cuda.cuda.o   ./obj/check_fields_cuda.cuda.o ./obj/compute_add_sources_acoustic_cuda.cuda.o ./obj/compute_add_sources_viscoelastic_cuda.cuda.o ./obj/compute_coupling_cuda.cuda.o ./obj/compute_forces_acoustic_cuda.cuda.o ./obj/compute_forces_viscoelastic_cuda.cuda.o ./obj/compute_kernels_cuda.cuda.o ./obj/compute_stacey_acoustic_cuda.cuda.o ./obj/compute_stacey_viscoelastic_cuda.cuda.o ./obj/initialize_cuda.cuda.o ./obj/noise_tomography_cuda.cuda.o ./obj/prepare_mesh_constants_cuda.cuda.o ./obj/save_and_compare_cpu_vs_gpu.cudacc.o ./obj/transfer_fields_cuda.cuda.o ./obj/update_displacement_cuda.cuda.o ./obj/write_seismograms_cuda.cuda.o    ./obj/part_decompose_mesh.dec.o ./obj/fault_scotch.dec.o ./obj/decompose_mesh.dec.o ./obj/program_decompose_mesh.dec.o  ./obj/decompose_mesh.mod ./obj/fault_scotch.mod ./obj/part_decompose_mesh.mod  ./bin/xscotch ./bin/xdecompose_mesh   ./obj/generate_databases_par.gen.o ./obj/tomography_par.gen.o ./obj/calc_jacobian.gen.o ./obj/fault_generate_databases.gen.o ./obj/create_mass_matrices.gen.o ./obj/create_regions_mesh.gen.o ./obj/finalize_databases.gen.o ./obj/generate_databases.gen.o ./obj/get_absorbing_boundary.gen.o ./obj/get_coupling_surfaces.gen.o ./obj/get_model.gen.o ./obj/get_MPI.gen.o ./obj/get_perm_color.gen.o ./obj/model_1d_cascadia.gen.o ./obj/model_1d_prem.gen.o ./obj/model_1d_socal.gen.o ./obj/model_aniso.gen.o ./obj/model_default.gen.o ./obj/model_external_values.gen.o ./obj/model_ipati.gen.o ./obj/model_gll.gen.o ./obj/model_salton_trough.gen.o ./obj/model_tomography.gen.o ./obj/pml_set_local_dampingcoeff.gen.o ./obj/program_generate_databases.gen.o ./obj/read_partition_files.gen.o ./obj/save_arrays_solver.gen.o ./obj/setup_color_perm.gen.o ./obj/setup_mesh.gen.o ./obj/memory_eval.gen.o  ./obj/generate_databases_adios_stubs.gen_noadios.o ./obj/create_regions_mesh_ext_par.mod ./obj/external_model.mod ./obj/fault_generate_databases.mod ./obj/generate_databases_par.mod ./obj/model_ipati_adios_mod.mod ./obj/salton_trough_par.mod ./obj/tomography_par.mod  ./bin/xgenerate_databases   ./obj/check_mesh_quality.mesh.o ./obj/compute_parameters.mesh.o ./obj/create_regions_mesh.mesh.o ./obj/create_visual_files.mesh.o ./obj/define_subregions.mesh.o ./obj/define_subregions_heuristic.mesh.o ./obj/define_superbrick.mesh.o ./obj/get_flags_boundaries.mesh.o ./obj/get_MPI_cutplanes_eta.mesh.o ./obj/get_MPI_cutplanes_xi.mesh.o ./obj/meshfem3D.mesh.o ./obj/program_meshfem3D.mesh.o ./obj/read_mesh_parameter_file.mesh.o ./obj/read_value_mesh_parameters.mesh.o ./obj/save_databases.mesh.o ./obj/store_boundaries.mesh.o ./obj/store_coords.mesh.o  ./obj/meshfem3D_adios_stubs.mesh_noadios.o ./obj/createregmesh.mod ./obj/readparfile.mod  ./bin/xmeshfem3D   ./obj/assemble_MPI_scalar.shared.o ./obj/check_mesh_resolution.shared.o ./obj/constants_mod.shared_module.o ./obj/create_name_database.shared.o ./obj/create_serial_name_database.shared.o ./obj/define_derivation_matrices.shared.o ./obj/detect_surface.shared.o ./obj/exit_mpi.shared.o ./obj/force_ftz.cc.o ./obj/get_attenuation_model.shared.o ./obj/get_cmt.shared.o ./obj/get_element_face.shared.o ./obj/get_force.shared.o ./obj/get_global.shared.o ./obj/get_jacobian_boundaries.shared.o ./obj/get_shape2D.shared.o ./obj/get_shape3D.shared.o ./obj/gll_library.shared.o ./obj/hex_nodes.shared.o ./obj/lagrange_poly.shared.o ./obj/netlib_specfun_erf.shared.o ./obj/param_reader.cc.o ./obj/prepare_assemble_MPI.shared.o ./obj/read_parameter_file.shared.o ./obj/read_topo_bathy_file.shared.o ./obj/read_value_parameters.shared.o ./obj/recompute_jacobian.shared.o ./obj/safe_alloc_mod.shared.o ./obj/save_header_file.shared.o ./obj/sort_array_coordinates.shared.o ./obj/unused_mod.shared_module.o ./obj/utm_geo.shared.o ./obj/write_c_binary.cc.o ./obj/write_VTK_data.shared.o  ./obj/parallel.sharedmpi.o ./obj/adios_manager_stubs.shared_noadios.o  ./obj/constants.mod ./obj/safe_alloc_mod.mod ./obj/unused_mod.mod  ./obj/adios_manager_mod.mod  ./obj/assemble_MPI_scalar.shared.o ./obj/check_mesh_resolution.shared.o ./obj/constants_mod.shared_module.o ./obj/create_name_database.shared.o ./obj/create_serial_name_database.shared.o ./obj/define_derivation_matrices.shared.o ./obj/detect_surface.shared.o ./obj/exit_mpi.shared.o ./obj/force_ftz.cc.o ./obj/get_attenuation_model.shared.o ./obj/get_cmt.shared.o ./obj/get_element_face.shared.o ./obj/get_force.shared.o ./obj/get_global.shared.o ./obj/get_jacobian_boundaries.shared.o ./obj/get_shape2D.shared.o ./obj/get_shape3D.shared.o ./obj/gll_library.shared.o ./obj/hex_nodes.shared.o ./obj/lagrange_poly.shared.o ./obj/netlib_specfun_erf.shared.o ./obj/param_reader.cc.o ./obj/prepare_assemble_MPI.shared.o ./obj/read_parameter_file.shared.o ./obj/read_topo_bathy_file.shared.o ./obj/read_value_parameters.shared.o ./obj/recompute_jacobian.shared.o ./obj/safe_alloc_mod.shared.o ./obj/save_header_file.shared.o ./obj/sort_array_coordinates.shared.o ./obj/unused_mod.shared_module.o ./obj/utm_geo.shared.o ./obj/write_c_binary.cc.o ./obj/write_VTK_data.shared.o  ./obj/parallel.sharedmpi.o ./obj/adios_manager_stubs.shared_noadios.o    ./obj/assemble_MPI_vector.spec.o ./obj/check_stability.spec.o ./obj/fault_solver_common.spec.o ./obj/fault_solver_dynamic.spec.o ./obj/fault_solver_kinematic.spec.o ./obj/gravity_perturbation.spec.o ./obj/comp_source_time_function.spec.o ./obj/compute_add_sources_acoustic.spec.o ./obj/compute_add_sources_viscoelastic.spec.o ./obj/compute_add_sources_poroelastic.spec.o ./obj/compute_adj_source_frechet.spec.o ./obj/compute_arrays_source.spec.o ./obj/compute_boundary_kernel.spec.o ./obj/compute_coupling_acoustic_el.spec.o ./obj/compute_coupling_acoustic_po.spec.o ./obj/compute_coupling_viscoelastic_ac.spec.o ./obj/compute_coupling_viscoelastic_po.spec.o ./obj/compute_coupling_poroelastic_ac.spec.o ./obj/compute_coupling_poroelastic_el.spec.o ./obj/compute_forces_acoustic_calling_routine.spec.o ./obj/compute_forces_acoustic_noDev.spec.o ./obj/compute_forces_acoustic_Dev.spec.o ./obj/compute_forces_viscoelastic_calling_routine.spec.o ./obj/compute_forces_viscoelastic_Dev.spec.o ./obj/compute_forces_viscoelastic_noDev.spec.o ./obj/compute_forces_poro_fluid_part.spec.o ./obj/compute_forces_poroelastic_calling_routine.spec.o ./obj/compute_forces_poro_solid_part.spec.o ./obj/compute_gradient.spec.o ./obj/compute_interpolated_dva.spec.o ./obj/compute_kernels.spec.o ./obj/compute_stacey_acoustic.spec.o ./obj/compute_stacey_viscoelastic.spec.o ./obj/compute_stacey_poroelastic.spec.o ./obj/compute_total_energy.spec.o ./obj/create_color_image.spec.o ./obj/detect_mesh_surfaces.spec.o ./obj/finalize_simulation.spec.o ./obj/initialize_simulation.spec.o ./obj/iterate_time.spec.o ./obj/locate_receivers.spec.o ./obj/locate_source.spec.o ./obj/make_gravity.spec.o ./obj/multiply_arrays_source.spec.o ./obj/noise_tomography.spec.o ./obj/pml_allocate_arrays.spec.o ./obj/pml_output_VTKs.spec.o ./obj/pml_compute_accel_contribution.spec.o ./obj/pml_compute_memory_variables.spec.o ./obj/pml_par.spec.o ./obj/prepare_timerun.spec.o ./obj/program_specfem3D.spec.o ./obj/read_mesh_databases.spec.o ./obj/save_adjoint_kernels.spec.o ./obj/setup_GLL_points.spec.o ./obj/setup_movie_meshes.spec.o ./obj/setup_sources_receivers.spec.o ./obj/specfem3D.spec.o ./obj/specfem3D_par.spec.o ./obj/update_displacement_scheme.spec.o ./obj/write_movie_output.spec.o ./obj/write_output_ASCII.spec.o ./obj/write_output_SU.spec.o ./obj/write_seismograms.spec.o  ./obj/specfem3D_gpu_cuda_method_stubs.cudacc.o  ./obj/specfem3D_adios_stubs.spec_noadios.o ./obj/fault_solver_common.mod ./obj/fault_solver_dynamic.mod ./obj/fault_solver_kinematic.mod ./obj/gravity_perturbation.mod ./obj/image_pnm_par.mod ./obj/pml_par.mod ./obj/specfem_par.mod ./obj/specfem_par_acoustic.mod ./obj/specfem_par_elastic.mod ./obj/specfem_par_poroelastic.mod ./obj/specfem_par_movie.mod ./obj/user_noise_distribution.mod  ./bin/xspecfem3D 
> 
>  
>   decomposing mesh...
> 
>  
> At line 357 of file src/decompose_mesh/decompose_mesh.F90
> 
> Fortran runtime error: End of file
> 
>  total number of nodes:
> 
>    nnodes =        43470
> 
>  total number of spectral elements:
> 
>    nspec =        39600
> 
>  materials:
> 
>    num_mat =            1
> 
>    num_mat =            1
> 
>    defined =            2 undefined =            0
> 
>  
>   running database generation on  4 processors...
> 
>  
> --------------------------------------------------------------------------
> 
> mpirun has detected an attempt to run as root. This is *strongly*
> 
> discouraged as any mistake (e.g., in defining TMPDIR) or bug can
> 
> result in catastrophic damage to the OS file system, leaving
> 
> your system in an unusable state.
> 
>  
> You can override this protection by adding the --allow-run-as-root
> 
> option to your cmd line. However, we reiterate our strong advice
> 
> against doing so - please do so at your own risk.
> 
> --------------------------------------------------------------------------
> 
>  
>   running solver on  4 processors...
> 
>  
> --------------------------------------------------------------------------
> 
> mpirun has detected an attempt to run as root. This is *strongly*
> 
> discouraged as any mistake (e.g., in defining TMPDIR) or bug can
> 
> result in catastrophic damage to the OS file system, leaving
> 
> your system in an unusable state.
> 
>  
> You can override this protection by adding the --allow-run-as-root
> 
> option to your cmd line. However, we reiterate our strong advice
> 
> against doing so - please do so at your own risk.
> 
> --------------------------------------------------------------------------
> 
>  
> see results in directory: OUTPUT_FILES/
> 
>  
> done
> 
> Sat Aug 1 17:41:52 CST 2015
> 
>  
>  
> It is obviously that it does not work well , and the simulation does not get the results .
> 
> I do not get how to solve the problem ,could you give me some suggestions ?
> 
>  
> 
> I am looking forward your reply. Thank you very much!
> 
>  
> 
>  
> 
> Sincerely Steve Hu
> 
> -- 
> ---------------------------------------------
> Steven Hu
> 
> _______________________________________________
> CIG-SEISMO mailing list
> CIG-SEISMO at geodynamics.org
> http://lists.geodynamics.org/cgi-bin/mailman/listinfo/cig-seismo