From ljhwang at ucdavis.edu Sun Mar 5 10:41:35 2017 From: ljhwang at ucdavis.edu (Lorraine Hwang) Date: Sun, 5 Mar 2017 10:41:35 -0800 Subject: [CIG-SEISMO] This Week's Webinar and Summer Workshops Message-ID: <18507CF1-C820-4296-8F6B-80D38A4A0CD8@ucdavis.edu> Don’t miss the following CIG events. Registration deadlines for summer workshops are fast approaching! ——————————————————————————————————————————————————— Thursday, March 9th, 2017 2:00 PM PT Adobe Connect: http://uc-d.adobeconnect.com/r28i3av93ti/ "Introduction to the spectral-infinite-element method" ~ by Hom Nath Gharti and Jeroen Tromp, Princeton University The governing equations for the elastic-gravitational deformation of an Earth model involve a perturbed gravitational potential. The gravitational potential is governed by Poisson’s equation inside the Earth and by Laplace’s equation in the rest of space. The infinite domain and large-scale nature of the problem poses difficult challenges for numerical simulations in 3D Earth models. In order to tackle these challenges, we introduce a spectral-infinite-element method by combining the spectral-element method with the mapped infinite-element method. Spectral elements are used to represent internal fields, and infinite elements represent the external gravitational field. Infinite elements naturally couple with spectral elements, thereby avoiding an iterative procedure which is necessary if the Poisson/Laplace equation has to be solved independently. Potential applications of new method include long-period seismic wave propagation, as well as quasistatic problems, such as post-earthquake relaxation and glacial isostatic adjustment. ——————————————————————————————————————————————————— REGISTRATION DEADLINES 2017 ASPECT Hackathon May 6-17 Blue Ridge, Georgia https://geodynamics.org/cig/events/calendar/2017-aspect-hack/?eID=1300 Registration Closes: March 31, 2017 Notification of Acceptance Begins: April 1, 2017 2017 Crustal Deformation Modeling Workshop June 26-30 Golden, Colorado https://geodynamics.org/cig/events/calendar/2017-cdm-workshop/?eID=1292 Registration Closes: May 15, 2017 Notification of travel support begins: April 1, 2017 *** Registration is first come first serve and limited to 60 participants -------------- next part -------------- An HTML attachment was scrubbed... URL: From xpdong at math.tsinghua.edu.cn Wed Mar 8 05:17:02 2017 From: xpdong at math.tsinghua.edu.cn (=?utf-8?Q?=E8=91=A3=E5=85=B4=E6=9C=8B?=) Date: Wed, 08 Mar 2017 13:17:02 +0000 Subject: [CIG-SEISMO] a question about mineos Message-ID: <20170308131702.62c663f8@moon.math.tsinghua.edu.cn> Dear All, I am using the software mineos , and the result from it is different from specfem method. It is puzzling that the result do not contain P-wave component(result in the mail attachments). I have been stuck with this problem for a few days. I will appreciate kind response on how to solve this problem. Best regards, Xingpeng Dong Department of Mathematical Scciences, Tsinghua Uinversity -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: result.zip Type: application/x-zip-compressed Size: 6256728 bytes Desc: not available URL: From komatitsch at lma.cnrs-mrs.fr Thu Mar 9 08:28:47 2017 From: komatitsch at lma.cnrs-mrs.fr (Dimitri Komatitsch) Date: Thu, 9 Mar 2017 17:28:47 +0100 Subject: [CIG-SEISMO] need help with define_external_model.f90 in specfem2D In-Reply-To: <420405147.3776314.1489076498783@mail.yahoo.com> References: <420405147.3776314.1489076498783.ref@mail.yahoo.com> <420405147.3776314.1489076498783@mail.yahoo.com> Message-ID: <527720d0-153e-da39-7526-642fb3b88d4d@lma.cnrs-mrs.fr> Hi Ignazio, Hi all, I think Alexis will know the answer because he actively uses SPECFEM2D, including for external models :-) Best wishes, Dimitri. On 03/09/2017 05:21 PM, ignazio damiani wrote: > Dear all, > I'm using specfem2D and I'm trying to assign an external velocity model. The aim is to create a model with lateral variation of seismic wave velocities and density. > I'm trying to use define_external_model.f90 file but even though I suppress the routine for the AK135F, the function doesn't work. > I always get the same error: > >> External model: ak135-f >> STOP wrong flag number > > This is what i'm doing: > firstly, i set in Par_file MODEL=external. > Secondly, in read_external_model.f90 i set: case ('external') call define_external_model_dummy(). >> And finally in define_external_model.f90 i leave the subroutine as define_external_model_dummy(), i suppress the AK135F routine and i modify the number of material_element(ispec) based on my coordinates and region. >> What's wrong? because it remains the error. > > Thanks for your help. > Thanks Dimitri for the help and for giving me your email. > > Best regards > > Ignazio Damiani > -- > Msc in Exploration and Applied Geophysics. Pisa. Italy > -- Dimitri Komatitsch, CNRS Research Director (DR CNRS) Laboratory of Mechanics and Acoustics, Marseille, France http://komatitsch.free.fr From ignaziodamiani at yahoo.it Thu Mar 9 08:21:38 2017 From: ignaziodamiani at yahoo.it (ignazio damiani) Date: Thu, 9 Mar 2017 16:21:38 +0000 (UTC) Subject: [CIG-SEISMO] need help with define_external_model.f90 in specfem2D References: <420405147.3776314.1489076498783.ref@mail.yahoo.com> Message-ID: <420405147.3776314.1489076498783@mail.yahoo.com> Dear all, I'm using specfem2D and I'm trying to assign an external velocity model. The aim is to create a model with lateral variation of seismic wave velocities and density. I'm trying to use define_external_model.f90 file but even though I suppress the routine for the AK135F, the function doesn't work. I always get the same error: > External model: ak135-f > STOP wrong flag number This is what i'm doing: firstly, i set in Par_file MODEL=external. Secondly, in read_external_model.f90 i set: case ('external') call define_external_model_dummy(). > And finally in define_external_model.f90 i leave the subroutine as define_external_model_dummy(), i suppress the AK135F routine and i modify the number of material_element(ispec) based on my coordinates and region. > What's wrong? because it remains the error. Thanks for your help. Thanks Dimitri for the help and for giving me your email. Best regards Ignazio Damiani -- Msc in Exploration and Applied Geophysics. Pisa. Italy From l.cebamanos at epcc.ed.ac.uk Fri Mar 17 11:47:52 2017 From: l.cebamanos at epcc.ed.ac.uk (Luis Cebamanos) Date: Fri, 17 Mar 2017 18:47:52 +0000 Subject: [CIG-SEISMO] Cray XC3- installation Message-ID: <96b53397-c82d-1337-0e47-e9714fe9cab5@epcc.ed.ac.uk> Hi, We are trying to build specfem2d on a Cray XC30, but we have failed so far. Is there instructions to build it on Crays? We have tried with GNU, Intel and Cray compilers: ./configure FC=ftn CC=cc MPIFC=ftn --with-mpi Thank you very much for your help. Luis Cebamanos -- The University of Edinburgh is a charitable body, registered in Scotland, with registration number SC005336. From komatitsch at lma.cnrs-mrs.fr Fri Mar 17 12:30:07 2017 From: komatitsch at lma.cnrs-mrs.fr (Dimitri Komatitsch) Date: Fri, 17 Mar 2017 20:30:07 +0100 Subject: [CIG-SEISMO] Cray XC3- installation In-Reply-To: <96b53397-c82d-1337-0e47-e9714fe9cab5@epcc.ed.ac.uk> References: <96b53397-c82d-1337-0e47-e9714fe9cab5@epcc.ed.ac.uk> Message-ID: <7c087b05-1e09-8543-fca0-53f3a7c347de@lma.cnrs-mrs.fr> Hi, Works fine here. Could you email us the error messages? Thanks, Dimitri. On 03/17/2017 07:47 PM, Luis Cebamanos wrote: > Hi, > > We are trying to build specfem2d on a Cray XC30, but we have failed so > far. Is there instructions to build it on Crays? We have tried with GNU, > Intel and Cray compilers: > > ./configure FC=ftn CC=cc MPIFC=ftn --with-mpi > > Thank you very much for your help. > > Luis Cebamanos > > > -- Dimitri Komatitsch, CNRS Research Director (DR CNRS) Laboratory of Mechanics and Acoustics, Marseille, France http://komatitsch.free.fr From l.cebamanos at epcc.ed.ac.uk Sat Mar 18 03:11:57 2017 From: l.cebamanos at epcc.ed.ac.uk (Luis Cebamanos) Date: Sat, 18 Mar 2017 10:11:57 +0000 Subject: [CIG-SEISMO] Cray XC30 installation Message-ID: <3e960898-e871-6ee5-cf56-9686c650b570@epcc.ed.ac.uk> Hi Dimitri, We are getting the following errors: checking whether we are using the GNU C compiler... yes checking whether cc accepts -g... yes checking for cc option to accept ISO C89... none needed checking for dummy main to link with Fortran libraries..... unknown configure: error: in `/home/z01/z01/lcebaman/specfem2d': configure: error: linking to Fortran libraries from C fails See `config.log' for more details having a look to config.log, I can see the error when checking Fortran libraries: configure:4111: checking for dummy main to link with Fortran libraries configure:4145: cc -o conftest -g -O2 conftest.c -L/opt/cray/cce/8.4.1/CC/x8 6-64/lib/x86-64 -L/opt/gcc/4.8.1/snos/lib64 /opt/cray/cce/8.4.1/craylibs/x86-64/ libmodules.a /opt/cray/cce/8.4.1/craylibs/x86-64/libomp.a /opt/cray/cce/8.4.1/cr aylibs/x86-64/libopenacc.a -L/opt/cray/dmapp/default/lib64 -L/opt/cray/mpt/7.2.6 /gni/mpich-cray/8.3/lib -L/opt/cray/libsci/13.2.0/CRAY/8.3/x86_64/lib -L/opt/cra y/rca/1.0.0-2.0502.57212.2.56.ari/lib64 -L/opt/cray/alps/5.2.3-2.0502.9295.14.14 .ari/lib64 -L/opt/cray/xpmem/0.1-2.0502.57015.1.15.ari/lib64 -L/opt/cray/dmapp/7 .0.1-1.0502.10246.8.47.ari/lib64 -L/opt/cray/pmi/5.0.7-1.0000.10678.155.25.ari/l ib64 -L/opt/cray/ugni/6.0-1.0502.10245.9.9.ari/lib64 -L/opt/cray/udreg/2.3.2-1.0 502.9889.2.20.ari/lib64 -L/opt/cray/atp/1.8.3/libApp -L/opt/cray/cce/8.4.1/crayl ibs/x86-64 -L/opt/cray/wlm_detect/1.0-1.0502.57063.1.1.ari/lib64 -lAtpSigHandler -lAtpSigHCommData -lpthread -lmpichf90_cray -lrt -lugni -lpmi -lsci_cray_mpi_mp -lm -lf -lsci_cray_mp -lmpich_cray -lcraymp -lpgas-dmapp -lfi -lu -ldmapp -ludr eg -lcray-c++-rts -lcraystdc++ -lxpmem -lalpslli -lwlm_detect -lalpsutil -lrca - lquadmath -lomp -lmodules -lcraymath -lgfortran -lcsup -latomic -ltcmalloc_minim al -lstdc++ -L/opt/gcc/4.8.1/snos/lib/gcc/x86_64-suse-linux/4.8.1 -L/opt/cray/cc e/8.4.1/cray-binutils/x86_64-unknown-linux-gnu/lib -L//usr/lib64 >&5 /opt/cray/cce/8.4.1/craylibs/x86-64/libtcmalloc_minimal.a(tcmalloc.o):(.bss+0x13 ): multiple definition of `FLAG__namespace_do_not_use_directly_use_DECLARE_bool_ instead::FLAGS_tcmalloc_abort_on_large_alloc' /opt/cray/cce/8.4.1/craylibs/x86-64/libtcmalloc_minimal.a(tcmalloc.o):(.bss+0x13 ): first defined here /opt/cray/cce/8.4.1/craylibs/x86-64/libtcmalloc_minimal.a(tcmalloc.o): In functi on `TCMallocImplementation::GetAllocatedSize(void*)': tcmalloc.cc:(.text+0x1f0): multiple definition of `TCMallocImplementation::GetAl locatedSize(void*)' /opt/cray/cce/8.4.1/craylibs/x86-64/libtcmalloc_minimal.a(tcmalloc.o):tcmalloc.c c:(.text+0x1f0): first defined here /opt/cray/cce/8.4.1/craylibs/x86-64/libtcmalloc_minimal.a(tcmalloc.o): In functi on `TCMallocImplementation::MarkThreadBusy()': Any idea on how to overcome this error? I am surprise nobody has seen it before on a Cray... Cheers, Luis >Hi, > >Works fine here. Could you email us the error messages? > >Thanks, >Dimitri. On 03/17/2017 07:47 PM, Luis Cebamanos wrote: >/Hi, />//>/We are trying to build specfem2d on a Cray XC30, but we have failed so />/far. Is there instructions to build it on Crays? We have tried with GNU, />/Intel and Cray compilers: />//>/./configure FC=ftn CC=cc MPIFC=ftn --with-mpi />//>/Thank you very much for your help. />//>/Luis Cebamanos />//>//>// -- Dimitri Komatitsch, CNRS Research Director (DR CNRS) Laboratory of Mechanics and Acoustics, Marseille, France http://komatitsch.free.fr -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- An embedded and charset-unspecified text was scrubbed... Name: not available URL: From komatitsch at lma.cnrs-mrs.fr Sat Mar 18 04:10:58 2017 From: komatitsch at lma.cnrs-mrs.fr (Dimitri Komatitsch) Date: Sat, 18 Mar 2017 12:10:58 +0100 Subject: [CIG-SEISMO] Cray XC30 installation In-Reply-To: <3e960898-e871-6ee5-cf56-9686c650b570@epcc.ed.ac.uk> References: <3e960898-e871-6ee5-cf56-9686c650b570@epcc.ed.ac.uk> Message-ID: <074affe5-6b24-3502-24d6-33357436b2dd@lma.cnrs-mrs.fr> Hi Luis, Thanks! It is apparently a classical problem when using the Cray Fortran compiler, not specific to SPECFEM. I found this page https://github.com/mpip/pfft/issues/11 in which people developing a different package (PFFT) seem to be having the same problem. On Piz Daint in Switzerland, which is also a Cray, we fixed that by using the script below, it should work in your case as well. But on another Cray (Titan at Oak Ridge) we do not need that for some reason. Best regards, Dimitri. ================================ the following modules are loaded in my case: Currently Loaded Modulefiles: 1) modules/3.2.10.5 10) dmapp/7.1.0-16.18 19) cray-mpich/7.5.0 2) eswrap/2.0.11-2.2 11) gni-headers/5.0.7-4.11 20) slurm/16.05.8-1 3) cce/8.5.5 12) xpmem/2.0.3_geb8008a-2.11 21) ddt/7.0 4) craype-network-aries 13) job/2.0.2_g98a4850-2.43 22) craype-haswell 5) craype/2.5.8 14) dvs/2.5_2.0.70_g1ddb68c-2.144 23) cudatoolkit/8.0.54_2.2.8_ga620558-2.1 6) cray-libsci/16.11.1 15) alps/6.2.5-20.1 24) xalt/daint-2016.11 7) udreg/2.3.2-4.14 16) rca/2.0.10_g66b76b7-2.51 25) daint-gpu 8) ugni/6.0.13-2.8 17) atp/2.0.4 9) pmi/5.0.10-1.0000.11050.0.0.ari 18) PrgEnv-cray/6.0.3 that is in .bashrc file, i have: module load PrgEnv-cray module load cudatoolkit module load daint-gpu the configuration uses: .. mpif90=ftn mpicc=cc f90=ftn cc=cc # run flags #flags="-eF -em -rm" #cflags="-h list=m” ## memory > 2GB #flags="-eF -em -rm -O3,fp3 -hpic -dynamic" #cflags="-h list=m -hpic -dynamic" # debug flags flags="-g -Rb -eF -rm -eC -eD" # -hfp2" cflags="-g -h list=m” ### ### CUDA ### CUDA_INC="${CRAY_CUDATOOLKIT_DIR}/include" CUDA_LIB="${CRAY_CUDATOOLKIT_DIR}/lib64” ### ### mpi.h / mpif.h ### MPI_INC="${CRAY_MPICH2_DIR}/include” ./configure \ --with-cuda=cuda5 CUDA_INC="$CUDA_INC" CUDA_LIB="$CUDA_LIB" MPI_INC="$MPI_INC” \ MPIFC=$mpif90 MPICC=$mpicc FC=$f90 CC=$cc CXX=$cc FLAGS_CHECK="$flags" CFLAGS="$cflags" works, but you probably want to use then cuda8 with the right GPU gencode format (sm_60,compute_60). best wishes, daniel > > On Jan 31, 2017, at 6:50 PM, Dimitri Komatitsch wrote: > > > > > > Hi Daniel, > > > > on Piz Daint the current "configure" script does not work, we type: > > > > module load daint-gpu > > > > and then: > > > > ./configure FC=ftn MPIFC=ftn CC=cc CXX=CC > > > > as indicated at http://user.cscs.ch/compiling_and_optimizing/compiling_your_code/index.html > > > > but the script does not work, we get > > > > configure: error: linking to Fortran libraries from C fails > > See `config.log' for more details > > > > and the error is apparently coming from 32-bit compilation or linking, since in `config.log' it complains about the size of a memory segment: > > > > > > configure:4308: checking for dummy main to link with Fortran libraries > > configure:4342: cc -o conftest -g -O2 conftest.c -L/opt/cray/pe/cce/8.5.5/CC/x86-64/lib/x86-64 -L/opt/gcc/4.8.1/snos/lib64 -L/usr/lib64 -L/lib64 /opt/cray/pe/cce/8.5.5/craylibs/x86-64/libmodules.a /opt/cray/pe/cce/8.5.5/craylibs/x86-64/libomp.a /opt/cray/pe/cce/8.5.5/craylibs/x86-64/libopenacc.a -L/opt/cray/dmapp/default/lib64 -L/opt/cray/pe/mpt/7.5.0/gni/mpich-cray/8.4/lib -L/opt/cray/pe/libsci/16.11.1/CRAY/8.3/x86_64/lib -L/opt/cray/rca/2.0.10_g66b76b7-2.51/lib64 -L/opt/cray/alps/6.2.5-20.1/lib64 -L/opt/cray/xpmem/2.0.3_geb8008a-2.11/lib64 -L/opt/cray/dmapp/7.1.0-16.18/lib64 -L/opt/cray/pe/pmi/5.0.10-1.0000.11050.0.0.ari/lib64 -L/opt/cray/ugni/6.0.13-2.8/lib64 -L/opt/cray/udreg/2.3.2-4.14/lib64 -L/opt/cray/pe/atp/2.0.4/libApp -L/opt/cray/pe/cce/8.5.5/craylibs/x86-64/pkgconfig/../ -L/opt/cray/wlm_detect/1.2.1-3.1/lib64 -lAtpSigHandler -lAtpSigHCommData -lpthread -lmpichf90_cray -lrt -lugni -lpmi -lsci_cray_mpi_mp -lm -lf -lsci_cray_mp -lmpich_cray -lcraymp -lpgas-dmapp -lfi -lu -ldmapp -ludreg -lcray-c++-rts -lcraystdc++ -lxpmem -lalpslli -lwlm_detect -lalpsutil -lrca -lquadmath -lomp -ldl -lmodules -lcraymath -lgfortran -lcsup -latomic -ltcmalloc_minimal -lstdc++ -L/opt/gcc/4.8.1/snos/lib/gcc/x86_64-suse-linux/4.8.1 -L/opt/cray/pe/cce/8.5.5/cray-binutils/x86_64-unknown-linux-gnu/lib >&5 > > /opt/cray/pe/cce/8.5.5/craylibs/x86-64/pkgconfig/..//libtcmalloc_minimal.a(tcmalloc.o):(.bss+0x13): multiple definition of `FLAG__namespace_do_not_use_directly_use_DECLARE_bool_instead::FLAGS_tcmalloc_abort_on_large_alloc' > > /opt/cray/pe/cce/8.5.5/craylibs/x86-64/pkgconfig/..//libtcmalloc_minimal.a(tcmalloc.o):(.bss+0x13): first defined here > > /opt/cray/pe/cce/8.5.5/craylibs/x86-64/pkgconfig/..//libtcmalloc_minimal.a(tcmalloc.o): In function `TCMallocImplementation::GetAllocatedSize(void*)': > > tcmalloc.cc:(.text+0x1f0): multiple definition of `TCMallocImplementation::GetAllocatedSize(void*)' > > /opt/cray/pe/cce/8.5.5/craylibs/x86-64/pkgconfig/..//libtcmalloc_minimal.a(tcmalloc.o):tcmalloc.cc:(.text+0x1f0): first defined here > > /opt/cray/pe/cce/8.5.5/craylibs/x86-64/pkgconfig/..//libtcmalloc_minimal.a(tcmalloc.o): In function `TCMallocImplementation::MarkThreadBusy()': > > tcmalloc.cc:(.text+0xb00): multiple definition of `TCMallocImplementation::MarkThreadBusy()' > > /opt/cray/pe/cce/8.5.5/craylibs/x86-64/pkgconfig/..//libtcmalloc_minimal.a(tcmalloc.o):tcmalloc.cc:(.text+0xb00): first defined here > > ================================ #!/bin/bash mpif90=ftn mpicc=cc f90=ftn cc=cc # run flags #flags="-eF -em -rm" #cflags="-h list=m" ## memory > 2GB #flags="-eF -em -rm -O3,fp3 -hpic -dynamic" #cflags="-h list=m -hpic -dynamic" # debug flags flags="-g -Rb -eF -rm -eC -eD" # -hfp2" cflags="-g -h list=m" ### ### CUDA ### CUDA_INC="${CRAY_CUDATOOLKIT_DIR}/include" CUDA_LIB="${CRAY_CUDATOOLKIT_DIR}/lib64" ### ### mpi.h / mpif.h ### MPI_INC="${CRAY_MPICH2_DIR}/include" ./configure \ --with-cuda=cuda5 CUDA_INC="$CUDA_INC" CUDA_LIB="$CUDA_LIB" MPI_INC="$MPI_INC" \ MPIFC=$mpif90 MPICC=$mpicc FC=$f90 CC=$cc CXX=$cc FLAGS_CHECK="$flags" CFLAGS="$cflags" =================================== On 03/18/2017 11:11 AM, Luis Cebamanos wrote: > Hi Dimitri, > > We are getting the following errors: > > checking whether we are using the GNU C compiler... yes > checking whether cc accepts -g... yes > checking for cc option to accept ISO C89... none needed > checking for dummy main to link with Fortran libraries..... unknown > configure: error: in `/home/z01/z01/lcebaman/specfem2d': > configure: error: linking to Fortran libraries from C fails > See `config.log' for more details > > having a look to config.log, I can see the error when checking > Fortran libraries: > > configure:4111: checking for dummy main to link with Fortran libraries > configure:4145: cc -o conftest -g -O2 conftest.c -L/opt/cray/cce/8.4.1/CC/x8 > 6-64/lib/x86-64 -L/opt/gcc/4.8.1/snos/lib64 /opt/cray/cce/8.4.1/craylibs/x86-64/ > libmodules.a /opt/cray/cce/8.4.1/craylibs/x86-64/libomp.a /opt/cray/cce/8.4.1/cr > aylibs/x86-64/libopenacc.a -L/opt/cray/dmapp/default/lib64 -L/opt/cray/mpt/7.2.6 > /gni/mpich-cray/8.3/lib -L/opt/cray/libsci/13.2.0/CRAY/8.3/x86_64/lib -L/opt/cra > y/rca/1.0.0-2.0502.57212.2.56.ari/lib64 -L/opt/cray/alps/5.2.3-2.0502.9295.14.14 > .ari/lib64 -L/opt/cray/xpmem/0.1-2.0502.57015.1.15.ari/lib64 -L/opt/cray/dmapp/7 > .0.1-1.0502.10246.8.47.ari/lib64 -L/opt/cray/pmi/5.0.7-1.0000.10678.155.25.ari/l > ib64 -L/opt/cray/ugni/6.0-1.0502.10245.9.9.ari/lib64 -L/opt/cray/udreg/2.3.2-1.0 > 502.9889.2.20.ari/lib64 -L/opt/cray/atp/1.8.3/libApp -L/opt/cray/cce/8.4.1/crayl > ibs/x86-64 -L/opt/cray/wlm_detect/1.0-1.0502.57063.1.1.ari/lib64 -lAtpSigHandler > -lAtpSigHCommData -lpthread -lmpichf90_cray -lrt -lugni -lpmi -lsci_cray_mpi_mp > -lm -lf -lsci_cray_mp -lmpich_cray -lcraymp -lpgas-dmapp -lfi -lu -ldmapp -ludr > eg -lcray-c++-rts -lcraystdc++ -lxpmem -lalpslli -lwlm_detect -lalpsutil -lrca - > lquadmath -lomp -lmodules -lcraymath -lgfortran -lcsup -latomic -ltcmalloc_minim > al -lstdc++ -L/opt/gcc/4.8.1/snos/lib/gcc/x86_64-suse-linux/4.8.1 -L/opt/cray/cc > e/8.4.1/cray-binutils/x86_64-unknown-linux-gnu/lib -L//usr/lib64 >&5 > /opt/cray/cce/8.4.1/craylibs/x86-64/libtcmalloc_minimal.a(tcmalloc.o):(.bss+0x13 > ): multiple definition of `FLAG__namespace_do_not_use_directly_use_DECLARE_bool_ > instead::FLAGS_tcmalloc_abort_on_large_alloc' > /opt/cray/cce/8.4.1/craylibs/x86-64/libtcmalloc_minimal.a(tcmalloc.o):(.bss+0x13 > ): first defined here > /opt/cray/cce/8.4.1/craylibs/x86-64/libtcmalloc_minimal.a(tcmalloc.o): In functi > on `TCMallocImplementation::GetAllocatedSize(void*)': > tcmalloc.cc:(.text+0x1f0): multiple definition of `TCMallocImplementation::GetAl > locatedSize(void*)' > /opt/cray/cce/8.4.1/craylibs/x86-64/libtcmalloc_minimal.a(tcmalloc.o):tcmalloc.c > c:(.text+0x1f0): first defined here > /opt/cray/cce/8.4.1/craylibs/x86-64/libtcmalloc_minimal.a(tcmalloc.o): In functi > on `TCMallocImplementation::MarkThreadBusy()': > > > Any idea on how to overcome this error? I am surprise nobody has seen it before on a Cray... > > Cheers, > Luis >>Hi, >> >>Works fine here. Could you email us the error messages? >> >>Thanks, >>Dimitri. > > On 03/17/2017 07:47 PM, Luis Cebamanos wrote: >>/Hi, />//>/We are trying to build specfem2d on a Cray XC30, but we have failed so />/far. Is there instructions to build it on Crays? We have tried with GNU, />/Intel and Cray compilers: />//>/./configure FC=ftn CC=cc MPIFC=ftn --with-mpi />//>/Thank you very much for your help. />//>/Luis Cebamanos />//>//>// > -- > Dimitri Komatitsch, CNRS Research Director (DR CNRS) > Laboratory of Mechanics and Acoustics, Marseille, France > http://komatitsch.free.fr > > > > > The University of Edinburgh is a charitable body, registered in > Scotland, with registration number SC005336. > -- Dimitri Komatitsch, CNRS Research Director (DR CNRS) Laboratory of Mechanics and Acoustics, Marseille, France http://komatitsch.free.fr From l.cebamanos at epcc.ed.ac.uk Sat Mar 18 05:21:26 2017 From: l.cebamanos at epcc.ed.ac.uk (Luis Cebamanos) Date: Sat, 18 Mar 2017 12:21:26 +0000 Subject: [CIG-SEISMO] Cray XC30 installation In-Reply-To: <074affe5-6b24-3502-24d6-33357436b2dd@lma.cnrs-mrs.fr> References: <3e960898-e871-6ee5-cf56-9686c650b570@epcc.ed.ac.uk> <074affe5-6b24-3502-24d6-33357436b2dd@lma.cnrs-mrs.fr> Message-ID: <5524e598-a076-dc52-b3ec-b051cb127bd3@epcc.ed.ac.uk> Hi Dimitri, Daniel, Thank you very much for your help. Exporting MPI_INC=${CRAY_MPICH2_DIR}/include and FCLIBS=" " solves the configuration problem. I haven't used CUDA since we don't have GPUs here but I will be working towards a KNL solution. I was wondering if there is already some work done on KNL. Cheers, Luis On 18/03/2017 11:10, Dimitri Komatitsch wrote: > > Hi Luis, > > Thanks! It is apparently a classical problem when using the Cray > Fortran compiler, not specific to SPECFEM. I found this page > https://github.com/mpip/pfft/issues/11 in which people developing a > different package (PFFT) seem to be having the same problem. > > On Piz Daint in Switzerland, which is also a Cray, we fixed that by > using the script below, it should work in your case as well. But on > another Cray (Titan at Oak Ridge) we do not need that for some reason. > > Best regards, > Dimitri. > > ================================ > > the following modules are loaded in my case: > > Currently Loaded Modulefiles: > 1) modules/3.2.10.5 10) dmapp/7.1.0-16.18 > 19) cray-mpich/7.5.0 > 2) eswrap/2.0.11-2.2 11) gni-headers/5.0.7-4.11 > 20) slurm/16.05.8-1 > 3) cce/8.5.5 12) > xpmem/2.0.3_geb8008a-2.11 21) ddt/7.0 > 4) craype-network-aries 13) > job/2.0.2_g98a4850-2.43 22) craype-haswell > 5) craype/2.5.8 14) > dvs/2.5_2.0.70_g1ddb68c-2.144 23) > cudatoolkit/8.0.54_2.2.8_ga620558-2.1 > 6) cray-libsci/16.11.1 15) alps/6.2.5-20.1 > 24) xalt/daint-2016.11 > 7) udreg/2.3.2-4.14 16) > rca/2.0.10_g66b76b7-2.51 25) daint-gpu > 8) ugni/6.0.13-2.8 17) atp/2.0.4 > 9) pmi/5.0.10-1.0000.11050.0.0.ari 18) PrgEnv-cray/6.0.3 > > > that is in .bashrc file, i have: > > module load PrgEnv-cray > module load cudatoolkit > module load daint-gpu > > > the configuration uses: > .. > > mpif90=ftn > mpicc=cc > f90=ftn > cc=cc > > # run flags > #flags="-eF -em -rm" > #cflags="-h list=m” > > ## memory > 2GB > #flags="-eF -em -rm -O3,fp3 -hpic -dynamic" > #cflags="-h list=m -hpic -dynamic" > > # debug flags > flags="-g -Rb -eF -rm -eC -eD" # -hfp2" > cflags="-g -h list=m” > > ### > ### CUDA > ### > CUDA_INC="${CRAY_CUDATOOLKIT_DIR}/include" > CUDA_LIB="${CRAY_CUDATOOLKIT_DIR}/lib64” > > ### > ### mpi.h / mpif.h > ### > MPI_INC="${CRAY_MPICH2_DIR}/include” > > ./configure \ > --with-cuda=cuda5 CUDA_INC="$CUDA_INC" CUDA_LIB="$CUDA_LIB" > MPI_INC="$MPI_INC” \ > MPIFC=$mpif90 MPICC=$mpicc FC=$f90 CC=$cc CXX=$cc FLAGS_CHECK="$flags" > CFLAGS="$cflags" > > works, but you probably want to use then cuda8 with the right GPU > gencode format (sm_60,compute_60). > > > best wishes, > daniel > > > > > On Jan 31, 2017, at 6:50 PM, Dimitri Komatitsch wrote: > > > > > > > > > Hi Daniel, > > > > > > on Piz Daint the current "configure" script does not work, we type: > > > > > > module load daint-gpu > > > > > > and then: > > > > > > ./configure FC=ftn MPIFC=ftn CC=cc CXX=CC > > > > > > as indicated at > http://user.cscs.ch/compiling_and_optimizing/compiling_your_code/index.html > > > > > > but the script does not work, we get > > > > > > configure: error: linking to Fortran libraries from C fails > > > See `config.log' for more details > > > > > > and the error is apparently coming from 32-bit compilation or > linking, since in `config.log' it complains about the size of a memory > segment: > > > > > > > > > configure:4308: checking for dummy main to link with Fortran > libraries > > > configure:4342: cc -o conftest -g -O2 conftest.c > -L/opt/cray/pe/cce/8.5.5/CC/x86-64/lib/x86-64 > -L/opt/gcc/4.8.1/snos/lib64 -L/usr/lib64 -L/lib64 > /opt/cray/pe/cce/8.5.5/craylibs/x86-64/libmodules.a > /opt/cray/pe/cce/8.5.5/craylibs/x86-64/libomp.a > /opt/cray/pe/cce/8.5.5/craylibs/x86-64/libopenacc.a > -L/opt/cray/dmapp/default/lib64 > -L/opt/cray/pe/mpt/7.5.0/gni/mpich-cray/8.4/lib > -L/opt/cray/pe/libsci/16.11.1/CRAY/8.3/x86_64/lib > -L/opt/cray/rca/2.0.10_g66b76b7-2.51/lib64 > -L/opt/cray/alps/6.2.5-20.1/lib64 > -L/opt/cray/xpmem/2.0.3_geb8008a-2.11/lib64 > -L/opt/cray/dmapp/7.1.0-16.18/lib64 > -L/opt/cray/pe/pmi/5.0.10-1.0000.11050.0.0.ari/lib64 > -L/opt/cray/ugni/6.0.13-2.8/lib64 -L/opt/cray/udreg/2.3.2-4.14/lib64 > -L/opt/cray/pe/atp/2.0.4/libApp > -L/opt/cray/pe/cce/8.5.5/craylibs/x86-64/pkgconfig/../ > -L/opt/cray/wlm_detect/1.2.1-3.1/lib64 -lAtpSigHandler > -lAtpSigHCommData -lpthread -lmpichf90_cray -lrt -lugni -lpmi > -lsci_cray_mpi_mp -lm -lf -lsci_cray_mp -lmpich_cray -lcraymp > -lpgas-dmapp -lfi -lu -ldmapp -ludreg -lcray-c++-rts -lcraystdc++ > -lxpmem -lalpslli -lwlm_detect -lalpsutil -lrca -lquadmath -lomp -ldl > -lmodules -lcraymath -lgfortran -lcsup -latomic -ltcmalloc_minimal > -lstdc++ -L/opt/gcc/4.8.1/snos/lib/gcc/x86_64-suse-linux/4.8.1 > -L/opt/cray/pe/cce/8.5.5/cray-binutils/x86_64-unknown-linux-gnu/lib >&5 > > > > /opt/cray/pe/cce/8.5.5/craylibs/x86-64/pkgconfig/..//libtcmalloc_minimal.a(tcmalloc.o):(.bss+0x13): > multiple definition of > `FLAG__namespace_do_not_use_directly_use_DECLARE_bool_instead::FLAGS_tcmalloc_abort_on_large_alloc' > > > > /opt/cray/pe/cce/8.5.5/craylibs/x86-64/pkgconfig/..//libtcmalloc_minimal.a(tcmalloc.o):(.bss+0x13): > first defined here > > > > /opt/cray/pe/cce/8.5.5/craylibs/x86-64/pkgconfig/..//libtcmalloc_minimal.a(tcmalloc.o): > In function `TCMallocImplementation::GetAllocatedSize(void*)': > > > tcmalloc.cc:(.text+0x1f0): multiple definition of > `TCMallocImplementation::GetAllocatedSize(void*)' > > > > /opt/cray/pe/cce/8.5.5/craylibs/x86-64/pkgconfig/..//libtcmalloc_minimal.a(tcmalloc.o):tcmalloc.cc:(.text+0x1f0): > first defined here > > > > /opt/cray/pe/cce/8.5.5/craylibs/x86-64/pkgconfig/..//libtcmalloc_minimal.a(tcmalloc.o): > In function `TCMallocImplementation::MarkThreadBusy()': > > > tcmalloc.cc:(.text+0xb00): multiple definition of > `TCMallocImplementation::MarkThreadBusy()' > > > > /opt/cray/pe/cce/8.5.5/craylibs/x86-64/pkgconfig/..//libtcmalloc_minimal.a(tcmalloc.o):tcmalloc.cc:(.text+0xb00): > first defined here > > > > > ================================ > > #!/bin/bash > > mpif90=ftn > mpicc=cc > f90=ftn > cc=cc > > # run flags > #flags="-eF -em -rm" > #cflags="-h list=m" > > ## memory > 2GB > #flags="-eF -em -rm -O3,fp3 -hpic -dynamic" > #cflags="-h list=m -hpic -dynamic" > > # debug flags > flags="-g -Rb -eF -rm -eC -eD" # -hfp2" > cflags="-g -h list=m" > > ### > ### CUDA > ### > CUDA_INC="${CRAY_CUDATOOLKIT_DIR}/include" > CUDA_LIB="${CRAY_CUDATOOLKIT_DIR}/lib64" > > ### > ### mpi.h / mpif.h > ### > MPI_INC="${CRAY_MPICH2_DIR}/include" > > ./configure \ > --with-cuda=cuda5 CUDA_INC="$CUDA_INC" CUDA_LIB="$CUDA_LIB" > MPI_INC="$MPI_INC" \ > MPIFC=$mpif90 MPICC=$mpicc FC=$f90 CC=$cc CXX=$cc FLAGS_CHECK="$flags" > CFLAGS="$cflags" > > =================================== > > > On 03/18/2017 11:11 AM, Luis Cebamanos wrote: >> Hi Dimitri, >> >> We are getting the following errors: >> >> checking whether we are using the GNU C compiler... yes >> checking whether cc accepts -g... yes >> checking for cc option to accept ISO C89... none needed >> checking for dummy main to link with Fortran libraries..... unknown >> configure: error: in `/home/z01/z01/lcebaman/specfem2d': >> configure: error: linking to Fortran libraries from C fails >> See `config.log' for more details >> >> having a look to config.log, I can see the error when checking >> Fortran libraries: >> >> configure:4111: checking for dummy main to link with Fortran libraries >> configure:4145: cc -o conftest -g -O2 conftest.c >> -L/opt/cray/cce/8.4.1/CC/x8 >> 6-64/lib/x86-64 -L/opt/gcc/4.8.1/snos/lib64 >> /opt/cray/cce/8.4.1/craylibs/x86-64/ >> libmodules.a /opt/cray/cce/8.4.1/craylibs/x86-64/libomp.a >> /opt/cray/cce/8.4.1/cr >> aylibs/x86-64/libopenacc.a -L/opt/cray/dmapp/default/lib64 >> -L/opt/cray/mpt/7.2.6 >> /gni/mpich-cray/8.3/lib -L/opt/cray/libsci/13.2.0/CRAY/8.3/x86_64/lib >> -L/opt/cra >> y/rca/1.0.0-2.0502.57212.2.56.ari/lib64 >> -L/opt/cray/alps/5.2.3-2.0502.9295.14.14 >> .ari/lib64 -L/opt/cray/xpmem/0.1-2.0502.57015.1.15.ari/lib64 >> -L/opt/cray/dmapp/7 >> .0.1-1.0502.10246.8.47.ari/lib64 >> -L/opt/cray/pmi/5.0.7-1.0000.10678.155.25.ari/l >> ib64 -L/opt/cray/ugni/6.0-1.0502.10245.9.9.ari/lib64 >> -L/opt/cray/udreg/2.3.2-1.0 >> 502.9889.2.20.ari/lib64 -L/opt/cray/atp/1.8.3/libApp >> -L/opt/cray/cce/8.4.1/crayl >> ibs/x86-64 -L/opt/cray/wlm_detect/1.0-1.0502.57063.1.1.ari/lib64 >> -lAtpSigHandler >> -lAtpSigHCommData -lpthread -lmpichf90_cray -lrt -lugni -lpmi >> -lsci_cray_mpi_mp >> -lm -lf -lsci_cray_mp -lmpich_cray -lcraymp -lpgas-dmapp -lfi -lu >> -ldmapp -ludr >> eg -lcray-c++-rts -lcraystdc++ -lxpmem -lalpslli -lwlm_detect >> -lalpsutil -lrca - >> lquadmath -lomp -lmodules -lcraymath -lgfortran -lcsup -latomic >> -ltcmalloc_minim >> al -lstdc++ -L/opt/gcc/4.8.1/snos/lib/gcc/x86_64-suse-linux/4.8.1 >> -L/opt/cray/cc >> e/8.4.1/cray-binutils/x86_64-unknown-linux-gnu/lib -L//usr/lib64 >&5 >> /opt/cray/cce/8.4.1/craylibs/x86-64/libtcmalloc_minimal.a(tcmalloc.o):(.bss+0x13 >> >> ): multiple definition of >> `FLAG__namespace_do_not_use_directly_use_DECLARE_bool_ >> instead::FLAGS_tcmalloc_abort_on_large_alloc' >> /opt/cray/cce/8.4.1/craylibs/x86-64/libtcmalloc_minimal.a(tcmalloc.o):(.bss+0x13 >> >> ): first defined here >> /opt/cray/cce/8.4.1/craylibs/x86-64/libtcmalloc_minimal.a(tcmalloc.o): >> In functi >> on `TCMallocImplementation::GetAllocatedSize(void*)': >> tcmalloc.cc:(.text+0x1f0): multiple definition of >> `TCMallocImplementation::GetAl >> locatedSize(void*)' >> /opt/cray/cce/8.4.1/craylibs/x86-64/libtcmalloc_minimal.a(tcmalloc.o):tcmalloc.c >> >> c:(.text+0x1f0): first defined here >> /opt/cray/cce/8.4.1/craylibs/x86-64/libtcmalloc_minimal.a(tcmalloc.o): >> In functi >> on `TCMallocImplementation::MarkThreadBusy()': >> >> >> Any idea on how to overcome this error? I am surprise nobody has seen >> it before on a Cray... >> >> Cheers, >> Luis >>> Hi, >>> >>> Works fine here. Could you email us the error messages? >>> >>> Thanks, >>> Dimitri. >> >> On 03/17/2017 07:47 PM, Luis Cebamanos wrote: >>> /Hi, />//>/We are trying to build specfem2d on a Cray XC30, but we >>> have failed so />/far. Is there instructions to build it on Crays? >>> We have tried with GNU, />/Intel and Cray compilers: >>> />//>/./configure FC=ftn CC=cc MPIFC=ftn --with-mpi />//>/Thank you >>> very much for your help. />//>/Luis Cebamanos />//>//>// >> -- >> Dimitri Komatitsch, CNRS Research Director (DR CNRS) >> Laboratory of Mechanics and Acoustics, Marseille, France >> http://komatitsch.free.fr >> >> >> >> >> The University of Edinburgh is a charitable body, registered in >> Scotland, with registration number SC005336. >> > -- The University of Edinburgh is a charitable body, registered in Scotland, with registration number SC005336. From komatitsch at lma.cnrs-mrs.fr Sat Mar 18 09:29:23 2017 From: komatitsch at lma.cnrs-mrs.fr (Dimitri Komatitsch) Date: Sat, 18 Mar 2017 17:29:23 +0100 Subject: [CIG-SEISMO] Can SPECEM3D do gravit inversion? In-Reply-To: <201703182308309725285@cea-igp.ac.cn> References: <201703182308309725285@cea-igp.ac.cn> Message-ID: <662df4b2-b656-6d42-1ba7-81dd7ec57fd2@lma.cnrs-mrs.fr> Hi Ming, Thanks for your message! This is indeed an interesting topic. My friend Roland Martin has a nice paper that has just appeared on this: http://komatitsch.free.fr/preprints/GJI_Martin_gravimetry_2017.pdf thus let me cc Roland to see what the current version of SPECFEM3D can do (it can do 3D modeling for sure, but I think the gravity inversion part was performed outside of SPECFEM using scripts for now (?); Roland can tell us more about this, he is the expert on that). Thanks! Cheers, Dimitri. On 03/18/2017 04:08 PM, mzhao at cea-igp.ac.cn wrote: > Hi Dimitri, > > I read a very interesting paper recently which use spectral element > method to do gravity inversion,I wonder if the SPECFEM3D can do gravity > inversion in the future? We have a lot of seismic data and gravity data in > our group,and I'm using spectral element method all the time,if it can > do both adjoint inversion and gravity inversion, that would be exciting! > > Cheers > Ming Zhao > ------------------------------------------------------------------------ > mzhao at cea-igp.ac.cn -- Dimitri Komatitsch, CNRS Research Director (DR CNRS) Laboratory of Mechanics and Acoustics, Marseille, France http://komatitsch.free.fr From jtromp at princeton.edu Sat Mar 18 09:32:39 2017 From: jtromp at princeton.edu (Jeroen Tromp) Date: Sat, 18 Mar 2017 12:32:39 -0400 Subject: [CIG-SEISMO] Can SPECEM3D do gravit inversion? In-Reply-To: <662df4b2-b656-6d42-1ba7-81dd7ec57fd2@lma.cnrs-mrs.fr> References: <201703182308309725285@cea-igp.ac.cn> <662df4b2-b656-6d42-1ba7-81dd7ec57fd2@lma.cnrs-mrs.fr> Message-ID: <49007416-392a-8dc3-cee2-f541881f11ff@princeton.edu> Dear all, This is something Hom Nath Gharti has worked on as well, combining a spectral-element method with an infinite-element method to solve Laplace's equation in outer space. See his recent CIG webinar: https://www.youtube.com/watch?v=uIuvv7tAcfk Best regards, Jeroen Tromp On 3/18/17 12:29 PM, Dimitri Komatitsch wrote: > > Hi Ming, > > Thanks for your message! This is indeed an interesting topic. My > friend Roland Martin has a nice paper that has just appeared on this: > http://komatitsch.free.fr/preprints/GJI_Martin_gravimetry_2017.pdf > > thus let me cc Roland to see what the current version of SPECFEM3D can > do (it can do 3D modeling for sure, but I think the gravity inversion > part was performed outside of SPECFEM using scripts for now (?); > Roland can tell us more about this, he is the expert on that). > > Thanks! > Cheers, > Dimitri. > > On 03/18/2017 04:08 PM, mzhao at cea-igp.ac.cn wrote: >> Hi Dimitri, >> >> I read a very interesting paper recently which use spectral element >> method to do gravity inversion,I wonder if the SPECFEM3D can do gravity >> inversion in the future? We have a lot of seismic data and gravity >> data in >> our group,and I'm using spectral element method all the time,if it can >> do both adjoint inversion and gravity inversion, that would be exciting! >> >> Cheers >> Ming Zhao >> ------------------------------------------------------------------------ >> mzhao at cea-igp.ac.cn > From komatitsch at lma.cnrs-mrs.fr Sat Mar 18 09:46:18 2017 From: komatitsch at lma.cnrs-mrs.fr (Dimitri Komatitsch) Date: Sat, 18 Mar 2017 17:46:18 +0100 Subject: [CIG-SEISMO] Can SPECEM3D do gravity inversion? In-Reply-To: <49007416-392a-8dc3-cee2-f541881f11ff@princeton.edu> References: <201703182308309725285@cea-igp.ac.cn> <662df4b2-b656-6d42-1ba7-81dd7ec57fd2@lma.cnrs-mrs.fr> <49007416-392a-8dc3-cee2-f541881f11ff@princeton.edu> Message-ID: Hi all, Nice! For gravity modeling, in the current version of SPECFEM3D one needs to change the following flag to .true. in setup/constants.h.in : !!----------------------------------------------------------- !! !! gravity integral calculations !! !!----------------------------------------------------------- !! DK DK for gravity integrals logical, parameter :: GRAVITY_INTEGRALS = .false. !!! .true. Best regards, Dimitri. On 03/18/2017 05:32 PM, Jeroen Tromp wrote: > Dear all, > > This is something Hom Nath Gharti has worked on as well, combining a > spectral-element method with an infinite-element method to solve > Laplace's equation in outer space. See his recent CIG webinar: > https://www.youtube.com/watch?v=uIuvv7tAcfk > > Best regards, > > Jeroen Tromp > > > On 3/18/17 12:29 PM, Dimitri Komatitsch wrote: >> >> Hi Ming, >> >> Thanks for your message! This is indeed an interesting topic. My >> friend Roland Martin has a nice paper that has just appeared on this: >> http://komatitsch.free.fr/preprints/GJI_Martin_gravimetry_2017.pdf >> >> thus let me cc Roland to see what the current version of SPECFEM3D can >> do (it can do 3D modeling for sure, but I think the gravity inversion >> part was performed outside of SPECFEM using scripts for now (?); >> Roland can tell us more about this, he is the expert on that). >> >> Thanks! >> Cheers, >> Dimitri. >> >> On 03/18/2017 04:08 PM, mzhao at cea-igp.ac.cn wrote: >>> Hi Dimitri, >>> >>> I read a very interesting paper recently which use spectral element >>> method to do gravity inversion,I wonder if the SPECFEM3D can do gravity >>> inversion in the future? We have a lot of seismic data and gravity >>> data in >>> our group,and I'm using spectral element method all the time,if it can >>> do both adjoint inversion and gravity inversion, that would be exciting! >>> >>> Cheers >>> Ming Zhao >>> ------------------------------------------------------------------------ >>> mzhao at cea-igp.ac.cn >> > > _______________________________________________ > CIG-SEISMO mailing list > CIG-SEISMO at geodynamics.org > http://lists.geodynamics.org/cgi-bin/mailman/listinfo/cig-seismo -- Dimitri Komatitsch, CNRS Research Director (DR CNRS) Laboratory of Mechanics and Acoustics, Marseille, France http://komatitsch.free.fr From komatitsch at lma.cnrs-mrs.fr Sat Mar 18 10:12:11 2017 From: komatitsch at lma.cnrs-mrs.fr (Dimitri Komatitsch) Date: Sat, 18 Mar 2017 18:12:11 +0100 Subject: [CIG-SEISMO] Can SPECEM3D do gravity inversion? In-Reply-To: References: <201703182308309725285@cea-igp.ac.cn> <662df4b2-b656-6d42-1ba7-81dd7ec57fd2@lma.cnrs-mrs.fr> <49007416-392a-8dc3-cee2-f541881f11ff@princeton.edu> Message-ID: <31988a88-b4c7-6965-1d95-47688505563f@lma.cnrs-mrs.fr> Hi all, We should move this parameter to the Par_file at some point. I have opened https://github.com/geodynamics/specfem3d/issues/1003 and https://github.com/geodynamics/specfem3d_globe/issues/563 Best, Dimitri. On 03/18/2017 05:46 PM, Dimitri Komatitsch wrote: > > Hi all, > > Nice! > > For gravity modeling, in the current version of SPECFEM3D one needs to > change the following flag to .true. in setup/constants.h.in : > > !!----------------------------------------------------------- > !! > !! gravity integral calculations > !! > !!----------------------------------------------------------- > > !! DK DK for gravity integrals > logical, parameter :: GRAVITY_INTEGRALS = .false. !!! .true. > > > Best regards, > Dimitri. > > > On 03/18/2017 05:32 PM, Jeroen Tromp wrote: >> Dear all, >> >> This is something Hom Nath Gharti has worked on as well, combining a >> spectral-element method with an infinite-element method to solve >> Laplace's equation in outer space. See his recent CIG webinar: >> https://www.youtube.com/watch?v=uIuvv7tAcfk >> >> Best regards, >> >> Jeroen Tromp >> >> >> On 3/18/17 12:29 PM, Dimitri Komatitsch wrote: >>> >>> Hi Ming, >>> >>> Thanks for your message! This is indeed an interesting topic. My >>> friend Roland Martin has a nice paper that has just appeared on this: >>> http://komatitsch.free.fr/preprints/GJI_Martin_gravimetry_2017.pdf >>> >>> thus let me cc Roland to see what the current version of SPECFEM3D can >>> do (it can do 3D modeling for sure, but I think the gravity inversion >>> part was performed outside of SPECFEM using scripts for now (?); >>> Roland can tell us more about this, he is the expert on that). >>> >>> Thanks! >>> Cheers, >>> Dimitri. >>> >>> On 03/18/2017 04:08 PM, mzhao at cea-igp.ac.cn wrote: >>>> Hi Dimitri, >>>> >>>> I read a very interesting paper recently which use spectral element >>>> method to do gravity inversion,I wonder if the SPECFEM3D can do gravity >>>> inversion in the future? We have a lot of seismic data and gravity >>>> data in >>>> our group,and I'm using spectral element method all the time,if it can >>>> do both adjoint inversion and gravity inversion, that would be >>>> exciting! >>>> >>>> Cheers >>>> Ming Zhao >>>> ------------------------------------------------------------------------ >>>> >>>> mzhao at cea-igp.ac.cn >>> >> >> _______________________________________________ >> CIG-SEISMO mailing list >> CIG-SEISMO at geodynamics.org >> http://lists.geodynamics.org/cgi-bin/mailman/listinfo/cig-seismo > -- Dimitri Komatitsch, CNRS Research Director (DR CNRS) Laboratory of Mechanics and Acoustics, Marseille, France http://komatitsch.free.fr From komatitsch at lma.cnrs-mrs.fr Sat Mar 18 10:14:49 2017 From: komatitsch at lma.cnrs-mrs.fr (Dimitri Komatitsch) Date: Sat, 18 Mar 2017 18:14:49 +0100 Subject: [CIG-SEISMO] Cray XC30 installation In-Reply-To: <5524e598-a076-dc52-b3ec-b051cb127bd3@epcc.ed.ac.uk> References: <3e960898-e871-6ee5-cf56-9686c650b570@epcc.ed.ac.uk> <074affe5-6b24-3502-24d6-33357436b2dd@lma.cnrs-mrs.fr> <5524e598-a076-dc52-b3ec-b051cb127bd3@epcc.ed.ac.uk> Message-ID: <685f314e-4c7c-bb90-e3f3-a5aed80cc6c7@lma.cnrs-mrs.fr> Hi Luis, Thanks a lot! I have added that to the users manual (of the three versions of the code: 2D, 3D, and 3D_GLOBE). Best regards, Dimitri. On 03/18/2017 01:21 PM, Luis Cebamanos wrote: > Hi Dimitri, Daniel, > > Thank you very much for your help. Exporting > MPI_INC=${CRAY_MPICH2_DIR}/include and FCLIBS=" " solves the > configuration problem. I haven't used CUDA since we don't have GPUs here > but I will be working towards a KNL solution. I was wondering if there > is already some work done on KNL. > > Cheers, > > Luis > > On 18/03/2017 11:10, Dimitri Komatitsch wrote: >> >> Hi Luis, >> >> Thanks! It is apparently a classical problem when using the Cray >> Fortran compiler, not specific to SPECFEM. I found this page >> https://github.com/mpip/pfft/issues/11 in which people developing a >> different package (PFFT) seem to be having the same problem. >> >> On Piz Daint in Switzerland, which is also a Cray, we fixed that by >> using the script below, it should work in your case as well. But on >> another Cray (Titan at Oak Ridge) we do not need that for some reason. >> >> Best regards, >> Dimitri. >> >> ================================ >> >> the following modules are loaded in my case: >> >> Currently Loaded Modulefiles: >> 1) modules/3.2.10.5 10) dmapp/7.1.0-16.18 >> 19) cray-mpich/7.5.0 >> 2) eswrap/2.0.11-2.2 11) gni-headers/5.0.7-4.11 >> 20) slurm/16.05.8-1 >> 3) cce/8.5.5 12) >> xpmem/2.0.3_geb8008a-2.11 21) ddt/7.0 >> 4) craype-network-aries 13) >> job/2.0.2_g98a4850-2.43 22) craype-haswell >> 5) craype/2.5.8 14) >> dvs/2.5_2.0.70_g1ddb68c-2.144 23) >> cudatoolkit/8.0.54_2.2.8_ga620558-2.1 >> 6) cray-libsci/16.11.1 15) alps/6.2.5-20.1 >> 24) xalt/daint-2016.11 >> 7) udreg/2.3.2-4.14 16) >> rca/2.0.10_g66b76b7-2.51 25) daint-gpu >> 8) ugni/6.0.13-2.8 17) atp/2.0.4 >> 9) pmi/5.0.10-1.0000.11050.0.0.ari 18) PrgEnv-cray/6.0.3 >> >> >> that is in .bashrc file, i have: >> >> module load PrgEnv-cray >> module load cudatoolkit >> module load daint-gpu >> >> >> the configuration uses: >> .. >> >> mpif90=ftn >> mpicc=cc >> f90=ftn >> cc=cc >> >> # run flags >> #flags="-eF -em -rm" >> #cflags="-h list=m” >> >> ## memory > 2GB >> #flags="-eF -em -rm -O3,fp3 -hpic -dynamic" >> #cflags="-h list=m -hpic -dynamic" >> >> # debug flags >> flags="-g -Rb -eF -rm -eC -eD" # -hfp2" >> cflags="-g -h list=m” >> >> ### >> ### CUDA >> ### >> CUDA_INC="${CRAY_CUDATOOLKIT_DIR}/include" >> CUDA_LIB="${CRAY_CUDATOOLKIT_DIR}/lib64” >> >> ### >> ### mpi.h / mpif.h >> ### >> MPI_INC="${CRAY_MPICH2_DIR}/include” >> >> ./configure \ >> --with-cuda=cuda5 CUDA_INC="$CUDA_INC" CUDA_LIB="$CUDA_LIB" >> MPI_INC="$MPI_INC” \ >> MPIFC=$mpif90 MPICC=$mpicc FC=$f90 CC=$cc CXX=$cc FLAGS_CHECK="$flags" >> CFLAGS="$cflags" >> >> works, but you probably want to use then cuda8 with the right GPU >> gencode format (sm_60,compute_60). >> >> >> best wishes, >> daniel >> >> >> > > On Jan 31, 2017, at 6:50 PM, Dimitri Komatitsch wrote: >> > > >> > > >> > > Hi Daniel, >> > > >> > > on Piz Daint the current "configure" script does not work, we type: >> > > >> > > module load daint-gpu >> > > >> > > and then: >> > > >> > > ./configure FC=ftn MPIFC=ftn CC=cc CXX=CC >> > > >> > > as indicated at >> http://user.cscs.ch/compiling_and_optimizing/compiling_your_code/index.html >> >> > > >> > > but the script does not work, we get >> > > >> > > configure: error: linking to Fortran libraries from C fails >> > > See `config.log' for more details >> > > >> > > and the error is apparently coming from 32-bit compilation or >> linking, since in `config.log' it complains about the size of a memory >> segment: >> > > >> > > >> > > configure:4308: checking for dummy main to link with Fortran >> libraries >> > > configure:4342: cc -o conftest -g -O2 conftest.c >> -L/opt/cray/pe/cce/8.5.5/CC/x86-64/lib/x86-64 >> -L/opt/gcc/4.8.1/snos/lib64 -L/usr/lib64 -L/lib64 >> /opt/cray/pe/cce/8.5.5/craylibs/x86-64/libmodules.a >> /opt/cray/pe/cce/8.5.5/craylibs/x86-64/libomp.a >> /opt/cray/pe/cce/8.5.5/craylibs/x86-64/libopenacc.a >> -L/opt/cray/dmapp/default/lib64 >> -L/opt/cray/pe/mpt/7.5.0/gni/mpich-cray/8.4/lib >> -L/opt/cray/pe/libsci/16.11.1/CRAY/8.3/x86_64/lib >> -L/opt/cray/rca/2.0.10_g66b76b7-2.51/lib64 >> -L/opt/cray/alps/6.2.5-20.1/lib64 >> -L/opt/cray/xpmem/2.0.3_geb8008a-2.11/lib64 >> -L/opt/cray/dmapp/7.1.0-16.18/lib64 >> -L/opt/cray/pe/pmi/5.0.10-1.0000.11050.0.0.ari/lib64 >> -L/opt/cray/ugni/6.0.13-2.8/lib64 -L/opt/cray/udreg/2.3.2-4.14/lib64 >> -L/opt/cray/pe/atp/2.0.4/libApp >> -L/opt/cray/pe/cce/8.5.5/craylibs/x86-64/pkgconfig/../ >> -L/opt/cray/wlm_detect/1.2.1-3.1/lib64 -lAtpSigHandler >> -lAtpSigHCommData -lpthread -lmpichf90_cray -lrt -lugni -lpmi >> -lsci_cray_mpi_mp -lm -lf -lsci_cray_mp -lmpich_cray -lcraymp >> -lpgas-dmapp -lfi -lu -ldmapp -ludreg -lcray-c++-rts -lcraystdc++ >> -lxpmem -lalpslli -lwlm_detect -lalpsutil -lrca -lquadmath -lomp -ldl >> -lmodules -lcraymath -lgfortran -lcsup -latomic -ltcmalloc_minimal >> -lstdc++ -L/opt/gcc/4.8.1/snos/lib/gcc/x86_64-suse-linux/4.8.1 >> -L/opt/cray/pe/cce/8.5.5/cray-binutils/x86_64-unknown-linux-gnu/lib >&5 >> > > >> /opt/cray/pe/cce/8.5.5/craylibs/x86-64/pkgconfig/..//libtcmalloc_minimal.a(tcmalloc.o):(.bss+0x13): >> multiple definition of >> `FLAG__namespace_do_not_use_directly_use_DECLARE_bool_instead::FLAGS_tcmalloc_abort_on_large_alloc' >> >> > > >> /opt/cray/pe/cce/8.5.5/craylibs/x86-64/pkgconfig/..//libtcmalloc_minimal.a(tcmalloc.o):(.bss+0x13): >> first defined here >> > > >> /opt/cray/pe/cce/8.5.5/craylibs/x86-64/pkgconfig/..//libtcmalloc_minimal.a(tcmalloc.o): >> In function `TCMallocImplementation::GetAllocatedSize(void*)': >> > > tcmalloc.cc:(.text+0x1f0): multiple definition of >> `TCMallocImplementation::GetAllocatedSize(void*)' >> > > >> /opt/cray/pe/cce/8.5.5/craylibs/x86-64/pkgconfig/..//libtcmalloc_minimal.a(tcmalloc.o):tcmalloc.cc:(.text+0x1f0): >> first defined here >> > > >> /opt/cray/pe/cce/8.5.5/craylibs/x86-64/pkgconfig/..//libtcmalloc_minimal.a(tcmalloc.o): >> In function `TCMallocImplementation::MarkThreadBusy()': >> > > tcmalloc.cc:(.text+0xb00): multiple definition of >> `TCMallocImplementation::MarkThreadBusy()' >> > > >> /opt/cray/pe/cce/8.5.5/craylibs/x86-64/pkgconfig/..//libtcmalloc_minimal.a(tcmalloc.o):tcmalloc.cc:(.text+0xb00): >> first defined here >> > > >> >> ================================ >> >> #!/bin/bash >> >> mpif90=ftn >> mpicc=cc >> f90=ftn >> cc=cc >> >> # run flags >> #flags="-eF -em -rm" >> #cflags="-h list=m" >> >> ## memory > 2GB >> #flags="-eF -em -rm -O3,fp3 -hpic -dynamic" >> #cflags="-h list=m -hpic -dynamic" >> >> # debug flags >> flags="-g -Rb -eF -rm -eC -eD" # -hfp2" >> cflags="-g -h list=m" >> >> ### >> ### CUDA >> ### >> CUDA_INC="${CRAY_CUDATOOLKIT_DIR}/include" >> CUDA_LIB="${CRAY_CUDATOOLKIT_DIR}/lib64" >> >> ### >> ### mpi.h / mpif.h >> ### >> MPI_INC="${CRAY_MPICH2_DIR}/include" >> >> ./configure \ >> --with-cuda=cuda5 CUDA_INC="$CUDA_INC" CUDA_LIB="$CUDA_LIB" >> MPI_INC="$MPI_INC" \ >> MPIFC=$mpif90 MPICC=$mpicc FC=$f90 CC=$cc CXX=$cc FLAGS_CHECK="$flags" >> CFLAGS="$cflags" >> >> =================================== >> >> >> On 03/18/2017 11:11 AM, Luis Cebamanos wrote: >>> Hi Dimitri, >>> >>> We are getting the following errors: >>> >>> checking whether we are using the GNU C compiler... yes >>> checking whether cc accepts -g... yes >>> checking for cc option to accept ISO C89... none needed >>> checking for dummy main to link with Fortran libraries..... unknown >>> configure: error: in `/home/z01/z01/lcebaman/specfem2d': >>> configure: error: linking to Fortran libraries from C fails >>> See `config.log' for more details >>> >>> having a look to config.log, I can see the error when checking >>> Fortran libraries: >>> >>> configure:4111: checking for dummy main to link with Fortran libraries >>> configure:4145: cc -o conftest -g -O2 conftest.c >>> -L/opt/cray/cce/8.4.1/CC/x8 >>> 6-64/lib/x86-64 -L/opt/gcc/4.8.1/snos/lib64 >>> /opt/cray/cce/8.4.1/craylibs/x86-64/ >>> libmodules.a /opt/cray/cce/8.4.1/craylibs/x86-64/libomp.a >>> /opt/cray/cce/8.4.1/cr >>> aylibs/x86-64/libopenacc.a -L/opt/cray/dmapp/default/lib64 >>> -L/opt/cray/mpt/7.2.6 >>> /gni/mpich-cray/8.3/lib -L/opt/cray/libsci/13.2.0/CRAY/8.3/x86_64/lib >>> -L/opt/cra >>> y/rca/1.0.0-2.0502.57212.2.56.ari/lib64 >>> -L/opt/cray/alps/5.2.3-2.0502.9295.14.14 >>> .ari/lib64 -L/opt/cray/xpmem/0.1-2.0502.57015.1.15.ari/lib64 >>> -L/opt/cray/dmapp/7 >>> .0.1-1.0502.10246.8.47.ari/lib64 >>> -L/opt/cray/pmi/5.0.7-1.0000.10678.155.25.ari/l >>> ib64 -L/opt/cray/ugni/6.0-1.0502.10245.9.9.ari/lib64 >>> -L/opt/cray/udreg/2.3.2-1.0 >>> 502.9889.2.20.ari/lib64 -L/opt/cray/atp/1.8.3/libApp >>> -L/opt/cray/cce/8.4.1/crayl >>> ibs/x86-64 -L/opt/cray/wlm_detect/1.0-1.0502.57063.1.1.ari/lib64 >>> -lAtpSigHandler >>> -lAtpSigHCommData -lpthread -lmpichf90_cray -lrt -lugni -lpmi >>> -lsci_cray_mpi_mp >>> -lm -lf -lsci_cray_mp -lmpich_cray -lcraymp -lpgas-dmapp -lfi -lu >>> -ldmapp -ludr >>> eg -lcray-c++-rts -lcraystdc++ -lxpmem -lalpslli -lwlm_detect >>> -lalpsutil -lrca - >>> lquadmath -lomp -lmodules -lcraymath -lgfortran -lcsup -latomic >>> -ltcmalloc_minim >>> al -lstdc++ -L/opt/gcc/4.8.1/snos/lib/gcc/x86_64-suse-linux/4.8.1 >>> -L/opt/cray/cc >>> e/8.4.1/cray-binutils/x86_64-unknown-linux-gnu/lib -L//usr/lib64 >&5 >>> /opt/cray/cce/8.4.1/craylibs/x86-64/libtcmalloc_minimal.a(tcmalloc.o):(.bss+0x13 >>> >>> ): multiple definition of >>> `FLAG__namespace_do_not_use_directly_use_DECLARE_bool_ >>> instead::FLAGS_tcmalloc_abort_on_large_alloc' >>> /opt/cray/cce/8.4.1/craylibs/x86-64/libtcmalloc_minimal.a(tcmalloc.o):(.bss+0x13 >>> >>> ): first defined here >>> /opt/cray/cce/8.4.1/craylibs/x86-64/libtcmalloc_minimal.a(tcmalloc.o): In >>> functi >>> on `TCMallocImplementation::GetAllocatedSize(void*)': >>> tcmalloc.cc:(.text+0x1f0): multiple definition of >>> `TCMallocImplementation::GetAl >>> locatedSize(void*)' >>> /opt/cray/cce/8.4.1/craylibs/x86-64/libtcmalloc_minimal.a(tcmalloc.o):tcmalloc.c >>> >>> c:(.text+0x1f0): first defined here >>> /opt/cray/cce/8.4.1/craylibs/x86-64/libtcmalloc_minimal.a(tcmalloc.o): In >>> functi >>> on `TCMallocImplementation::MarkThreadBusy()': >>> >>> >>> Any idea on how to overcome this error? I am surprise nobody has seen >>> it before on a Cray... >>> >>> Cheers, >>> Luis >>>> Hi, >>>> >>>> Works fine here. Could you email us the error messages? >>>> >>>> Thanks, >>>> Dimitri. >>> >>> On 03/17/2017 07:47 PM, Luis Cebamanos wrote: >>>> /Hi, />//>/We are trying to build specfem2d on a Cray XC30, but we >>>> have failed so />/far. Is there instructions to build it on Crays? >>>> We have tried with GNU, />/Intel and Cray compilers: >>>> />//>/./configure FC=ftn CC=cc MPIFC=ftn --with-mpi />//>/Thank you >>>> very much for your help. />//>/Luis Cebamanos />//>//>// >>> -- >>> Dimitri Komatitsch, CNRS Research Director (DR CNRS) >>> Laboratory of Mechanics and Acoustics, Marseille, France >>> http://komatitsch.free.fr >>> >>> >>> >>> >>> The University of Edinburgh is a charitable body, registered in >>> Scotland, with registration number SC005336. >>> >> > > -- Dimitri Komatitsch, CNRS Research Director (DR CNRS) Laboratory of Mechanics and Acoustics, Marseille, France http://komatitsch.free.fr From komatitsch at lma.cnrs-mrs.fr Sat Mar 18 10:37:09 2017 From: komatitsch at lma.cnrs-mrs.fr (Dimitri Komatitsch) Date: Sat, 18 Mar 2017 18:37:09 +0100 Subject: [CIG-SEISMO] Can SPECEM3D do gravity inversion? In-Reply-To: <201703190100213105151@cea-igp.ac.cn> References: <201703182308309725285@cea-igp.ac.cn> <662df4b2-b656-6d42-1ba7-81dd7ec57fd2@lma.cnrs-mrs.fr> <49007416-392a-8dc3-cee2-f541881f11ff@princeton.edu> <201703190100213105151@cea-igp.ac.cn> Message-ID: Dear Ming, Perfect! Please let us know if you have any feedback (and if you improve it please commit your changes to Git or email them to me and I will add them to the official version of the code and will list your name in the list of authors of the code). Thanks, Cheers, Dimitri. On 03/18/2017 06:00 PM, mzhao at cea-igp.ac.cn wrote: > Dear all, > > Coooooool!!I can help to have a try.Many thanks! > > Cheers > Ming > ------------------------------------------------------------------------ > mzhao at cea-igp.ac.cn > > *From:* Dimitri Komatitsch > *Date:* 2017-03-19 00:46 > *To:* mzhao at cea-igp.ac.cn ; > cig-seismo at geodynamics.org ; Roland > Martin ; Sébastien Chevrot > ; Jeroen Tromp > > *Subject:* Re: [CIG-SEISMO] Can SPECEM3D do gravity inversion? > > Hi all, > > Nice! > > For gravity modeling, in the current version of SPECFEM3D one needs to > change the following flag to .true. in setup/constants.h.in : > > !!----------------------------------------------------------- > !! > !! gravity integral calculations > !! > !!----------------------------------------------------------- > > !! DK DK for gravity integrals > logical, parameter :: GRAVITY_INTEGRALS = .false. !!! .true. > > > Best regards, > Dimitri. > > > On 03/18/2017 05:32 PM, Jeroen Tromp wrote: >> Dear all, >> >> This is something Hom Nath Gharti has worked on as well, combining a >> spectral-element method with an infinite-element method to solve >> Laplace's equation in outer space. See his recent CIG webinar: >> https://www.youtube.com/watch?v=uIuvv7tAcfk >> >> Best regards, >> >> Jeroen Tromp >> >> >> On 3/18/17 12:29 PM, Dimitri Komatitsch wrote: >>> >>> Hi Ming, >>> >>> Thanks for your message! This is indeed an interesting topic. My >>> friend Roland Martin has a nice paper that has just appeared on this: >>> http://komatitsch.free.fr/preprints/GJI_Martin_gravimetry_2017.pdf >>> >>> thus let me cc Roland to see what the current version of SPECFEM3D can >>> do (it can do 3D modeling for sure, but I think the gravity inversion >>> part was performed outside of SPECFEM using scripts for now (?); >>> Roland can tell us more about this, he is the expert on that). >>> >>> Thanks! >>> Cheers, >>> Dimitri. >>> >>> On 03/18/2017 04:08 PM, mzhao at cea-igp.ac.cn wrote: >>>> Hi Dimitri, >>>> >>>> I read a very interesting paper recently which use spectral element >>>> method to do gravity inversion,I wonder if the SPECFEM3D can do gravity >>>> inversion in the future? We have a lot of seismic data and gravity >>>> data in >>>> our group,and I'm using spectral element method all the time,if it can >>>> do both adjoint inversion and gravity inversion, that would be exciting! >>>> >>>> Cheers >>>> Ming Zhao >>>> ------------------------------------------------------------------------ >>>> mzhao at cea-igp.ac.cn >>> >> >> _______________________________________________ >> CIG-SEISMO mailing list >> CIG-SEISMO at geodynamics.org >> http://lists.geodynamics.org/cgi-bin/mailman/listinfo/cig-seismo > > -- > Dimitri Komatitsch, CNRS Research Director (DR CNRS) > Laboratory of Mechanics and Acoustics, Marseille, France > http://komatitsch.free.fr > -- Dimitri Komatitsch, CNRS Research Director (DR CNRS) Laboratory of Mechanics and Acoustics, Marseille, France http://komatitsch.free.fr From alexis.bottero at gmail.com Mon Mar 20 06:01:25 2017 From: alexis.bottero at gmail.com (Alexis Bottero) Date: Mon, 20 Mar 2017 14:01:25 +0100 Subject: [CIG-SEISMO] need help with define_external_model.f90 in specfem2D In-Reply-To: <527720d0-153e-da39-7526-642fb3b88d4d@lma.cnrs-mrs.fr> References: <420405147.3776314.1489076498783.ref@mail.yahoo.com> <420405147.3776314.1489076498783@mail.yahoo.com> <527720d0-153e-da39-7526-642fb3b88d4d@lma.cnrs-mrs.fr> Message-ID: Hi Ignazio, Sorry for the delay, I was in holidays. Doing what you want to do in specfem2d is easy. To set an arbitrary velocity model you have to use the option TOMOGRAPHY_FILE that I have implemented and that you have to set like that in the Par_file when you define the velocity model: 2 -1 9999 9999 9999 9999 9999 9999 9999 0 0 0 0 0 0 # external tomography file TOMOGRAPHY_FILE = ./profile.xyz The program understand that the velocity model number 2 has to be read in the file profile.xyz as a regular grid it will then deal with the interpolation. You can set what you want in place of the 9999, it does not matter. The file profile.xyz has to be written under the following format: ! The xyz file TOMOGRAPHY_FILE that describe the tomography should be located in the TOMOGRAPHY_PATH ! directory, set in the Par_file. The format of the file, as read from define_external_model_from_xyz_file.f90 looks like : ! ! ORIGIN_X ORIGIN_Z END_X END_Z ! SPACING_X SPACING_Z ! NX NZ ! VP_MIN VP_MAX VS_MIN VS_MAX RHO_MIN RHO_MAX ! x(1) z(1) vp vs rho ! x(2) z(1) vp vs rho ! ... ! x(NX) z(1) vp vs rho ! x(1) z(2) vp vs rho ! x(2) z(2) vp vs rho ! ... ! x(NX) z(2) vp vs rho ! x(1) z(3) vp vs rho ! ... ! ... ! x(NX) z(NZ) vp vs rho ! ! Where : ! _x and z must be increasing ! _ORIGIN_X, END_X are, respectively, the coordinates of the initial and final tomographic ! grid points along the x direction (in meters) ! _ORIGIN_Z, END_Z are, respectively, the coordinates of the initial and final tomographic ! grid points along the z direction (in meters) ! _SPACING_X, SPACING_Z are the spacing between the tomographic grid points along the x ! and z directions, respectively (in meters) ! _NX, NZ are the number of grid points along the spatial directions x and z, ! respectively; NX is given by [(END_X - ORIGIN_X)/SPACING_X]+1; NZ is the same as NX, but ! for z direction. ! _VP_MIN, VP_MAX, VS_MIN, VS_MAX, RHO_MIN, RHO_MAX are the minimum and maximum values of ! the wave speed vp and vs (in m.s-1) and of the density rho (in kg.m-3); these values ! could be the actual limits of the tomographic parameters in the grid or the minimum ! and maximum values to which we force the cut of velocity and density in the model. ! _After these first four lines, in the file file_name the tomographic grid points are ! listed with the corresponding values of vp, vs and rho, scanning the grid along the x ! coordinate (from ORIGIN_X to END_X with step of SPACING_X) for each given z (from ORIGIN_Z ! to END_Z, with step of SPACING_Z). I send you a working script writing such file for a 1D profile (velocity depending just on depth). Read carefully all the comments but in particular the lines containing "!!WARNING!!" that I have added for you. run ./createTomographyFile.py and look at the file profile.xyz created. In your case vp, vs and rho will be 2D arrays, you have to modify that. For the moment just one model can be read from an external file but it is not very difficult to implement that. Likewise viscoelastic parameters can not vary with position for now. Please keep me informed! Cheers, Alexis Bottero, PhD student Tel: +0033 6 95 17 00 97 Waves and imaging Laboratoire de Mécanique et d'Acoustique de Marseille 2017-03-09 17:28 GMT+01:00 Dimitri Komatitsch : > > Hi Ignazio, Hi all, > > I think Alexis will know the answer because he actively uses SPECFEM2D, > including for external models :-) > > Best wishes, > Dimitri. > > > On 03/09/2017 05:21 PM, ignazio damiani wrote: > >> Dear all, >> I'm using specfem2D and I'm trying to assign an external velocity model. >> The aim is to create a model with lateral variation of seismic wave >> velocities and density. >> I'm trying to use define_external_model.f90 file but even though I >> suppress the routine for the AK135F, the function doesn't work. >> I always get the same error: >> >> External model: ak135-f >>> STOP wrong flag number >>> >> >> This is what i'm doing: >> firstly, i set in Par_file MODEL=external. >> Secondly, in read_external_model.f90 i set: case ('external') call >> define_external_model_dummy(). >> >>> And finally in define_external_model.f90 i leave the subroutine as >>> define_external_model_dummy(), i suppress the AK135F routine and i modify >>> the number of material_element(ispec) based on my coordinates and region. >>> What's wrong? because it remains the error. >>> >> >> Thanks for your help. >> Thanks Dimitri for the help and for giving me your email. >> >> Best regards >> >> Ignazio Damiani >> -- >> Msc in Exploration and Applied Geophysics. Pisa. Italy >> >> > -- > Dimitri Komatitsch, CNRS Research Director (DR CNRS) > Laboratory of Mechanics and Acoustics, Marseille, France > http://komatitsch.free.fr > -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: createTomographyFile.py Type: text/x-python Size: 3092 bytes Desc: not available URL: From mathias.pilch at tu-dortmund.de Wed Mar 22 07:54:47 2017 From: mathias.pilch at tu-dortmund.de (Mathias Pilch) Date: Wed, 22 Mar 2017 15:54:47 +0100 Subject: [CIG-SEISMO] Time step value and CPML, displacement blowing up Message-ID: Dear SPECFEM3D developers, my name is Mathias Pilch and I’m a physics student in Germany and working on my master's thesis. I’m using SPECFEM3D to simulate seismic signals of tracked vehicles, which is done for a verification project of the Physics and Disarmament Group of Lehrstuhl Experimentelle Physik III at the Technische Universität Dortmund. I am sending a copy of this mail to my advisor Juergen Altmann. We find SPECFEM3D an extremely useful tool and are very grateful to you for developing and maintaining it. I’m currently using the devel branch of SPECFEM3D. There seems to be a problem concerning the usage of CPML absorbing boundary conditions and using a time step value DT that is below a certain value. In my case, under many different conditions, there seems to be a threshold value of 1.75d-5 for DT when using CPML. Any value that is smaller than that leads to a crash of the output solver. The models I’m using are a variation of EXAMPLES/CPML_examples/homogeneous_halfspace_HEX8_elastic_absorbing_CPML_5sides with the difference being a smaller geometry, e. g. 120 m * 9 m * 12 m, with element size 0.2 m resulting in 600 * 45 * 60 = 1620000 elements. Basically I’m making sure that all element edges have the same length. In this example, the specfem3D solver fails with *** Minimum period resolved = 7.22559926E-04 *** Maximum suggested time step = 1.56957449E-05 *** for DT : 1.0000000000000001E-005 *** Max stability for wave velocities = 0.318557680 even though the DT chosen is below dt_suggested, and cmax = max stability for wave velocities is below 0.5. It also fails when taking a value closer to dt_suggested, e.g. 1.5d-5. See attachment "model_with_120*9*12" for output and log files. The usual error message when using a value below 1.75d-5 looks like: Program received signal SIGFPE: Floating-point exception - erroneous arithmetic operation. Backtrace for this error: Program received signal SIGFPE: Floating-point exception - erroneous arithmetic operation. Backtrace for this error: #0 0x2AF167374B97 #1 0x2AF167373D90 #2 0x3D3D83269F #0 0x2B3DD69DCB97 #1 0x2B3DD69DBD90 #2 0x3D3D83269F #3 0x404DD1 in check_stability_ at check_stability.f90:91 (discriminator 13) #3 0x404DD1 in check_stability_ at check_stability.f90:91 (discriminator 13) #4 0x478376 in iterate_time_ at iterate_time.F90:185 #4 0x478376 in iterate_time_ at iterate_time.F90:185 #5 0x4FA00D in specfem3d_ at specfem3D.f90:375 #6 0x40319F in xspecfem3d at program_specfem3D.f90:34 #7 0x3D3D81ED5C #5 0x4FA00D in specfem3d_ at specfem3D.f90:375 #6 0x40319F in xspecfem3d at program_specfem3D.f90:34 #7 0x3D3D81ED5C [lido-bl43112][[46853,1],104][btl_tcp_frag.c:237:mca_btl_tcp_frag_recv] mca_btl_tcp_frag_recv: readv failed: Connection reset by peer (104) [lido-bl43141][[46853,1],101][btl_tcp_frag.c:237:mca_btl_tcp_frag_recv] mca_btl_tcp_frag_recv: readv failed: Connection reset by peer (104) -------------------------------------------------------------------------- mpirun noticed that process rank 102 with PID 0 on node lido-bl43131.lidocluster.hp exited on signal 8 (Floating point exception). Which is I think caused by the Max norm displacement vector U in all slices (m) blowing up when running the solver. I used the .f90 programs in utils/CPML/, namely xadd_CPML_layers_to_an_existing_mesh xconvert_external_layers_of_a_given_mesh_to_CPML_layers to create the CPML layers with a thickness of three element sizes, which seems to work as expected. Surprisingly the code will not crash when using a thickness of only one element size, in any other case it crashes again. I also varied the percentage of CPML elements in the mesh, whether it’s around 50 % or 20 %, it still crashes. I don’t think there is an upper bound on CPML percentage, but I wanted to verify that. To rule out any error on my part I took the model EXAMPLES/CPML_examples/homogeneous_halfspace_HEX8_elastic_absorbing_CPML_5sides and set DT = 1.d-5 NSTEPS = 1000000. in DATA/Par_file. Lo and behold, it crashes again. Please take a look at max_norm_displacement.pdf in the attachment directory “cpml_example_edited". which shows quite a weird behaviour. The half duration (or f0) is set to 5.0, so at -10 s there should be no signal, but still, the displacement blows up even at that point. Those crashes only occur when using CPML as boundary condition and when DT is too low, however with Stacey it works fine with any DT value, although the absorption isn’t nearly as desirable. Switching to the SPECFEM3D master branch doesn’t help, the same error persists. May it be possible that this a fundamental problem of CPML, when DT values are below 1.75d-5? Could it be that another or a second stability condition applies for CPML? Or could this still be a problem on my side, and it works completely as intended on yours? Best regards and thank you for developing SPECFEM3D. Mathias -------------- next part -------------- A non-text attachment was scrubbed... Name: model_with_120*9*12.zip Type: application/zip Size: 2312343 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: cpml_example_edited.zip Type: application/zip Size: 1339463 bytes Desc: not available URL: -------------- next part -------------- From 3010160008 at cugb.edu.cn Sun Mar 26 23:12:51 2017 From: 3010160008 at cugb.edu.cn (=?UTF-8?B?5byg5piM5qaV?=) Date: Mon, 27 Mar 2017 14:12:51 +0800 (GMT+08:00) Subject: [CIG-SEISMO] Ask for Help Message-ID: <47aa6419.37ef.15b0e654bc8.Coremail.3010160008@cugb.edu.cn> Dear All: I'm a Ph.D. student of China University of Geosciences (BeiJing), majoring in geophysics. I have read a paper "Monteiller V, Chevrot S, Komatitsch D, et al. Three-dimensional full waveform inversion of short-period teleseismic wavefields based upon the SEM–DSM hybrid method[J]. Geophysical Journal International, 2015, 202(2):811-827.". The paper said "Our full waveform inversion software package SPECFEM3D,including the inversion routines and the coupling with DSM presented above, is available open source at geodynamics.org.". I have downloaded the software package SPECFEM3D from geodynamics.org. However, I can't find the DSM part which is said to be used for teleseismic wavefields modelling in the software. I write this e-mail to ask whether and where I can get the DSM part of the software package. I am looking forward to your reply! Thank you! ​ sincerely Zhang Chang Rong 2017.3.27 -------------- next part -------------- An HTML attachment was scrubbed... URL: