[cig-commits] r17250 - seismo/3D/SPECFEM3D/trunk
dkomati1 at geodynamics.org
dkomati1 at geodynamics.org
Sun Oct 10 13:54:04 PDT 2010
Author: dkomati1
Date: 2010-10-10 13:54:04 -0700 (Sun, 10 Oct 2010)
New Revision: 17250
Modified:
seismo/3D/SPECFEM3D/trunk/todo_list_please_dont_remove.txt
Log:
updated the todo list
Modified: seismo/3D/SPECFEM3D/trunk/todo_list_please_dont_remove.txt
===================================================================
--- seismo/3D/SPECFEM3D/trunk/todo_list_please_dont_remove.txt 2010-10-10 19:56:07 UTC (rev 17249)
+++ seismo/3D/SPECFEM3D/trunk/todo_list_please_dont_remove.txt 2010-10-10 20:54:04 UTC (rev 17250)
@@ -2,81 +2,10 @@
To-do list for SPECFEM3D, by Dimitri Komatitsch
-----------------------------------------------
-NOTE (by Daniel): the 3D code here uses a potential of (density * displacement),
- identical to the 2D code (see also the note in compute_forces_acoustic.f90).
- thus, the remark below is partly obsolete, having
- acoustic-acoustic discontinuities in your 3D model is fine.
+- Partitioning should be done using a parallel partitioner such as ParMETIS and/or SCOTCH (PT-SCOTCH). Reading the (distributed or not) mesh, building the graph, distributing, and partitioning is not a problem, but redistributing the graph/mesh according to the partitioning obtained might well be. Should add support for parMETIS or PT-Scotch (i.e. decompose the mesh in parallel rather than serial; this will be required for very large meshes, too large to fit on a serial machine). Serial domain decomposition is already a limitation in the case of regional seismology, because serial meshes can be very large. One option would be to switch to ParMetis or PT-Scotch, i.e., the parallel version of these domain decomposition packages. I know that PT-Scotch works great and is pretty fast. Another option is ZOLTAN from Sandia National Labs, but I do not know if it is serial or parallel. I think adding support for PT-Scotch should not be too difficult if we give it a very simple initial "analytical" partition, e.g. if we have NSPEC spectral elements and NPROC domains, put the first 1..NPROC elements in domain 0, then NPROC+1...2*NPROC in domain 1 etc. That's your initial domain decomposition; then give that to ParMETIS or PT-Scotch and ask it to optimize by moving elements between partitions.
+- For binary files (for instance Carl Tape's m16 model) we should use NETCDF to store them and read them back; that is the only clean way of ensuring portability between different systems; and NETCDF is free and open source therefore there is no reason not to do it.
-Date: Fri, 23 Jul 2010 01:30:16 +0200
-From: Dimitri Komatitsch <dimitri.komatitsch at univ-pau.fr>
-To: Anne Sieminski <anne.sieminski at obs.ujf-grenoble.fr>
-CC: Jeroen Tromp <jtromp at princeton.edu>, Min Chen <mchen at gps.caltech.edu>, Vala Hjorleifsdottir <vala at ldeo.columbia.edu>, Brian Savage <savage at uri.edu>, Shiann-Jong Lee <sjlee at earth.sinica.edu.tw>, Dimitri Komatitsch <dimitri.komatitsch at univ-pau.fr>, Roland Martin <roland.martin at univ-pau.fr>, Bernhard Schuberth <mail at bernhard-schuberth.de>, Carl Tape <carltape at fas.harvard.edu>, Anne Sieminski <anne.sieminski at obs.ujf-grenoble.fr>, Paul Friberg <p.friberg at isti.com>, Kasper van Wijk <kasper at cgiss.boisestate.edu>, Dylan Mikesell <dmikesell at cgiss.boisestate.edu>, Federica Magnoni <federica.magnoni at ingv.it>, Ebru Bozdag <bozdag at princeton.edu>, Hejun Zhu <hejunzhu at princeton.edu>, Pieyre Le Loher <pieyre.le_loher at inria.fr>, Christina Morency <cmorency at Princeton.EDU>, Emanuele Casarotti <emanuele.casarotti at gmail.com>, Piero Basini <basini at bo.ingv.it>, Emmanuel Chaljub <Emmanuel.Chaljub at obs.ujf-grenoble.fr>, Tarje Nissen-Meyer <tarje at princeton.edu>, Qinya Liu <liuqy at physics.utoronto.ca>, Yang Luo <yangl at Princeton.EDU>, Daniel Peter <dpeter at Princeton.EDU>, Ying Zhou <yingz at vt.edu>
-Subject: Re: milieux acoustiques
-
-Dear Anne,
-
-I answer in English and cc all the other developers because other people
-may be interested. Yes, SPECFEM3D version 2 (which is a beta
-version right now) can handle acoustic media in addition to
-(visco)elastic layers. Daniel Peter has added support for that
-(based in part on our implementation in the 2D version SPECFEM2D-6.0.0
-I guess, or on what was already implemented in SPECFEM3D_GLOBE, i.e.,
-the fluid outer core of the Earth).
-
-However, there is a hidden problem if you want to use acoustic layers
-for oil industry simulations: our acoustic formulation in the 3D code
-(at least in SPECFEM3D_GLOBE for sure) can only handle one acoustic
-layer with a smooth velocity gradient (which is the case in the outer
-core) but not a first-order acoustic-acoustic discontinuity, which from
-a physical point of view does not make much sense. *BUT* this implies
-that if you take an elastic oil industry model and make it acoustic
-(with layers) by only considering Vp, as people often do in oil
-companies, the code will fail (more precisely, and unfortunately, it
-will not crash but will produce seismograms that will not be correct).
-
-This is because for acoustic-acoustic interfaces one should use a
-potential of (density * displacement) rather than a potential of
-displacement only in order to enforce continuity of pressure
-automatically in the weak formulation of the problem.
-
-In the 2D code that's what I do (which means you can safely use
-the 2D code for acoustic oil industry models). But I think that
-this is not done in the 3D code because I assume Daniel took the
-acoustic code from SPECFEM3D_GLOBE, in which case a potential
-of displacement is used and acoustic-acoustic interfaces are not handled
-properly.
-
-I guess one day we should switch to a potential of (density *
-displacement) in the 3D code, as I did in the 2D code two years ago.
-Otherwise I think one day some users will use it for oil industry models
-and will get erroneous seismograms. That's (very) easy to do but
-unfortunately I do not have time for now...
-
-Hope this helps,
-Cheers,
-
-Dimitri.
-
-
-On 06/23/2010 05:42 PM, Anne Sieminski wrote:
-> Bonjour Dimitri,
->
-> Je vois que SPECFEM2D-6.0.0 (disponible sur le site de CIG) permet de
-> faire des calculs dans des modËles acoustiques. La mÍme option est-elle
-> en projet pour SPECFEM3D (ancien SPECFEM3D_BASIN) ? J'ai un peu perdu le
-> fil de toutes les nouveautÈs... DÈsolÈe !
->
-> Merci,
-> Anne
-
-- serial domain decomposition is already a limitation in the case of regional seismology, because serial meshes can be very large. One option would be to switch to ParMetis or PT-Scotch, i.e., the parallel version of these domain decomposition packages. I know that PT-Scotch works great and is pretty fast. Another option is ZOLTAN from Sandia National Labs, but I do not know if it is serial or parallel.
-
-- update the manual before the first official release of version 2.0
-
-- For binary files (for instance Carl Tape's m16 model) we should use NETCDF to store them and read them back;
-that is the only clean way of ensuring portability between different systems; and NETCDF is free and open source therefore there is no reason not to do it.
-
- there is something in SPECFEM3D that we noticed a few years ago
regarding attenuation but never fixed: on page 813 of our 1999 paper
I used a trick suggested by Robertsson et al. (1994)
@@ -99,7 +28,7 @@
- we could add support for the TetGen mesh creation package, and cut each
tetrahedron into four hexahedra using the barycenter. This could help design
-meshes for sedimentary basins, in particular near basin edges
+meshes for sedimentary basins, in particular near basin edges. Celine Blitz is probably going to work on that in Oct, Nov and Dec 2010.
- we could add support for different polynomial degrees in different elements (p-adaptivity)
@@ -118,13 +47,11 @@
- automatic detection (coloring / flags) of the free surface and of the PML absorbing elements in CUBIT; read this from input files in the solver
-- Pieyre should add C-PML following Roland's 2D implementation
+- Mila or Pieyre should add C-PML following Roland's 2D implementation
-- should add support for parMETIS or PT-Scotch (i.e. decompose the mesh in parallel rather than serial; this will be required for very large meshes, too large to fit on a serial machine)
+- etudier comment lire un maillage CUBIT stocke au format natif de CUBIT (Exodus, base sur netCDF) depuis SPECFEM3D. On pourrait utiliser la commande "ncdump" dans CUBIT si necessaire d'apres Emanuele Casarotti. Une autre option serait d'utiliser le format de stockage ABAQUS dans CUBIT.
-- etudier comment lire un maillage CUBIT stocke au format natif de CUBIT (Exodus, base sur netCDF) depuis SPECFEM3D. On pourrait utiliser la commande "ncdump" dans CUBIT si necessaire d'aprËs Emanuele Casarotti. Une autre option serait d'utiliser le format de stockage ABAQUS dans CUBIT.
-
by Nicolas Le Goff :
--------------------
@@ -136,7 +63,6 @@
- Materials should be associated to elements before pre_meshfem, but it is not always possible (that was the case for Celine Blitz's asteroid mesh with regolith because of limitations in the CUBIT software with huge meshes). An example of how we can isolate elements in contact with the envelope and associate them with another material is commented in specfem3D.f90 (look for string 'NL NL REGOLITH') tough it should in fact be inside meshfem3D, not specfem3D. Identifying the envelope is based on the NGLL points, so there is no problem with tranfering it to meshfem3D.
- - Calculations on elements in specfem3D for regular meshes should be merged inside compute_forces (the basic calculation is okay, but not attenuation, absorbing boundaries, ...) as there is no reason to differenciate those.
MISC :
@@ -148,12 +74,8 @@
- Receivers distance to source along the surface computed for an asteroid can be done by computing the shortest path in a graph. No problem in serial, doing so in parallel might be tricky.
- - Partitioning should be done in meshfem3D, using a parallel partitioner such as ParMETIS and/or SCOTCH (PT-SCOTCH). Reading the (distributed or not) mesh, building the graph, distributing, and partitioning is not a problem, but redistributing the graph/mesh according to the partitioning obtained might well be.
+ - Distinguish between elements in contact with the envelope, and those that are in contact with a fracture : source, receivers, ... anything based on the envelope detection will have to take into account those fractures. Really tricky without prior information on the mesh.
- - Adding absorbing boundaries, attenuation, PML, moment sources, ... for external meshes.
-
- - Differentiate between elements in contact with the envelope, and those that are in contact with a fracture : source, receivers, ... anything based on the envelope detection will have to take into account those fractures. Really tricky without prior information on the mesh.
-
- Settle the issue of normal recording for the receivers. The normal is unique, but the plane can be described using two vectors at random (while conserving an orthonormal reference), so we have no reason to choose a pair of vectors compared to the others.i
@@ -171,8 +93,7 @@
1/ create a CUBIT mesh
Possibly decimate it with "decimate_mesh"
-2/ check his quality with Dimitri's Fortran code
-"check_mesh_quality_CUBIT_Abaqus"
+2/ check its quality with Dimitri's Fortran code "check_mesh_quality_CUBIT_Abaqus"
3/ export it with Emanuele's scripts, in decompose_mesh_SCOTCH/OUTPUT_FILES
@@ -211,3 +132,69 @@
in SCOTCH to call METIS instead). But this is a bad idea because
the partition obtained is worse.
+- NOTE (by Daniel Peter): the 3D code here uses a potential of (density * displacement),
+ identical to the 2D code (see also the note in compute_forces_acoustic.f90).
+ thus, the remark below is obsolete, having
+ acoustic-acoustic discontinuities in your 3D model is fine.
+
+Date: Fri, 23 Jul 2010 01:30:16 +0200
+From: Dimitri Komatitsch <dimitri.komatitsch at univ-pau.fr>
+To: Anne Sieminski <anne.sieminski at obs.ujf-grenoble.fr>
+CC: Jeroen Tromp <jtromp at princeton.edu>, Min Chen <mchen at gps.caltech.edu>, Vala Hjorleifsdottir <vala at ldeo.columbia.edu>, Brian Savage <savage at uri.edu>, Shiann-Jong Lee <sjlee at earth.sinica.edu.tw>, Dimitri Komatitsch <dimitri.komatitsch at univ-pau.fr>, Roland Martin <roland.martin at univ-pau.fr>, Bernhard Schuberth <mail at bernhard-schuberth.de>, Carl Tape <carltape at fas.harvard.edu>, Anne Sieminski <anne.sieminski at obs.ujf-grenoble.fr>, Paul Friberg <p.friberg at isti.com>, Kasper van Wijk <kasper at cgiss.boisestate.edu>, Dylan Mikesell <dmikesell at cgiss.boisestate.edu>, Federica Magnoni <federica.magnoni at ingv.it>, Ebru Bozdag <bozdag at princeton.edu>, Hejun Zhu <hejunzhu at princeton.edu>, Pieyre Le Loher <pieyre.le_loher at inria.fr>, Christina Morency <cmorency at Princeton.EDU>, Emanuele Casarotti <emanuele.casarotti at gmail.com>, Piero Basini <basini at bo.ingv.it>, Emmanuel Chaljub <Emmanuel.Chaljub at obs.ujf-grenoble.fr>, Tarje Nissen-Meyer <tarje at princeton.edu>, Qinya Liu <liuqy at physics.utoronto.ca>, Yang Luo <yangl at Princeton.EDU>, Daniel Peter <dpeter at Princeton.EDU>, Ying Zhou <yingz at vt.edu>
+Subject: Re: milieux acoustiques
+
+Dear Anne,
+
+I answer in English and cc all the other developers because other people
+may be interested. Yes, SPECFEM3D version 2 (which is a beta
+version right now) can handle acoustic media in addition to
+(visco)elastic layers. Daniel Peter has added support for that
+(based in part on our implementation in the 2D version SPECFEM2D-6.0.0
+I guess, or on what was already implemented in SPECFEM3D_GLOBE, i.e.,
+the fluid outer core of the Earth).
+
+However, there is a hidden problem if you want to use acoustic layers
+for oil industry simulations: our acoustic formulation in the 3D code
+(at least in SPECFEM3D_GLOBE for sure) can only handle one acoustic
+layer with a smooth velocity gradient (which is the case in the outer
+core) but not a first-order acoustic-acoustic discontinuity, which from
+a physical point of view does not make much sense. *BUT* this implies
+that if you take an elastic oil industry model and make it acoustic
+(with layers) by only considering Vp, as people often do in oil
+companies, the code will fail (more precisely, and unfortunately, it
+will not crash but will produce seismograms that will not be correct).
+
+This is because for acoustic-acoustic interfaces one should use a
+potential of (density * displacement) rather than a potential of
+displacement only in order to enforce continuity of pressure
+automatically in the weak formulation of the problem.
+
+In the 2D code that's what I do (which means you can safely use
+the 2D code for acoustic oil industry models). But I think that
+this is not done in the 3D code because I assume Daniel took the
+acoustic code from SPECFEM3D_GLOBE, in which case a potential
+of displacement is used and acoustic-acoustic interfaces are not handled
+properly.
+
+I guess one day we should switch to a potential of (density *
+displacement) in the 3D code, as I did in the 2D code two years ago.
+Otherwise I think one day some users will use it for oil industry models
+and will get erroneous seismograms. That's (very) easy to do but
+unfortunately I do not have time for now...
+
+Hope this helps,
+Cheers,
+
+Dimitri.
+
+On 06/23/2010 05:42 PM, Anne Sieminski wrote:
+> Bonjour Dimitri,
+>
+> Je vois que SPECFEM2D-6.0.0 (disponible sur le site de CIG) permet de
+> faire des calculs dans des modeles acoustiques. La meme option est-elle
+> en projet pour SPECFEM3D (ancien SPECFEM3D_BASIN) ? J'ai un peu perdu le
+> fil de toutes les nouveautes... Desolee !
+>
+> Merci,
+> Anne
+
More information about the CIG-COMMITS
mailing list