From komatitsch at lma.cnrs-mrs.fr Sun Apr 2 11:18:37 2017 From: komatitsch at lma.cnrs-mrs.fr (Dimitri Komatitsch) Date: Sun, 2 Apr 2017 20:18:37 +0200 Subject: [CIG-SEISMO] Ask for Help In-Reply-To: <47aa6419.37ef.15b0e654bc8.Coremail.3010160008@cugb.edu.cn> References: <47aa6419.37ef.15b0e654bc8.Coremail.3010160008@cugb.edu.cn> Message-ID: Hi Zhang, Yes, that package is not merged yet, let me cc Yi Wang, who has it as a tar file, he will send it to you. In the next few months we will merge it into the official CIG version but it first needs to be cleaned because it is based on an older version of SPECFEM3D. Thank you, Best regards, Dimitri. On 03/27/2017 08:12 AM, 张昌榕 wrote: > Dear All: > > I'm a Ph.D. student of China University of Geosciences (BeiJing), > majoring in geophysics. I have read a paper "Monteiller V, Chevrot S, > Komatitsch D, et al. Three-dimensional full waveform inversion of > short-period teleseismic wavefields based upon the SEM–DSM hybrid > method[J]. Geophysical Journal International, 2015, 202(2):811-827.". > The paper said "Our full waveform inversion software package > SPECFEM3D,including the inversion routines and the coupling with DSM > presented above, is available open source at geodynamics.org.". > I have downloaded the software package SPECFEM3D from > geodynamics.org. However, I can't find the DSM part which is said to be > used for teleseismic wavefields modelling in the software. > I write this e-mail to ask whether and where I can get the DSM > part of the software package. > I am looking forward to your reply！ Thank you! > ​ > > > sincerely > > > Zhang Chang Rong > > > 2017.3.27 > > > > _______________________________________________ > CIG-SEISMO mailing list > CIG-SEISMO at geodynamics.org > http://lists.geodynamics.org/cgi-bin/mailman/listinfo/cig-seismo > -- Dimitri Komatitsch, CNRS Research Director (DR CNRS) Laboratory of Mechanics and Acoustics, Marseille, France http://komatitsch.free.fr From melchior.schuh-senlis8 at etu.univ-lorraine.fr Wed Apr 5 00:46:46 2017 From: melchior.schuh-senlis8 at etu.univ-lorraine.fr (Melchior Schuh-Senlis) Date: Wed, 5 Apr 2017 09:46:46 +0200 (CEST) Subject: [CIG-SEISMO] SW4 on a fully anisotropic model with elasticity matrix varying in the model Message-ID: <291679357.34384.1491378406732.JavaMail.zimbra@etu.univ-lorraine.fr> Hi, I am currently working in the GeoRessources laboratory in Nancy, in the RING team departement. I came across an issue while wanting to use the SW4 software to simulate wave propagation in a test model. Here is my question : Since the anistropy is only available on sw4 with the 'ablock' command, would it be possible to implement anisotropy with a different elasticity matrix on each grid point by inputing an 'ablock' command around each point of the grid ? (I do realise this would take a long time to read the command file, but for the moment I would like to know if it is possible, and if it is, roughly how much of an impact it would have on the simulation itself in terms of computational time and cost) I hope my question was clear, and await for your answer. If you have any problem answering please feel free to ask for details. Best Regards, Melchior Schuh-Senlis Etudiant en 3ème année à l'ENSG Cursus géologie numérique -------------- next part -------------- An HTML attachment was scrubbed... URL: From petersson1 at llnl.gov Mon Apr 10 17:29:23 2017 From: petersson1 at llnl.gov (Petersson, Anders) Date: Tue, 11 Apr 2017 00:29:23 +0000 Subject: [CIG-SEISMO] Re; SW4 on a fully anisotropic model Message-ID: <8131042A-6A33-462A-95A6-1C4F2116E440@llnl.gov> You should be able to specify as many ablock commands as you please. But be aware that all MPI tasks read the SW4 input file, so you might eventually run out of memory if you have a grid. /Anders On 4/5/17, 12:00 PM, "CIG-SEISMO on behalf of cig-seismo-request at geodynamics.org" wrote: Send CIG-SEISMO mailing list submissions to cig-seismo at geodynamics.org To subscribe or unsubscribe via the World Wide Web, visit http://lists.geodynamics.org/cgi-bin/mailman/listinfo/cig-seismo or, via email, send a message with subject or body 'help' to cig-seismo-request at geodynamics.org You can reach the person managing the list at cig-seismo-owner at geodynamics.org When replying, please edit your Subject line so it is more specific than "Re: Contents of CIG-SEISMO digest..." Today's Topics: 1. SW4 on a fully anisotropic model with elasticity matrix varying in the model (Melchior Schuh-Senlis) ---------------------------------------------------------------------- Message: 1 Date: Wed, 5 Apr 2017 09:46:46 +0200 (CEST) From: Melchior Schuh-Senlis To: cig-seismo at geodynamics.org Subject: [CIG-SEISMO] SW4 on a fully anisotropic model with elasticity matrix varying in the model Message-ID: <291679357.34384.1491378406732.JavaMail.zimbra at etu.univ-lorraine.fr> Content-Type: text/plain; charset="utf-8" Hi, I am currently working in the GeoRessources laboratory in Nancy, in the RING team departement. I came across an issue while wanting to use the SW4 software to simulate wave propagation in a test model. Here is my question : Since the anistropy is only available on sw4 with the 'ablock' command, would it be possible to implement anisotropy with a different elasticity matrix on each grid point by inputing an 'ablock' command around each point of the grid ? (I do realise this would take a long time to read the command file, but for the moment I would like to know if it is possible, and if it is, roughly how much of an impact it would have on the simulation itself in terms of computational time and cost) I hope my question was clear, and await for your answer. If you have any problem answering please feel free to ask for details. Best Regards, Melchior Schuh-Senlis Etudiant en 3ème année à l'ENSG Cursus géologie numérique -------------- next part -------------- An HTML attachment was scrubbed... URL: ------------------------------ Subject: Digest Footer _______________________________________________ CIG-SEISMO mailing list CIG-SEISMO at geodynamics.org http://lists.geodynamics.org/cgi-bin/mailman/listinfo/cig-seismo ------------------------------ End of CIG-SEISMO Digest, Vol 111, Issue 2 ****************************************** From sevan.adourian at berkeley.edu Tue Apr 11 10:39:53 2017 From: sevan.adourian at berkeley.edu (Sevan Adourian) Date: Tue, 11 Apr 2017 19:39:53 +0200 Subject: [CIG-SEISMO] single force solution Message-ID: Hi cig-seismo community, I'm trying to compute Green's functions using Specfem3D-Globe, using a single force point source solution coupled with a source time function that basically represents a filtered Dirac delta. In order to test my implementation, I imposed a source at a point A and looked at the obtained Green's function at a receiver B. Then, I switched the position of the source and the receiver and compared the 2 in order to check the reciprocity of those Green's functions. It turned out that I have a perfect reciprocity (in the frequency band that correspond to my filtered Dirac delta) only on the component in which I input the force, while the other components don't quite match. I strongly suspected a reference frame rotation issue, so I tried to input the force in the global xyz coordinate system (i.e. adding the force directly in the accel_crust_mantle array in the comp_add_sources subroutine, without multiplying with the nu_source matrix) and looked at the Green's functions in this xyz coordinates system (ie, directly writing in the seismograms structure the values of ux, uy and uz computed in the compute_seismogram subroutine), but I still have this issue. Would anyone have an idea of where this probable rotation issue could come from? I'm working on specfem3D-Globe v6.0.0. Thanks in advance for your help! Sévan -- Sevan Adourian Masters Student at Ecole Normale Superieure de Paris sevan.adourian at ens.fr -------------- next part -------------- An HTML attachment was scrubbed... URL: From hgharti at princeton.edu Tue Apr 11 11:46:47 2017 From: hgharti at princeton.edu (Hom Nath Gharti) Date: Tue, 11 Apr 2017 14:46:47 -0400 Subject: [CIG-SEISMO] single force solution In-Reply-To: References: Message-ID: Hi Sévan, Do you also switch the orientations of source and receiver? Thanks, Hom Nath On Tue, Apr 11, 2017 at 1:39 PM, Sevan Adourian wrote: > Hi cig-seismo community, > > I'm trying to compute Green's functions using Specfem3D-Globe, using a > single force point source solution coupled with a source time function that > basically represents a filtered Dirac delta. In order to test my > implementation, I imposed a source at a point A and looked at the obtained > Green's function at a receiver B. Then, I switched the position of the > source and the receiver and compared the 2 in order to check the > reciprocity of those Green's functions. > > It turned out that I have a perfect reciprocity (in the frequency band > that correspond to my filtered Dirac delta) only on the component in which > I input the force, while the other components don't quite match. > > I strongly suspected a reference frame rotation issue, so I tried to input > the force in the global xyz coordinate system (i.e. adding the force > directly in the accel_crust_mantle array in the comp_add_sources > subroutine, without multiplying with the nu_source matrix) and looked at > the Green's functions in this xyz coordinates system (ie, directly writing > in the seismograms structure the values of ux, uy and uz computed in the > compute_seismogram subroutine), but I still have this issue. Would anyone > have an idea of where this probable rotation issue could come from? > > I'm working on specfem3D-Globe v6.0.0. > > Thanks in advance for your help! > Sévan > > -- > Sevan Adourian > Masters Student at Ecole Normale Superieure de Paris > sevan.adourian at ens.fr > -------------- next part -------------- An HTML attachment was scrubbed... URL: From sevan.adourian at berkeley.edu Wed Apr 12 02:24:07 2017 From: sevan.adourian at berkeley.edu (Sevan Adourian) Date: Wed, 12 Apr 2017 11:24:07 +0200 Subject: [CIG-SEISMO] single force solution In-Reply-To: References: Message-ID: Hi Hom Nath, Thanks for your answer and concern. Since you suggest something that is not directly "specfem related", can I send you directly an email so that we can discuss it? And then I can post the solution once we'll find it, in order to close the topic! Thanks again, Sévan On Tue, Apr 11, 2017 at 8:46 PM, Hom Nath Gharti wrote: > Hi Sévan, > > Do you also switch the orientations of source and receiver? > > Thanks, > Hom Nath > > On Tue, Apr 11, 2017 at 1:39 PM, Sevan Adourian < > sevan.adourian at berkeley.edu> wrote: > >> Hi cig-seismo community, >> >> I'm trying to compute Green's functions using Specfem3D-Globe, using a >> single force point source solution coupled with a source time function that >> basically represents a filtered Dirac delta. In order to test my >> implementation, I imposed a source at a point A and looked at the obtained >> Green's function at a receiver B. Then, I switched the position of the >> source and the receiver and compared the 2 in order to check the >> reciprocity of those Green's functions. >> >> It turned out that I have a perfect reciprocity (in the frequency band >> that correspond to my filtered Dirac delta) only on the component in which >> I input the force, while the other components don't quite match. >> >> I strongly suspected a reference frame rotation issue, so I tried to >> input the force in the global xyz coordinate system (i.e. adding the force >> directly in the accel_crust_mantle array in the comp_add_sources >> subroutine, without multiplying with the nu_source matrix) and looked at >> the Green's functions in this xyz coordinates system (ie, directly writing >> in the seismograms structure the values of ux, uy and uz computed in the >> compute_seismogram subroutine), but I still have this issue. Would anyone >> have an idea of where this probable rotation issue could come from? >> >> I'm working on specfem3D-Globe v6.0.0. >> >> Thanks in advance for your help! >> Sévan >> >> -- >> Sevan Adourian >> Masters Student at Ecole Normale Superieure de Paris >> sevan.adourian at ens.fr >> > > > _______________________________________________ > CIG-SEISMO mailing list > CIG-SEISMO at geodynamics.org > http://lists.geodynamics.org/cgi-bin/mailman/listinfo/cig-seismo > -- Sevan Adourian Masters Student at Ecole Normale Superieure de Paris sevan.adourian at ens.fr -------------- next part -------------- An HTML attachment was scrubbed... URL: From komatitsch at lma.cnrs-mrs.fr Wed Apr 12 03:36:38 2017 From: komatitsch at lma.cnrs-mrs.fr (Dimitri Komatitsch) Date: Wed, 12 Apr 2017 12:36:38 +0200 Subject: [CIG-SEISMO] single force solution In-Reply-To: References: Message-ID: Hi Sevan, I guess you can keep cc'ing the list because it is an issue Clément, Vadim, Paul, Sébastien and I have been facing as well in our coupling with external codes, thus we are interested in knowing more about the potential problem. Thanks, Best wishes, Dimitri. On 04/12/2017 11:24 AM, Sevan Adourian wrote: > Hi Hom Nath, > > Thanks for your answer and concern. Since you suggest something that is > not directly "specfem related", can I send you directly an email so that > we can discuss it? And then I can post the solution once we'll find it, > in order to close the topic! > > Thanks again, > Sévan > > On Tue, Apr 11, 2017 at 8:46 PM, Hom Nath Gharti > wrote: > > Hi Sévan, > > Do you also switch the orientations of source and receiver? > > Thanks, > Hom Nath > > On Tue, Apr 11, 2017 at 1:39 PM, Sevan Adourian > > > wrote: > > Hi cig-seismo community, > > I'm trying to compute Green's functions using Specfem3D-Globe, > using a single force point source solution coupled with a source > time function that basically represents a filtered Dirac delta. > In order to test my implementation, I imposed a source at a > point A and looked at the obtained Green's function at a > receiver B. Then, I switched the position of the source and the > receiver and compared the 2 in order to check the reciprocity of > those Green's functions. > > It turned out that I have a perfect reciprocity (in the > frequency band that correspond to my filtered Dirac delta) only > on the component in which I input the force, while the other > components don't quite match. > > I strongly suspected a reference frame rotation issue, so I > tried to input the force in the global xyz coordinate system > (i.e. adding the force directly in the accel_crust_mantle array > in the comp_add_sources subroutine, without multiplying with the > nu_source matrix) and looked at the Green's functions in this > xyz coordinates system (ie, directly writing in the seismograms > structure the values of ux, uy and uz computed in the > compute_seismogram subroutine), but I still have this issue. > Would anyone have an idea of where this probable rotation issue > could come from? > > I'm working on specfem3D-Globe v6.0.0. > > Thanks in advance for your help! > Sévan > > -- > Sevan Adourian > Masters Student at Ecole Normale Superieure de Paris > sevan.adourian at ens.fr > > > > _______________________________________________ > CIG-SEISMO mailing list > CIG-SEISMO at geodynamics.org > http://lists.geodynamics.org/cgi-bin/mailman/listinfo/cig-seismo > > > > > > -- > Sevan Adourian > Masters Student at Ecole Normale Superieure de Paris > sevan.adourian at ens.fr > > > _______________________________________________ > CIG-SEISMO mailing list > CIG-SEISMO at geodynamics.org > http://lists.geodynamics.org/cgi-bin/mailman/listinfo/cig-seismo > -- Dimitri Komatitsch, CNRS Research Director (DR CNRS) Laboratory of Mechanics and Acoustics, Marseille, France http://komatitsch.free.fr From hgharti at princeton.edu Wed Apr 12 04:06:15 2017 From: hgharti at princeton.edu (Hom Nath Gharti) Date: Wed, 12 Apr 2017 07:06:15 -0400 Subject: [CIG-SEISMO] single force solution In-Reply-To: <7d7c9aae9ef741c1be7a87fd1dcc1c71@CSGHUB209W.pu.win.princeton.edu> References: <7d7c9aae9ef741c1be7a87fd1dcc1c71@CSGHUB209W.pu.win.princeton.edu> Message-ID: Hi Sévan, I am about to finish new point force implementation. I will let you know once it is ready for testing. Best, Hom Nath On Wed, Apr 12, 2017 at 6:36 AM, Dimitri Komatitsch < komatitsch at lma.cnrs-mrs.fr> wrote: > > Hi Sevan, > > I guess you can keep cc'ing the list because it is an issue Clément, > Vadim, Paul, Sébastien and I have been facing as well in our coupling > with external codes, thus we are interested in knowing more about the > potential problem. > > Thanks, > Best wishes, > Dimitri. > > On 04/12/2017 11:24 AM, Sevan Adourian wrote: > > Hi Hom Nath, > > > > Thanks for your answer and concern. Since you suggest something that is > > not directly "specfem related", can I send you directly an email so that > > we can discuss it? And then I can post the solution once we'll find it, > > in order to close the topic! > > > > Thanks again, > > Sévan > > > > On Tue, Apr 11, 2017 at 8:46 PM, Hom Nath Gharti > > wrote: > > > > Hi Sévan, > > > > Do you also switch the orientations of source and receiver? > > > > Thanks, > > Hom Nath > > > > On Tue, Apr 11, 2017 at 1:39 PM, Sevan Adourian > > > > > wrote: > > > > Hi cig-seismo community, > > > > I'm trying to compute Green's functions using Specfem3D-Globe, > > using a single force point source solution coupled with a source > > time function that basically represents a filtered Dirac delta. > > In order to test my implementation, I imposed a source at a > > point A and looked at the obtained Green's function at a > > receiver B. Then, I switched the position of the source and the > > receiver and compared the 2 in order to check the reciprocity of > > those Green's functions. > > > > It turned out that I have a perfect reciprocity (in the > > frequency band that correspond to my filtered Dirac delta) only > > on the component in which I input the force, while the other > > components don't quite match. > > > > I strongly suspected a reference frame rotation issue, so I > > tried to input the force in the global xyz coordinate system > > (i.e. adding the force directly in the accel_crust_mantle array > > in the comp_add_sources subroutine, without multiplying with the > > nu_source matrix) and looked at the Green's functions in this > > xyz coordinates system (ie, directly writing in the seismograms > > structure the values of ux, uy and uz computed in the > > compute_seismogram subroutine), but I still have this issue. > > Would anyone have an idea of where this probable rotation issue > > could come from? > > > > I'm working on specfem3D-Globe v6.0.0. > > > > Thanks in advance for your help! > > Sévan > > > > -- > > Sevan Adourian > > Masters Student at Ecole Normale Superieure de Paris > > sevan.adourian at ens.fr > > > > > > > > _______________________________________________ > > CIG-SEISMO mailing list > > CIG-SEISMO at geodynamics.org > > http://lists.geodynamics.org/cgi-bin/mailman/listinfo/cig-seismo > > > > > > > > > > > > -- > > Sevan Adourian > > Masters Student at Ecole Normale Superieure de Paris > > sevan.adourian at ens.fr > > > > > > _______________________________________________ > > CIG-SEISMO mailing list > > CIG-SEISMO at geodynamics.org > > http://lists.geodynamics.org/cgi-bin/mailman/listinfo/cig-seismo > > > > -- > Dimitri Komatitsch, CNRS Research Director (DR CNRS) > Laboratory of Mechanics and Acoustics, Marseille, France > http://komatitsch.free.fr > _______________________________________________ > CIG-SEISMO mailing list > CIG-SEISMO at geodynamics.org > http://lists.geodynamics.org/cgi-bin/mailman/listinfo/cig-seismo > -------------- next part -------------- An HTML attachment was scrubbed... URL: From komatitsch at lma.cnrs-mrs.fr Wed Apr 12 04:09:20 2017 From: komatitsch at lma.cnrs-mrs.fr (Dimitri Komatitsch) Date: Wed, 12 Apr 2017 13:09:20 +0200 Subject: [CIG-SEISMO] single force solution In-Reply-To: References: <7d7c9aae9ef741c1be7a87fd1dcc1c71@CSGHUB209W.pu.win.princeton.edu> Message-ID: <05da2f2b-95cf-aab3-44f9-a82590fe1d2d@lma.cnrs-mrs.fr> Hi Hom Nath, Perfect, thanks! Then please remember to commit it to GitHub. Best regards, Dimitri. On 04/12/2017 01:06 PM, Hom Nath Gharti wrote: > Hi Sévan, > > I am about to finish new point force implementation. I will let you know > once it is ready for testing. > > Best, > Hom Nath > > On Wed, Apr 12, 2017 at 6:36 AM, Dimitri Komatitsch > > wrote: > > > Hi Sevan, > > I guess you can keep cc'ing the list because it is an issue Clément, > Vadim, Paul, Sébastien and I have been facing as well in our coupling > with external codes, thus we are interested in knowing more about the > potential problem. > > Thanks, > Best wishes, > Dimitri. > > On 04/12/2017 11:24 AM, Sevan Adourian wrote: > > Hi Hom Nath, > > > > Thanks for your answer and concern. Since you suggest something that is > > not directly "specfem related", can I send you directly an email so that > > we can discuss it? And then I can post the solution once we'll find it, > > in order to close the topic! > > > > Thanks again, > > Sévan > > > > On Tue, Apr 11, 2017 at 8:46 PM, Hom Nath Gharti > > >> wrote: > > > > Hi Sévan, > > > > Do you also switch the orientations of source and receiver? > > > > Thanks, > > Hom Nath > > > > On Tue, Apr 11, 2017 at 1:39 PM, Sevan Adourian > > > >> > > wrote: > > > > Hi cig-seismo community, > > > > I'm trying to compute Green's functions using Specfem3D-Globe, > > using a single force point source solution coupled with a > source > > time function that basically represents a filtered Dirac > delta. > > In order to test my implementation, I imposed a source at a > > point A and looked at the obtained Green's function at a > > receiver B. Then, I switched the position of the source > and the > > receiver and compared the 2 in order to check the > reciprocity of > > those Green's functions. > > > > It turned out that I have a perfect reciprocity (in the > > frequency band that correspond to my filtered Dirac delta) > only > > on the component in which I input the force, while the other > > components don't quite match. > > > > I strongly suspected a reference frame rotation issue, so I > > tried to input the force in the global xyz coordinate system > > (i.e. adding the force directly in the accel_crust_mantle > array > > in the comp_add_sources subroutine, without multiplying > with the > > nu_source matrix) and looked at the Green's functions in this > > xyz coordinates system (ie, directly writing in the > seismograms > > structure the values of ux, uy and uz computed in the > > compute_seismogram subroutine), but I still have this issue. > > Would anyone have an idea of where this probable rotation > issue > > could come from? > > > > I'm working on specfem3D-Globe v6.0.0. > > > > Thanks in advance for your help! > > Sévan > > > > -- > > Sevan Adourian > > Masters Student at Ecole Normale Superieure de Paris > > sevan.adourian at ens.fr > > > > > > > > > > _______________________________________________ > > CIG-SEISMO mailing list > > CIG-SEISMO at geodynamics.org > > > > > http://lists.geodynamics.org/cgi-bin/mailman/listinfo/cig-seismo > > > > > > > > > > > > > > > -- > > Sevan Adourian > > Masters Student at Ecole Normale Superieure de Paris > > sevan.adourian at ens.fr > > > > > > > > _______________________________________________ > > CIG-SEISMO mailing list > > CIG-SEISMO at geodynamics.org > > http://lists.geodynamics.org/cgi-bin/mailman/listinfo/cig-seismo > > > > > -- > Dimitri Komatitsch, CNRS Research Director (DR CNRS) > Laboratory of Mechanics and Acoustics, Marseille, France > http://komatitsch.free.fr > _______________________________________________ > CIG-SEISMO mailing list > CIG-SEISMO at geodynamics.org > http://lists.geodynamics.org/cgi-bin/mailman/listinfo/cig-seismo > > > > > > _______________________________________________ > CIG-SEISMO mailing list > CIG-SEISMO at geodynamics.org > http://lists.geodynamics.org/cgi-bin/mailman/listinfo/cig-seismo > -- Dimitri Komatitsch, CNRS Research Director (DR CNRS) Laboratory of Mechanics and Acoustics, Marseille, France http://komatitsch.free.fr From hgharti at princeton.edu Wed Apr 12 04:24:14 2017 From: hgharti at princeton.edu (Hom Nath Gharti) Date: Wed, 12 Apr 2017 07:24:14 -0400 Subject: [CIG-SEISMO] single force solution In-Reply-To: <0be21ac50a984e85b12745abd04ba911@CSGHUB209W.pu.win.princeton.edu> References: <7d7c9aae9ef741c1be7a87fd1dcc1c71@CSGHUB209W.pu.win.princeton.edu> <0be21ac50a984e85b12745abd04ba911@CSGHUB209W.pu.win.princeton.edu> Message-ID: Hi Dimitri, I will commit! Best, Hom Nath On Wed, Apr 12, 2017, 7:09 AM Dimitri Komatitsch wrote: > > Hi Hom Nath, > > Perfect, thanks! > Then please remember to commit it to GitHub. > > Best regards, > Dimitri. > > On 04/12/2017 01:06 PM, Hom Nath Gharti wrote: > > Hi Sévan, > > > > I am about to finish new point force implementation. I will let you know > > once it is ready for testing. > > > > Best, > > Hom Nath > > > > On Wed, Apr 12, 2017 at 6:36 AM, Dimitri Komatitsch > > > wrote: > > > > > > Hi Sevan, > > > > I guess you can keep cc'ing the list because it is an issue Clément, > > Vadim, Paul, Sébastien and I have been facing as well in our coupling > > with external codes, thus we are interested in knowing more about the > > potential problem. > > > > Thanks, > > Best wishes, > > Dimitri. > > > > On 04/12/2017 11:24 AM, Sevan Adourian wrote: > > > Hi Hom Nath, > > > > > > Thanks for your answer and concern. Since you suggest something > that is > > > not directly "specfem related", can I send you directly an email > so that > > > we can discuss it? And then I can post the solution once we'll > find it, > > > in order to close the topic! > > > > > > Thanks again, > > > Sévan > > > > > > On Tue, Apr 11, 2017 at 8:46 PM, Hom Nath Gharti < > hgharti at princeton.edu > > > >> > wrote: > > > > > > Hi Sévan, > > > > > > Do you also switch the orientations of source and receiver? > > > > > > Thanks, > > > Hom Nath > > > > > > On Tue, Apr 11, 2017 at 1:39 PM, Sevan Adourian > > > > > > > >> > > > wrote: > > > > > > Hi cig-seismo community, > > > > > > I'm trying to compute Green's functions using > Specfem3D-Globe, > > > using a single force point source solution coupled with a > > source > > > time function that basically represents a filtered Dirac > > delta. > > > In order to test my implementation, I imposed a source at a > > > point A and looked at the obtained Green's function at a > > > receiver B. Then, I switched the position of the source > > and the > > > receiver and compared the 2 in order to check the > > reciprocity of > > > those Green's functions. > > > > > > It turned out that I have a perfect reciprocity (in the > > > frequency band that correspond to my filtered Dirac delta) > > only > > > on the component in which I input the force, while the > other > > > components don't quite match. > > > > > > I strongly suspected a reference frame rotation issue, so I > > > tried to input the force in the global xyz coordinate > system > > > (i.e. adding the force directly in the accel_crust_mantle > > array > > > in the comp_add_sources subroutine, without multiplying > > with the > > > nu_source matrix) and looked at the Green's functions in > this > > > xyz coordinates system (ie, directly writing in the > > seismograms > > > structure the values of ux, uy and uz computed in the > > > compute_seismogram subroutine), but I still have this > issue. > > > Would anyone have an idea of where this probable rotation > > issue > > > could come from? > > > > > > I'm working on specfem3D-Globe v6.0.0. > > > > > > Thanks in advance for your help! > > > Sévan > > > > > > -- > > > Sevan Adourian > > > Masters Student at Ecole Normale Superieure de Paris > > > sevan.adourian at ens.fr > > > > > > > > > > > > > > > _______________________________________________ > > > CIG-SEISMO mailing list > > > CIG-SEISMO at geodynamics.org > > geodynamics.org>> > > > > > http://lists.geodynamics.org/cgi-bin/mailman/listinfo/cig-seismo > > > > > > > > > > > > > > > > > > > > > > > > -- > > > Sevan Adourian > > > Masters Student at Ecole Normale Superieure de Paris > > > sevan.adourian at ens.fr > > > > > > > > > > > > _______________________________________________ > > > CIG-SEISMO mailing list > > > CIG-SEISMO at geodynamics.org > > > http://lists.geodynamics.org/cgi-bin/mailman/listinfo/cig-seismo > > > > > > > > > -- > > Dimitri Komatitsch, CNRS Research Director (DR CNRS) > > Laboratory of Mechanics and Acoustics, Marseille, France > > http://komatitsch.free.fr > > _______________________________________________ > > CIG-SEISMO mailing list > > CIG-SEISMO at geodynamics.org > > http://lists.geodynamics.org/cgi-bin/mailman/listinfo/cig-seismo > > > > > > > > > > > > _______________________________________________ > > CIG-SEISMO mailing list > > CIG-SEISMO at geodynamics.org > > http://lists.geodynamics.org/cgi-bin/mailman/listinfo/cig-seismo > > > > -- > Dimitri Komatitsch, CNRS Research Director (DR CNRS) > Laboratory of Mechanics and Acoustics, Marseille, France > http://komatitsch.free.fr > _______________________________________________ > CIG-SEISMO mailing list > CIG-SEISMO at geodynamics.org > http://lists.geodynamics.org/cgi-bin/mailman/listinfo/cig-seismo -------------- next part -------------- An HTML attachment was scrubbed... URL: From sevan.adourian at berkeley.edu Wed Apr 12 04:36:14 2017 From: sevan.adourian at berkeley.edu (Sevan Adourian) Date: Wed, 12 Apr 2017 13:36:14 +0200 Subject: [CIG-SEISMO] single force solution In-Reply-To: References: <7d7c9aae9ef741c1be7a87fd1dcc1c71@CSGHUB209W.pu.win.princeton.edu> <0be21ac50a984e85b12745abd04ba911@CSGHUB209W.pu.win.princeton.edu> Message-ID: Hi Hom Nath and Dimitri, Thank you Dimitri and Hom Nath for the feedback. Hom Nath, will your new implementation be on the model of what's done in specfem Cartesian (i.e. with the FORCESOLUTION file), and will it support a source that is not on a node of an element without relocating this source? And will it still use a ricker as a source time function? Thanks, Sévan On Wed, Apr 12, 2017 at 1:24 PM, Hom Nath Gharti wrote: > Hi Dimitri, > > I will commit! > > Best, > Hom Nath > > On Wed, Apr 12, 2017, 7:09 AM Dimitri Komatitsch < > komatitsch at lma.cnrs-mrs.fr> wrote: > >> >> Hi Hom Nath, >> >> Perfect, thanks! >> Then please remember to commit it to GitHub. >> >> Best regards, >> Dimitri. >> >> On 04/12/2017 01:06 PM, Hom Nath Gharti wrote: >> > Hi Sévan, >> > >> > I am about to finish new point force implementation. I will let you know >> > once it is ready for testing. >> > >> > Best, >> > Hom Nath >> > >> > On Wed, Apr 12, 2017 at 6:36 AM, Dimitri Komatitsch >> > > wrote: >> > >> > >> > Hi Sevan, >> > >> > I guess you can keep cc'ing the list because it is an issue Clément, >> > Vadim, Paul, Sébastien and I have been facing as well in our >> coupling >> > with external codes, thus we are interested in knowing more about >> the >> > potential problem. >> > >> > Thanks, >> > Best wishes, >> > Dimitri. >> > >> > On 04/12/2017 11:24 AM, Sevan Adourian wrote: >> > > Hi Hom Nath, >> > > >> > > Thanks for your answer and concern. Since you suggest something >> that is >> > > not directly "specfem related", can I send you directly an email >> so that >> > > we can discuss it? And then I can post the solution once we'll >> find it, >> > > in order to close the topic! >> > > >> > > Thanks again, >> > > Sévan >> > > >> > > On Tue, Apr 11, 2017 at 8:46 PM, Hom Nath Gharti < >> hgharti at princeton.edu >> > > >> >> wrote: >> > > >> > > Hi Sévan, >> > > >> > > Do you also switch the orientations of source and receiver? >> > > >> > > Thanks, >> > > Hom Nath >> > > >> > > On Tue, Apr 11, 2017 at 1:39 PM, Sevan Adourian >> > > > > >> > > >> > >> >> > > wrote: >> > > >> > > Hi cig-seismo community, >> > > >> > > I'm trying to compute Green's functions using >> Specfem3D-Globe, >> > > using a single force point source solution coupled with a >> > source >> > > time function that basically represents a filtered Dirac >> > delta. >> > > In order to test my implementation, I imposed a source at >> a >> > > point A and looked at the obtained Green's function at a >> > > receiver B. Then, I switched the position of the source >> > and the >> > > receiver and compared the 2 in order to check the >> > reciprocity of >> > > those Green's functions. >> > > >> > > It turned out that I have a perfect reciprocity (in the >> > > frequency band that correspond to my filtered Dirac delta) >> > only >> > > on the component in which I input the force, while the >> other >> > > components don't quite match. >> > > >> > > I strongly suspected a reference frame rotation issue, so >> I >> > > tried to input the force in the global xyz coordinate >> system >> > > (i.e. adding the force directly in the accel_crust_mantle >> > array >> > > in the comp_add_sources subroutine, without multiplying >> > with the >> > > nu_source matrix) and looked at the Green's functions in >> this >> > > xyz coordinates system (ie, directly writing in the >> > seismograms >> > > structure the values of ux, uy and uz computed in the >> > > compute_seismogram subroutine), but I still have this >> issue. >> > > Would anyone have an idea of where this probable rotation >> > issue >> > > could come from? >> > > >> > > I'm working on specfem3D-Globe v6.0.0. >> > > >> > > Thanks in advance for your help! >> > > Sévan >> > > >> > > -- >> > > Sevan Adourian >> > > Masters Student at Ecole Normale Superieure de Paris >> > > sevan.adourian at ens.fr >> > > >> > > >> > > >> > > >> > > _______________________________________________ >> > > CIG-SEISMO mailing list >> > > CIG-SEISMO at geodynamics.org > > >> > > .org>> >> > > >> > http://lists.geodynamics.org/cgi-bin/mailman/listinfo/cig-seismo >> > >> > > >> > > > > >> > > >> > > >> > > >> > > >> > > -- >> > > Sevan Adourian >> > > Masters Student at Ecole Normale Superieure de Paris >> > > sevan.adourian at ens.fr >> > > >> > > >> > > >> > > _______________________________________________ >> > > CIG-SEISMO mailing list >> > > CIG-SEISMO at geodynamics.org >> > > http://lists.geodynamics.org/cgi-bin/mailman/listinfo/cig-seismo >> > >> > > >> > >> > -- >> > Dimitri Komatitsch, CNRS Research Director (DR CNRS) >> > Laboratory of Mechanics and Acoustics, Marseille, France >> > http://komatitsch.free.fr >> > _______________________________________________ >> > CIG-SEISMO mailing list >> > CIG-SEISMO at geodynamics.org >> > http://lists.geodynamics.org/cgi-bin/mailman/listinfo/cig-seismo >> > >> > >> > >> > >> > >> > _______________________________________________ >> > CIG-SEISMO mailing list >> > CIG-SEISMO at geodynamics.org >> > http://lists.geodynamics.org/cgi-bin/mailman/listinfo/cig-seismo >> > >> >> -- >> Dimitri Komatitsch, CNRS Research Director (DR CNRS) >> Laboratory of Mechanics and Acoustics, Marseille, France >> http://komatitsch.free.fr >> _______________________________________________ >> CIG-SEISMO mailing list >> CIG-SEISMO at geodynamics.org >> http://lists.geodynamics.org/cgi-bin/mailman/listinfo/cig-seismo > > > _______________________________________________ > CIG-SEISMO mailing list > CIG-SEISMO at geodynamics.org > http://lists.geodynamics.org/cgi-bin/mailman/listinfo/cig-seismo > -- Sevan Adourian Masters Student at Ecole Normale Superieure de Paris sevan.adourian at ens.fr -------------- next part -------------- An HTML attachment was scrubbed... URL: From hgharti at princeton.edu Wed Apr 12 04:51:28 2017 From: hgharti at princeton.edu (Hom Nath Gharti) Date: Wed, 12 Apr 2017 11:51:28 +0000 Subject: [CIG-SEISMO] single force solution In-Reply-To: References: <7d7c9aae9ef741c1be7a87fd1dcc1c71@CSGHUB209W.pu.win.princeton.edu> <0be21ac50a984e85b12745abd04ba911@CSGHUB209W.pu.win.princeton.edu> Message-ID: Hi Sévan, New implementation is very similar, but the source time function will have several options. I make it more general because some people were interested on a moving source with sine source time function, for exampme. Best, Hom Nath On Wed, Apr 12, 2017, 7:36 AM Sevan Adourian wrote: > Hi Hom Nath and Dimitri, > > Thank you Dimitri and Hom Nath for the feedback. Hom Nath, will your new > implementation be on the model of what's done in specfem Cartesian (i.e. > with the FORCESOLUTION file), and will it support a source that is not on a > node of an element without relocating this source? And will it still use a > ricker as a source time function? > > Thanks, > Sévan > > > On Wed, Apr 12, 2017 at 1:24 PM, Hom Nath Gharti > wrote: > > Hi Dimitri, > > I will commit! > > Best, > Hom Nath > > On Wed, Apr 12, 2017, 7:09 AM Dimitri Komatitsch < > komatitsch at lma.cnrs-mrs.fr> wrote: > > > Hi Hom Nath, > > Perfect, thanks! > Then please remember to commit it to GitHub. > > Best regards, > Dimitri. > > On 04/12/2017 01:06 PM, Hom Nath Gharti wrote: > > Hi Sévan, > > > > I am about to finish new point force implementation. I will let you know > > once it is ready for testing. > > > > Best, > > Hom Nath > > > > On Wed, Apr 12, 2017 at 6:36 AM, Dimitri Komatitsch > > > wrote: > > > > > > Hi Sevan, > > > > I guess you can keep cc'ing the list because it is an issue Clément, > > Vadim, Paul, Sébastien and I have been facing as well in our coupling > > with external codes, thus we are interested in knowing more about the > > potential problem. > > > > Thanks, > > Best wishes, > > Dimitri. > > > > On 04/12/2017 11:24 AM, Sevan Adourian wrote: > > > Hi Hom Nath, > > > > > > Thanks for your answer and concern. Since you suggest something > that is > > > not directly "specfem related", can I send you directly an email > so that > > > we can discuss it? And then I can post the solution once we'll > find it, > > > in order to close the topic! > > > > > > Thanks again, > > > Sévan > > > > > > On Tue, Apr 11, 2017 at 8:46 PM, Hom Nath Gharti < > hgharti at princeton.edu > > > >> > wrote: > > > > > > Hi Sévan, > > > > > > Do you also switch the orientations of source and receiver? > > > > > > Thanks, > > > Hom Nath > > > > > > On Tue, Apr 11, 2017 at 1:39 PM, Sevan Adourian > > > > > > > > >> > > > wrote: > > > > > > Hi cig-seismo community, > > > > > > I'm trying to compute Green's functions using > Specfem3D-Globe, > > > using a single force point source solution coupled with a > > source > > > time function that basically represents a filtered Dirac > > delta. > > > In order to test my implementation, I imposed a source at a > > > point A and looked at the obtained Green's function at a > > > receiver B. Then, I switched the position of the source > > and the > > > receiver and compared the 2 in order to check the > > reciprocity of > > > those Green's functions. > > > > > > It turned out that I have a perfect reciprocity (in the > > > frequency band that correspond to my filtered Dirac delta) > > only > > > on the component in which I input the force, while the > other > > > components don't quite match. > > > > > > I strongly suspected a reference frame rotation issue, so I > > > tried to input the force in the global xyz coordinate > system > > > (i.e. adding the force directly in the accel_crust_mantle > > array > > > in the comp_add_sources subroutine, without multiplying > > with the > > > nu_source matrix) and looked at the Green's functions in > this > > > xyz coordinates system (ie, directly writing in the > > seismograms > > > structure the values of ux, uy and uz computed in the > > > compute_seismogram subroutine), but I still have this > issue. > > > Would anyone have an idea of where this probable rotation > > issue > > > could come from? > > > > > > I'm working on specfem3D-Globe v6.0.0. > > > > > > Thanks in advance for your help! > > > Sévan > > > > > > -- > > > Sevan Adourian > > > Masters Student at Ecole Normale Superieure de Paris > > > sevan.adourian at ens.fr > > > > > > > > > > > > > > > _______________________________________________ > > > CIG-SEISMO mailing list > > > CIG-SEISMO at geodynamics.org > > CIG-SEISMO at geodynamics.org>> > > > > > http://lists.geodynamics.org/cgi-bin/mailman/listinfo/cig-seismo > > > > > > > > > > > > > > > > > > > > > > > > -- > > > Sevan Adourian > > > Masters Student at Ecole Normale Superieure de Paris > > > sevan.adourian at ens.fr > > > > > > > > > > > > _______________________________________________ > > > CIG-SEISMO mailing list > > > CIG-SEISMO at geodynamics.org > > > http://lists.geodynamics.org/cgi-bin/mailman/listinfo/cig-seismo > > > > > > > > > -- > > Dimitri Komatitsch, CNRS Research Director (DR CNRS) > > Laboratory of Mechanics and Acoustics, Marseille, France > > http://komatitsch.free.fr > > _______________________________________________ > > CIG-SEISMO mailing list > > CIG-SEISMO at geodynamics.org > > http://lists.geodynamics.org/cgi-bin/mailman/listinfo/cig-seismo > > > > > > > > > > > > _______________________________________________ > > CIG-SEISMO mailing list > > CIG-SEISMO at geodynamics.org > > http://lists.geodynamics.org/cgi-bin/mailman/listinfo/cig-seismo > > > > -- > Dimitri Komatitsch, CNRS Research Director (DR CNRS) > Laboratory of Mechanics and Acoustics, Marseille, France > http://komatitsch.free.fr > _______________________________________________ > CIG-SEISMO mailing list > CIG-SEISMO at geodynamics.org > http://lists.geodynamics.org/cgi-bin/mailman/listinfo/cig-seismo > > > _______________________________________________ > CIG-SEISMO mailing list > CIG-SEISMO at geodynamics.org > http://lists.geodynamics.org/cgi-bin/mailman/listinfo/cig-seismo > > > > > -- > Sevan Adourian > Masters Student at Ecole Normale Superieure de Paris > sevan.adourian at ens.fr > -------------- next part -------------- An HTML attachment was scrubbed... URL: From sevan.adourian at berkeley.edu Wed Apr 12 06:01:42 2017 From: sevan.adourian at berkeley.edu (Sevan Adourian) Date: Wed, 12 Apr 2017 15:01:42 +0200 Subject: [CIG-SEISMO] single force solution In-Reply-To: References: <7d7c9aae9ef741c1be7a87fd1dcc1c71@CSGHUB209W.pu.win.princeton.edu> <0be21ac50a984e85b12745abd04ba911@CSGHUB209W.pu.win.princeton.edu> Message-ID: Hi Hom Nath and Dimitri, Alright then, I'll be waiting for the tests of your new version then! Dimitri, may I ask you which problems you have faced in the Green's functions computation you mentioned earlier? Do you also have one component that is satisfying reciprocity? I'm asking because I'm trying to couple specfem with another external code as well. Thanks and regards, Sévan On Wed, Apr 12, 2017 at 1:51 PM, Hom Nath Gharti wrote: > Hi Sévan, > > New implementation is very similar, but the source time function will have > several options. I make it more general because some people were interested > on a moving source with sine source time function, for exampme. > > Best, > Hom Nath > > On Wed, Apr 12, 2017, 7:36 AM Sevan Adourian > wrote: > >> Hi Hom Nath and Dimitri, >> >> Thank you Dimitri and Hom Nath for the feedback. Hom Nath, will your new >> implementation be on the model of what's done in specfem Cartesian (i.e. >> with the FORCESOLUTION file), and will it support a source that is not on a >> node of an element without relocating this source? And will it still use a >> ricker as a source time function? >> >> Thanks, >> Sévan >> >> >> On Wed, Apr 12, 2017 at 1:24 PM, Hom Nath Gharti >> wrote: >> >> Hi Dimitri, >> >> I will commit! >> >> Best, >> Hom Nath >> >> On Wed, Apr 12, 2017, 7:09 AM Dimitri Komatitsch < >> komatitsch at lma.cnrs-mrs.fr> wrote: >> >> >> Hi Hom Nath, >> >> Perfect, thanks! >> Then please remember to commit it to GitHub. >> >> Best regards, >> Dimitri. >> >> On 04/12/2017 01:06 PM, Hom Nath Gharti wrote: >> > Hi Sévan, >> > >> > I am about to finish new point force implementation. I will let you know >> > once it is ready for testing. >> > >> > Best, >> > Hom Nath >> > >> > On Wed, Apr 12, 2017 at 6:36 AM, Dimitri Komatitsch >> > > wrote: >> > >> > >> > Hi Sevan, >> > >> > I guess you can keep cc'ing the list because it is an issue Clément, >> > Vadim, Paul, Sébastien and I have been facing as well in our >> coupling >> > with external codes, thus we are interested in knowing more about >> the >> > potential problem. >> > >> > Thanks, >> > Best wishes, >> > Dimitri. >> > >> > On 04/12/2017 11:24 AM, Sevan Adourian wrote: >> > > Hi Hom Nath, >> > > >> > > Thanks for your answer and concern. Since you suggest something >> that is >> > > not directly "specfem related", can I send you directly an email >> so that >> > > we can discuss it? And then I can post the solution once we'll >> find it, >> > > in order to close the topic! >> > > >> > > Thanks again, >> > > Sévan >> > > >> > > On Tue, Apr 11, 2017 at 8:46 PM, Hom Nath Gharti < >> hgharti at princeton.edu >> > > >> >> wrote: >> > > >> > > Hi Sévan, >> > > >> > > Do you also switch the orientations of source and receiver? >> > > >> > > Thanks, >> > > Hom Nath >> > > >> > > On Tue, Apr 11, 2017 at 1:39 PM, Sevan Adourian >> > > > > >> > > >> > >> >> > > wrote: >> > > >> > > Hi cig-seismo community, >> > > >> > > I'm trying to compute Green's functions using >> Specfem3D-Globe, >> > > using a single force point source solution coupled with a >> > source >> > > time function that basically represents a filtered Dirac >> > delta. >> > > In order to test my implementation, I imposed a source at >> a >> > > point A and looked at the obtained Green's function at a >> > > receiver B. Then, I switched the position of the source >> > and the >> > > receiver and compared the 2 in order to check the >> > reciprocity of >> > > those Green's functions. >> > > >> > > It turned out that I have a perfect reciprocity (in the >> > > frequency band that correspond to my filtered Dirac delta) >> > only >> > > on the component in which I input the force, while the >> other >> > > components don't quite match. >> > > >> > > I strongly suspected a reference frame rotation issue, so >> I >> > > tried to input the force in the global xyz coordinate >> system >> > > (i.e. adding the force directly in the accel_crust_mantle >> > array >> > > in the comp_add_sources subroutine, without multiplying >> > with the >> > > nu_source matrix) and looked at the Green's functions in >> this >> > > xyz coordinates system (ie, directly writing in the >> > seismograms >> > > structure the values of ux, uy and uz computed in the >> > > compute_seismogram subroutine), but I still have this >> issue. >> > > Would anyone have an idea of where this probable rotation >> > issue >> > > could come from? >> > > >> > > I'm working on specfem3D-Globe v6.0.0. >> > > >> > > Thanks in advance for your help! >> > > Sévan >> > > >> > > -- >> > > Sevan Adourian >> > > Masters Student at Ecole Normale Superieure de Paris >> > > sevan.adourian at ens.fr >> > > >> > > >> > > >> > > >> > > _______________________________________________ >> > > CIG-SEISMO mailing list >> > > CIG-SEISMO at geodynamics.org > > >> > > geodynamics.org>> >> > > >> > http://lists.geodynamics.org/cgi-bin/mailman/listinfo/cig-seismo >> > >> > > >> > > > > >> > > >> > > >> > > >> > > >> > > -- >> > > Sevan Adourian >> > > Masters Student at Ecole Normale Superieure de Paris >> > > sevan.adourian at ens.fr >> > > >> > > >> > > >> > > _______________________________________________ >> > > CIG-SEISMO mailing list >> > > CIG-SEISMO at geodynamics.org >> > > http://lists.geodynamics.org/cgi-bin/mailman/listinfo/cig-seismo >> > >> > > >> > >> > -- >> > Dimitri Komatitsch, CNRS Research Director (DR CNRS) >> > Laboratory of Mechanics and Acoustics, Marseille, France >> > http://komatitsch.free.fr >> > _______________________________________________ >> > CIG-SEISMO mailing list >> > CIG-SEISMO at geodynamics.org >> > http://lists.geodynamics.org/cgi-bin/mailman/listinfo/cig-seismo >> > >> > >> > >> > >> > >> > _______________________________________________ >> > CIG-SEISMO mailing list >> > CIG-SEISMO at geodynamics.org >> > http://lists.geodynamics.org/cgi-bin/mailman/listinfo/cig-seismo >> > >> >> -- >> Dimitri Komatitsch, CNRS Research Director (DR CNRS) >> Laboratory of Mechanics and Acoustics, Marseille, France >> http://komatitsch.free.fr >> _______________________________________________ >> CIG-SEISMO mailing list >> CIG-SEISMO at geodynamics.org >> http://lists.geodynamics.org/cgi-bin/mailman/listinfo/cig-seismo >> >> >> _______________________________________________ >> CIG-SEISMO mailing list >> CIG-SEISMO at geodynamics.org >> http://lists.geodynamics.org/cgi-bin/mailman/listinfo/cig-seismo >> >> >> >> >> -- >> Sevan Adourian >> Masters Student at Ecole Normale Superieure de Paris >> sevan.adourian at ens.fr >> > > _______________________________________________ > CIG-SEISMO mailing list > CIG-SEISMO at geodynamics.org > http://lists.geodynamics.org/cgi-bin/mailman/listinfo/cig-seismo > -- Sevan Adourian Masters Student at Ecole Normale Superieure de Paris sevan.adourian at ens.fr -------------- next part -------------- An HTML attachment was scrubbed... URL: From komatitsch at lma.cnrs-mrs.fr Wed Apr 12 06:10:09 2017 From: komatitsch at lma.cnrs-mrs.fr (Dimitri Komatitsch) Date: Wed, 12 Apr 2017 15:10:09 +0200 Subject: [CIG-SEISMO] single force solution In-Reply-To: References: <7d7c9aae9ef741c1be7a87fd1dcc1c71@CSGHUB209W.pu.win.princeton.edu> <0be21ac50a984e85b12745abd04ba911@CSGHUB209W.pu.win.princeton.edu> Message-ID: <82087a05-fbab-7c64-6c37-fec864274887@lma.cnrs-mrs.fr> Hi Sevan, Yes, in earlier work (http://komatitsch.free.fr/preprints/GJI2_Vadim_2015.pdf and http://komatitsch.free.fr/preprints/Geology_Chevrot_2016.pdf) we have coupled SPECFEM3D with DSM and that worked fine, however more recently we have tried to couple SPECFEM3D with AxiSEM to do FWI locally with our wavefield injection technique (also similar to e.g. Masson and Romanowicz's box tomography approach) and so far we get some weird extra phases, possibly something we are doing wrong in some rotations. Not sure if the problem is the same as yours, it could also be a small mistake we made in the representation theorem coupling integral or something like that. Since you are at Berkeley, is this related to what Yder was doing? (I think he was using finite differences, not SPECFEM?). Thanks, Best wishes, Dimitri. On 04/12/2017 03:01 PM, Sevan Adourian wrote: > Hi Hom Nath and Dimitri, > > Alright then, I'll be waiting for the tests of your new version then! > Dimitri, may I ask you which problems you have faced in the Green's > functions computation you mentioned earlier? Do you also have one > component that is satisfying reciprocity? I'm asking because I'm trying > to couple specfem with another external code as well. > > Thanks and regards, > Sévan > > On Wed, Apr 12, 2017 at 1:51 PM, Hom Nath Gharti > wrote: > > Hi Sévan, > > New implementation is very similar, but the source time function > will have several options. I make it more general because some > people were interested on a moving source with sine source time > function, for exampme. > > Best, > Hom Nath > > > On Wed, Apr 12, 2017, 7:36 AM Sevan Adourian > > > wrote: > > Hi Hom Nath and Dimitri, > > Thank you Dimitri and Hom Nath for the feedback. Hom Nath, will > your new implementation be on the model of what's done in > specfem Cartesian (i.e. with the FORCESOLUTION file), and will > it support a source that is not on a node of an element without > relocating this source? And will it still use a ricker as a > source time function? > > Thanks, > Sévan > > > On Wed, Apr 12, 2017 at 1:24 PM, Hom Nath Gharti > > wrote: > > Hi Dimitri, > > I will commit! > > Best, > Hom Nath > > On Wed, Apr 12, 2017, 7:09 AM Dimitri Komatitsch > > wrote: > > > Hi Hom Nath, > > Perfect, thanks! > Then please remember to commit it to GitHub. > > Best regards, > Dimitri. > > On 04/12/2017 01:06 PM, Hom Nath Gharti wrote: > > Hi Sévan, > > > > I am about to finish new point force implementation. I will let you know > > once it is ready for testing. > > > > Best, > > Hom Nath > > > > On Wed, Apr 12, 2017 at 6:36 AM, Dimitri Komatitsch > > > >> wrote: > > > > > > Hi Sevan, > > > > I guess you can keep cc'ing the list because it is an issue Clément, > > Vadim, Paul, Sébastien and I have been facing as well in our coupling > > with external codes, thus we are interested in knowing more about the > > potential problem. > > > > Thanks, > > Best wishes, > > Dimitri. > > > > On 04/12/2017 11:24 AM, Sevan Adourian wrote: > > > Hi Hom Nath, > > > > > > Thanks for your answer and concern. Since you suggest something that is > > > not directly "specfem related", can I send you directly an email so that > > > we can discuss it? And then I can post the solution once we'll find it, > > > in order to close the topic! > > > > > > Thanks again, > > > Sévan > > > > > > On Tue, Apr 11, 2017 at 8:46 PM, Hom Nath Gharti > > > > > > >>> wrote: > > > > > > Hi Sévan, > > > > > > Do you also switch the orientations of source and receiver? > > > > > > Thanks, > > > Hom Nath > > > > > > On Tue, Apr 11, 2017 at 1:39 PM, Sevan Adourian > > > > > > > > > > > >>> > > > wrote: > > > > > > Hi cig-seismo community, > > > > > > I'm trying to compute Green's functions > using Specfem3D-Globe, > > > using a single force point source > solution coupled with a > > source > > > time function that basically represents > a filtered Dirac > > delta. > > > In order to test my implementation, I > imposed a source at a > > > point A and looked at the obtained > Green's function at a > > > receiver B. Then, I switched the > position of the source > > and the > > > receiver and compared the 2 in order to > check the > > reciprocity of > > > those Green's functions. > > > > > > It turned out that I have a perfect > reciprocity (in the > > > frequency band that correspond to my > filtered Dirac delta) > > only > > > on the component in which I input the > force, while the other > > > components don't quite match. > > > > > > I strongly suspected a reference frame > rotation issue, so I > > > tried to input the force in the global > xyz coordinate system > > > (i.e. adding the force directly in the > accel_crust_mantle > > array > > > in the comp_add_sources subroutine, > without multiplying > > with the > > > nu_source matrix) and looked at the > Green's functions in this > > > xyz coordinates system (ie, directly > writing in the > > seismograms > > > structure the values of ux, uy and uz > computed in the > > > compute_seismogram subroutine), but I > still have this issue. > > > Would anyone have an idea of where this > probable rotation > > issue > > > could come from? > > > > > > I'm working on specfem3D-Globe v6.0.0. > > > > > > Thanks in advance for your help! > > > Sévan > > > > > > -- > > > Sevan Adourian > > > Masters Student at Ecole Normale > Superieure de Paris > > > sevan.adourian at ens.fr > > > > > > >> > > > > > > > > > > > > _______________________________________________ > > > CIG-SEISMO mailing list > > > CIG-SEISMO at geodynamics.org > > > > > > >> > > > > > http://lists.geodynamics.org/cgi-bin/mailman/listinfo/cig-seismo > > > > > > > > > > > >> > > > > > > > > > > > > > > > -- > > > Sevan Adourian > > > Masters Student at Ecole Normale Superieure de Paris > > > sevan.adourian at ens.fr > > > > > >> > > > > > > > > > _______________________________________________ > > > CIG-SEISMO mailing list > > > CIG-SEISMO at geodynamics.org > > > > > > http://lists.geodynamics.org/cgi-bin/mailman/listinfo/cig-seismo > > > > > > > > > > > -- > > Dimitri Komatitsch, CNRS Research Director (DR CNRS) > > Laboratory of Mechanics and Acoustics, Marseille, France > > http://komatitsch.free.fr > > _______________________________________________ > > CIG-SEISMO mailing list > > CIG-SEISMO at geodynamics.org > > > > > http://lists.geodynamics.org/cgi-bin/mailman/listinfo/cig-seismo > > > > > > > > > > > > > > _______________________________________________ > > CIG-SEISMO mailing list > > CIG-SEISMO at geodynamics.org > > > http://lists.geodynamics.org/cgi-bin/mailman/listinfo/cig-seismo > > > > > -- > Dimitri Komatitsch, CNRS Research Director (DR CNRS) > Laboratory of Mechanics and Acoustics, Marseille, France > http://komatitsch.free.fr > _______________________________________________ > CIG-SEISMO mailing list > CIG-SEISMO at geodynamics.org > > http://lists.geodynamics.org/cgi-bin/mailman/listinfo/cig-seismo > > > > _______________________________________________ > CIG-SEISMO mailing list > CIG-SEISMO at geodynamics.org > http://lists.geodynamics.org/cgi-bin/mailman/listinfo/cig-seismo > > > > > > -- > Sevan Adourian > Masters Student at Ecole Normale Superieure de Paris > sevan.adourian at ens.fr > > > _______________________________________________ > CIG-SEISMO mailing list > CIG-SEISMO at geodynamics.org > http://lists.geodynamics.org/cgi-bin/mailman/listinfo/cig-seismo > > > > > > -- > Sevan Adourian > Masters Student at Ecole Normale Superieure de Paris > sevan.adourian at ens.fr > > > _______________________________________________ > CIG-SEISMO mailing list > CIG-SEISMO at geodynamics.org > http://lists.geodynamics.org/cgi-bin/mailman/listinfo/cig-seismo > -- Dimitri Komatitsch, CNRS Research Director (DR CNRS) Laboratory of Mechanics and Acoustics, Marseille, France http://komatitsch.free.fr From sevan.adourian at berkeley.edu Wed Apr 12 06:26:46 2017 From: sevan.adourian at berkeley.edu (Sevan Adourian) Date: Wed, 12 Apr 2017 15:26:46 +0200 Subject: [CIG-SEISMO] single force solution In-Reply-To: <82087a05-fbab-7c64-6c37-fec864274887@lma.cnrs-mrs.fr> References: <7d7c9aae9ef741c1be7a87fd1dcc1c71@CSGHUB209W.pu.win.princeton.edu> <0be21ac50a984e85b12745abd04ba911@CSGHUB209W.pu.win.princeton.edu> <82087a05-fbab-7c64-6c37-fec864274887@lma.cnrs-mrs.fr> Message-ID: Hi Dimitri, Thanks for your answer! Yes, I'm continuing Yder's work on box tomography, and I'm trying to allow the stations to be outside of the box (which is why I need to compute the Green's functions from an "outside" receiver to each point of the mirror for a later convolution with the wavefield). In my case I need to couple SPECFEM with a slightly modified RegSEM. But reading what you describe, the problem might still come from me because the issues I'm facing are on the SPECFEM side only. So if you tell me it already worked, I might still do something wrong. Regards, Sévan On Wed, Apr 12, 2017 at 3:10 PM, Dimitri Komatitsch < komatitsch at lma.cnrs-mrs.fr> wrote: > > Hi Sevan, > > Yes, in earlier work (http://komatitsch.free.fr/pre > prints/GJI2_Vadim_2015.pdf and http://komatitsch.free.fr/prep > rints/Geology_Chevrot_2016.pdf) we have coupled SPECFEM3D with DSM and > that worked fine, however more recently we have tried to couple SPECFEM3D > with AxiSEM to do FWI locally with our wavefield injection technique (also > similar to e.g. Masson and Romanowicz's box tomography approach) and so far > we get some weird extra phases, possibly something we are doing wrong in > some rotations. Not sure if the problem is the same as yours, it could also > be a small mistake we made in the representation theorem coupling integral > or something like that. > > Since you are at Berkeley, is this related to what Yder was doing? (I > think he was using finite differences, not SPECFEM?). > > Thanks, > Best wishes, > > Dimitri. > > On 04/12/2017 03:01 PM, Sevan Adourian wrote: > >> Hi Hom Nath and Dimitri, >> >> Alright then, I'll be waiting for the tests of your new version then! >> Dimitri, may I ask you which problems you have faced in the Green's >> functions computation you mentioned earlier? Do you also have one >> component that is satisfying reciprocity? I'm asking because I'm trying >> to couple specfem with another external code as well. >> >> Thanks and regards, >> Sévan >> >> On Wed, Apr 12, 2017 at 1:51 PM, Hom Nath Gharti > > wrote: >> >> Hi Sévan, >> >> New implementation is very similar, but the source time function >> will have several options. I make it more general because some >> people were interested on a moving source with sine source time >> function, for exampme. >> >> Best, >> Hom Nath >> >> >> On Wed, Apr 12, 2017, 7:36 AM Sevan Adourian >> > >> wrote: >> >> Hi Hom Nath and Dimitri, >> >> Thank you Dimitri and Hom Nath for the feedback. Hom Nath, will >> your new implementation be on the model of what's done in >> specfem Cartesian (i.e. with the FORCESOLUTION file), and will >> it support a source that is not on a node of an element without >> relocating this source? And will it still use a ricker as a >> source time function? >> >> Thanks, >> Sévan >> >> >> On Wed, Apr 12, 2017 at 1:24 PM, Hom Nath Gharti >> > wrote: >> >> Hi Dimitri, >> >> I will commit! >> >> Best, >> Hom Nath >> >> On Wed, Apr 12, 2017, 7:09 AM Dimitri Komatitsch >> > > wrote: >> >> >> Hi Hom Nath, >> >> Perfect, thanks! >> Then please remember to commit it to GitHub. >> >> Best regards, >> Dimitri. >> >> On 04/12/2017 01:06 PM, Hom Nath Gharti wrote: >> > Hi Sévan, >> > >> > I am about to finish new point force implementation. I >> will let you know >> > once it is ready for testing. >> > >> > Best, >> > Hom Nath >> > >> > On Wed, Apr 12, 2017 at 6:36 AM, Dimitri Komatitsch >> > > >> > >> >> wrote: >> > >> > >> > Hi Sevan, >> > >> > I guess you can keep cc'ing the list because it is >> an issue Clément, >> > Vadim, Paul, Sébastien and I have been facing as >> well in our coupling >> > with external codes, thus we are interested in >> knowing more about the >> > potential problem. >> > >> > Thanks, >> > Best wishes, >> > Dimitri. >> > >> > On 04/12/2017 11:24 AM, Sevan Adourian wrote: >> > > Hi Hom Nath, >> > > >> > > Thanks for your answer and concern. Since you >> suggest something that is >> > > not directly "specfem related", can I send you >> directly an email so that >> > > we can discuss it? And then I can post the >> solution once we'll find it, >> > > in order to close the topic! >> > > >> > > Thanks again, >> > > Sévan >> > > >> > > On Tue, Apr 11, 2017 at 8:46 PM, Hom Nath Gharti < >> hgharti at princeton.edu >> > > >> > > > hgharti at princeton.edu> >> > >>> wrote: >> > > >> > > Hi Sévan, >> > > >> > > Do you also switch the orientations of source >> and receiver? >> > > >> > > Thanks, >> > > Hom Nath >> > > >> > > On Tue, Apr 11, 2017 at 1:39 PM, Sevan >> Adourian >> > > > >> > > > >> > > >> >> > > >>> >> > > wrote: >> > > >> > > Hi cig-seismo community, >> > > >> > > I'm trying to compute Green's functions >> using Specfem3D-Globe, >> > > using a single force point source >> solution coupled with a >> > source >> > > time function that basically represents >> a filtered Dirac >> > delta. >> > > In order to test my implementation, I >> imposed a source at a >> > > point A and looked at the obtained >> Green's function at a >> > > receiver B. Then, I switched the >> position of the source >> > and the >> > > receiver and compared the 2 in order to >> check the >> > reciprocity of >> > > those Green's functions. >> > > >> > > It turned out that I have a perfect >> reciprocity (in the >> > > frequency band that correspond to my >> filtered Dirac delta) >> > only >> > > on the component in which I input the >> force, while the other >> > > components don't quite match. >> > > >> > > I strongly suspected a reference frame >> rotation issue, so I >> > > tried to input the force in the global >> xyz coordinate system >> > > (i.e. adding the force directly in the >> accel_crust_mantle >> > array >> > > in the comp_add_sources subroutine, >> without multiplying >> > with the >> > > nu_source matrix) and looked at the >> Green's functions in this >> > > xyz coordinates system (ie, directly >> writing in the >> > seismograms >> > > structure the values of ux, uy and uz >> computed in the >> > > compute_seismogram subroutine), but I >> still have this issue. >> > > Would anyone have an idea of where this >> probable rotation >> > issue >> > > could come from? >> > > >> > > I'm working on specfem3D-Globe v6.0.0. >> > > >> > > Thanks in advance for your help! >> > > Sévan >> > > >> > > -- >> > > Sevan Adourian >> > > Masters Student at Ecole Normale >> Superieure de Paris >> > > sevan.adourian at ens.fr >> >> > > >> > > >> > >> >> > > >> > > >> > > >> > > _____________________________ >> __________________ >> > > CIG-SEISMO mailing list >> > > CIG-SEISMO at geodynamics.org >> >> > > >> > > >> > >> >> > > >> > http://lists.geodynamics.org/c >> gi-bin/mailman/listinfo/cig-seismo >> > cgi-bin/mailman/listinfo/cig-seismo> >> > > org/cgi-bin/mailman/listinfo/cig-seismo >> > cgi-bin/mailman/listinfo/cig-seismo>> >> > > >> > > cgi-bin/mailman/listinfo/cig-seismo >> > cgi-bin/mailman/listinfo/cig-seismo> >> > > org/cgi-bin/mailman/listinfo/cig-seismo >> > cgi-bin/mailman/listinfo/cig-seismo>>> >> > > >> > > >> > > >> > > >> > > -- >> > > Sevan Adourian >> > > Masters Student at Ecole Normale Superieure de >> Paris >> > > sevan.adourian at ens.fr > sevan.adourian at ens.fr> >> > > >> > > >> > >> >> > > >> > > >> > > _______________________________________________ >> > > CIG-SEISMO mailing list >> > > CIG-SEISMO at geodynamics.org >> >> > > >> > > http://lists.geodynamics.org/c >> gi-bin/mailman/listinfo/cig-seismo >> > cgi-bin/mailman/listinfo/cig-seismo> >> > > org/cgi-bin/mailman/listinfo/cig-seismo >> > cgi-bin/mailman/listinfo/cig-seismo>> >> > > >> > >> > -- >> > Dimitri Komatitsch, CNRS Research Director (DR CNRS) >> > Laboratory of Mechanics and Acoustics, Marseille, >> France >> > http://komatitsch.free.fr >> > _______________________________________________ >> > CIG-SEISMO mailing list >> > CIG-SEISMO at geodynamics.org >> >> > > >> > http://lists.geodynamics.org/ >> cgi-bin/mailman/listinfo/cig-seismo >> > cgi-bin/mailman/listinfo/cig-seismo> >> > > org/cgi-bin/mailman/listinfo/cig-seismo >> > cgi-bin/mailman/listinfo/cig-seismo>> >> > >> > >> > >> > >> > _______________________________________________ >> > CIG-SEISMO mailing list >> > CIG-SEISMO at geodynamics.org >> >> > http://lists.geodynamics.org/c >> gi-bin/mailman/listinfo/cig-seismo >> > cgi-bin/mailman/listinfo/cig-seismo> >> > >> >> -- >> Dimitri Komatitsch, CNRS Research Director (DR CNRS) >> Laboratory of Mechanics and Acoustics, Marseille, France >> http://komatitsch.free.fr >> _______________________________________________ >> CIG-SEISMO mailing list >> CIG-SEISMO at geodynamics.org >> >> http://lists.geodynamics.org/c >> gi-bin/mailman/listinfo/cig-seismo >> > cgi-bin/mailman/listinfo/cig-seismo> >> >> >> _______________________________________________ >> CIG-SEISMO mailing list >> CIG-SEISMO at geodynamics.org > > >> http://lists.geodynamics.org/cgi-bin/mailman/listinfo/cig-se >> ismo >> > eismo> >> >> >> >> >> -- >> Sevan Adourian >> Masters Student at Ecole Normale Superieure de Paris >> sevan.adourian at ens.fr >> >> >> _______________________________________________ >> CIG-SEISMO mailing list >> CIG-SEISMO at geodynamics.org >> http://lists.geodynamics.org/cgi-bin/mailman/listinfo/cig-seismo >> >> >> >> >> >> -- >> Sevan Adourian >> Masters Student at Ecole Normale Superieure de Paris >> sevan.adourian at ens.fr >> >> >> _______________________________________________ >> CIG-SEISMO mailing list >> CIG-SEISMO at geodynamics.org >> http://lists.geodynamics.org/cgi-bin/mailman/listinfo/cig-seismo >> >> > -- > Dimitri Komatitsch, CNRS Research Director (DR CNRS) > Laboratory of Mechanics and Acoustics, Marseille, France > http://komatitsch.free.fr > _______________________________________________ > CIG-SEISMO mailing list > CIG-SEISMO at geodynamics.org > http://lists.geodynamics.org/cgi-bin/mailman/listinfo/cig-seismo > -- Sevan Adourian Masters Student at Ecole Normale Superieure de Paris sevan.adourian at ens.fr -------------- next part -------------- An HTML attachment was scrubbed... URL: From komatitsch at lma.cnrs-mrs.fr Wed Apr 12 07:45:09 2017 From: komatitsch at lma.cnrs-mrs.fr (Dimitri Komatitsch) Date: Wed, 12 Apr 2017 16:45:09 +0200 Subject: [CIG-SEISMO] single force solution In-Reply-To: References: <7d7c9aae9ef741c1be7a87fd1dcc1c71@CSGHUB209W.pu.win.princeton.edu> <0be21ac50a984e85b12745abd04ba911@CSGHUB209W.pu.win.princeton.edu> <82087a05-fbab-7c64-6c37-fec864274887@lma.cnrs-mrs.fr> Message-ID: <6ae7ac33-cc73-2086-8b5a-2fe1fed8f5f3@lma.cnrs-mrs.fr> Hi Sevan, Hi all, This is very good news, it will be very interesting. I think Jessica Irving was also working on this with a student at Princeton but I am not sure how far they went (and I do not know if it was with SPECFEM; for sure they have not committed changes to SPECFEM). Are you based in Paris or in Berkeley? If you are in Paris we could meet at ENS on May 10 if you want to discuss that (it would be easier with a pen and paper + white board :-) , I will be in Paris for a meeting and would have time to stop by. Thanks, Best wishes, Dimitri. On 04/12/2017 03:26 PM, Sevan Adourian wrote: > Hi Dimitri, > > Thanks for your answer! Yes, I'm continuing Yder's work on box > tomography, and I'm trying to allow the stations to be outside of the > box (which is why I need to compute the Green's functions from an > "outside" receiver to each point of the mirror for a later convolution > with the wavefield). In my case I need to couple SPECFEM with a slightly > modified RegSEM. > > But reading what you describe, the problem might still come from me > because the issues I'm facing are on the SPECFEM side only. So if you > tell me it already worked, I might still do something wrong. > > Regards, > Sévan > > On Wed, Apr 12, 2017 at 3:10 PM, Dimitri Komatitsch > > wrote: > > > Hi Sevan, > > Yes, in earlier work > (http://komatitsch.free.fr/preprints/GJI2_Vadim_2015.pdf > and > http://komatitsch.free.fr/preprints/Geology_Chevrot_2016.pdf > ) we > have coupled SPECFEM3D with DSM and that worked fine, however more > recently we have tried to couple SPECFEM3D with AxiSEM to do FWI > locally with our wavefield injection technique (also similar to e.g. > Masson and Romanowicz's box tomography approach) and so far we get > some weird extra phases, possibly something we are doing wrong in > some rotations. Not sure if the problem is the same as yours, it > could also be a small mistake we made in the representation theorem > coupling integral or something like that. > > Since you are at Berkeley, is this related to what Yder was doing? > (I think he was using finite differences, not SPECFEM?). > > Thanks, > Best wishes, > > Dimitri. > > On 04/12/2017 03:01 PM, Sevan Adourian wrote: > > Hi Hom Nath and Dimitri, > > Alright then, I'll be waiting for the tests of your new version > then! > Dimitri, may I ask you which problems you have faced in the Green's > functions computation you mentioned earlier? Do you also have one > component that is satisfying reciprocity? I'm asking because I'm > trying > to couple specfem with another external code as well. > > Thanks and regards, > Sévan > > On Wed, Apr 12, 2017 at 1:51 PM, Hom Nath Gharti > > >> > wrote: > > Hi Sévan, > > New implementation is very similar, but the source time function > will have several options. I make it more general because some > people were interested on a moving source with sine source time > function, for exampme. > > Best, > Hom Nath > > > On Wed, Apr 12, 2017, 7:36 AM Sevan Adourian > > >> > wrote: > > Hi Hom Nath and Dimitri, > > Thank you Dimitri and Hom Nath for the feedback. Hom > Nath, will > your new implementation be on the model of what's done in > specfem Cartesian (i.e. with the FORCESOLUTION file), > and will > it support a source that is not on a node of an element > without > relocating this source? And will it still use a ricker as a > source time function? > > Thanks, > Sévan > > > On Wed, Apr 12, 2017 at 1:24 PM, Hom Nath Gharti > > >> > wrote: > > Hi Dimitri, > > I will commit! > > Best, > Hom Nath > > On Wed, Apr 12, 2017, 7:09 AM Dimitri Komatitsch > > >> wrote: > > > Hi Hom Nath, > > Perfect, thanks! > Then please remember to commit it to GitHub. > > Best regards, > Dimitri. > > On 04/12/2017 01:06 PM, Hom Nath Gharti wrote: > > Hi Sévan, > > > > I am about to finish new point force > implementation. I will let you know > > once it is ready for testing. > > > > Best, > > Hom Nath > > > > On Wed, Apr 12, 2017 at 6:36 AM, Dimitri > Komatitsch > > > > > > > >>> wrote: > > > > > > Hi Sevan, > > > > I guess you can keep cc'ing the list > because it is an issue Clément, > > Vadim, Paul, Sébastien and I have been > facing as well in our coupling > > with external codes, thus we are > interested in knowing more about the > > potential problem. > > > > Thanks, > > Best wishes, > > Dimitri. > > > > On 04/12/2017 11:24 AM, Sevan Adourian wrote: > > > Hi Hom Nath, > > > > > > Thanks for your answer and concern. > Since you suggest something that is > > > not directly "specfem related", can I > send you directly an email so that > > > we can discuss it? And then I can post > the solution once we'll find it, > > > in order to close the topic! > > > > > > Thanks again, > > > Sévan > > > > > > On Tue, Apr 11, 2017 at 8:46 PM, Hom > Nath Gharti > > > >> > > > > > > >>>> wrote: > > > > > > Hi Sévan, > > > > > > Do you also switch the orientations > of source and receiver? > > > > > > Thanks, > > > Hom Nath > > > > > > On Tue, Apr 11, 2017 at 1:39 PM, > Sevan Adourian > > > > > > > > >> > > > > > > > > >>>> > > > wrote: > > > > > > Hi cig-seismo community, > > > > > > I'm trying to compute Green's > functions > using Specfem3D-Globe, > > > using a single force point source > solution coupled with a > > source > > > time function that basically > represents > a filtered Dirac > > delta. > > > In order to test my > implementation, I > imposed a source at a > > > point A and looked at the obtained > Green's function at a > > > receiver B. Then, I switched the > position of the source > > and the > > > receiver and compared the 2 in > order to > check the > > reciprocity of > > > those Green's functions. > > > > > > It turned out that I have a perfect > reciprocity (in the > > > frequency band that correspond to my > filtered Dirac delta) > > only > > > on the component in which I > input the > force, while the other > > > components don't quite match. > > > > > > I strongly suspected a reference > frame > rotation issue, so I > > > tried to input the force in the > global > xyz coordinate system > > > (i.e. adding the force directly > in the > accel_crust_mantle > > array > > > in the comp_add_sources subroutine, > without multiplying > > with the > > > nu_source matrix) and looked at the > Green's functions in this > > > xyz coordinates system (ie, directly > writing in the > > seismograms > > > structure the values of ux, uy > and uz > computed in the > > > compute_seismogram subroutine), > but I > still have this issue. > > > Would anyone have an idea of > where this > probable rotation > > issue > > > could come from? > > > > > > I'm working on specfem3D-Globe > v6.0.0. > > > > > > Thanks in advance for your help! > > > Sévan > > > > > > -- > > > Sevan Adourian > > > Masters Student at Ecole Normale > Superieure de Paris > > > sevan.adourian at ens.fr > > > > > >> > > > > > > >>> > > > > > > > > > > > > > _______________________________________________ > > > CIG-SEISMO mailing list > > > CIG-SEISMO at geodynamics.org > > > > > >> > > > > > > >>> > > > > > > http://lists.geodynamics.org/cgi-bin/mailman/listinfo/cig-seismo > > > > > > > > > >> > > > > > > > > > > > > > > >>> > > > > > > > > > > > > > > > -- > > > Sevan Adourian > > > Masters Student at Ecole Normale > Superieure de Paris > > > sevan.adourian at ens.fr > > > > >> > > > > > > >>> > > > > > > > > > > _______________________________________________ > > > CIG-SEISMO mailing list > > > CIG-SEISMO at geodynamics.org > > > > > >> > > > > http://lists.geodynamics.org/cgi-bin/mailman/listinfo/cig-seismo > > > > > > > > > >> > > > > > > > -- > > Dimitri Komatitsch, CNRS Research Director > (DR CNRS) > > Laboratory of Mechanics and Acoustics, > Marseille, France > > http://komatitsch.free.fr > > > _______________________________________________ > > CIG-SEISMO mailing list > > CIG-SEISMO at geodynamics.org > > > > > >> > > > http://lists.geodynamics.org/cgi-bin/mailman/listinfo/cig-seismo > > > > > > > > >> > > > > > > > > > > _______________________________________________ > > CIG-SEISMO mailing list > > CIG-SEISMO at geodynamics.org > > > > > > http://lists.geodynamics.org/cgi-bin/mailman/listinfo/cig-seismo > > > > > > > > -- > Dimitri Komatitsch, CNRS Research Director (DR CNRS) > Laboratory of Mechanics and Acoustics, > Marseille, France > http://komatitsch.free.fr > _______________________________________________ > CIG-SEISMO mailing list > CIG-SEISMO at geodynamics.org > > > > > http://lists.geodynamics.org/cgi-bin/mailman/listinfo/cig-seismo > > > > > > > _______________________________________________ > CIG-SEISMO mailing list > CIG-SEISMO at geodynamics.org > > > > > http://lists.geodynamics.org/cgi-bin/mailman/listinfo/cig-seismo > > > > > > > > > -- > Sevan Adourian > Masters Student at Ecole Normale Superieure de Paris > sevan.adourian at ens.fr > > > > > _______________________________________________ > CIG-SEISMO mailing list > CIG-SEISMO at geodynamics.org > > > > > http://lists.geodynamics.org/cgi-bin/mailman/listinfo/cig-seismo > > > > > > > > > -- > Sevan Adourian > Masters Student at Ecole Normale Superieure de Paris > sevan.adourian at ens.fr > > > > > _______________________________________________ > CIG-SEISMO mailing list > CIG-SEISMO at geodynamics.org > http://lists.geodynamics.org/cgi-bin/mailman/listinfo/cig-seismo > > > > -- > Dimitri Komatitsch, CNRS Research Director (DR CNRS) > Laboratory of Mechanics and Acoustics, Marseille, France > http://komatitsch.free.fr > _______________________________________________ > CIG-SEISMO mailing list > CIG-SEISMO at geodynamics.org > http://lists.geodynamics.org/cgi-bin/mailman/listinfo/cig-seismo > > > > > > -- > Sevan Adourian > Masters Student at Ecole Normale Superieure de Paris > sevan.adourian at ens.fr > > > _______________________________________________ > CIG-SEISMO mailing list > CIG-SEISMO at geodynamics.org > http://lists.geodynamics.org/cgi-bin/mailman/listinfo/cig-seismo > -- Dimitri Komatitsch, CNRS Research Director (DR CNRS) Laboratory of Mechanics and Acoustics, Marseille, France http://komatitsch.free.fr From michael.gineste at ntnu.no Wed Apr 12 12:51:06 2017 From: michael.gineste at ntnu.no (Michael Gineste) Date: Wed, 12 Apr 2017 21:51:06 +0200 Subject: [CIG-SEISMO] Specfem2D wave field dump bug Message-ID: <33ca3cc7-576c-9fc0-9253-818da46d2ea5@ntnu.no> Hi Specfem2D team, First I would like to thank you for making and maintaining such an excellent piece of software. I encountered a bug when wanting to dump the entire wave field at some rate but with no need of the postscript plot output (switching this off as postscript export is rather slow). Running the code with output_postscript_snapshot = .false. and output_wavefield_dumps = .true. would result in a segmentation fault. The reason being that the variable this_is_the_first_we_dump was not initialised correctly leading to other variables not being allocated. I have attached my simpel workaround as a diff file, which might not be the optimal solution but it worked for my purpose. As a sidenote, the wavefield dump input NSTEP_BETWEEN_OUTPUT_WAVE_DUMPS is not used directly, so one must set NSTEP_BETWEEN_OUTPUT_IMAGES to the value one wants. In continuation of this I have question that I hoped you could help me with. I have used the function utils/recombine_all_slices_from_binary_dump.f90 as offset for making a slightly more flexible recombining routine. But I have found that the search for duplicates is a really slow process when the mesh is large (as is said in the printout message) and this becomes quite the bottleneck for my use. So I was wandering if this information of which coordinates interfacing partitions shares, were not already calculated in the preparation of a simulation? So that I could export this information at same time of dumping grid and load this later, saving me the (brute force) search for duplicates. I have spent some time looking through the Specfem2D code, but I have not managed to conclude whether this information on coordinate duplicates between partitions is available or not. Could you help me with confirming/refute that this information is being generated somewhere in the code, and if this information really is there, point me towards the area/variable names that I should look at? My code version is (git) commit dd9cc6b798da8b4ff7137eb8d96fd2a6bc4685eb, I cloned the stable repo fairly recently. Thank you. Best regards, Michael Gineste PhD Candidate Department of Mathematical Sciences Norwegian University of Science and Technology (NTNU) -------------- next part -------------- A non-text attachment was scrubbed... Name: prepare_timerun.diff Type: text/x-patch Size: 821 bytes Desc: not available URL: From sevan.adourian at berkeley.edu Wed Apr 12 23:55:32 2017 From: sevan.adourian at berkeley.edu (Sevan Adourian) Date: Thu, 13 Apr 2017 08:55:32 +0200 Subject: [CIG-SEISMO] single force solution In-Reply-To: <6ae7ac33-cc73-2086-8b5a-2fe1fed8f5f3@lma.cnrs-mrs.fr> References: <7d7c9aae9ef741c1be7a87fd1dcc1c71@CSGHUB209W.pu.win.princeton.edu> <0be21ac50a984e85b12745abd04ba911@CSGHUB209W.pu.win.princeton.edu> <82087a05-fbab-7c64-6c37-fec864274887@lma.cnrs-mrs.fr> <6ae7ac33-cc73-2086-8b5a-2fe1fed8f5f3@lma.cnrs-mrs.fr> Message-ID: Hi Dimitri, Yes I'm currently in Paris and I'd be happy to meet with you at ENS! I think I solved my problem though, and as thought in a first place, it's not really related to specfem.. Sorry to have brought the subject here.. Thanks a lot for the help, Sévan На 12 апр. 2017 г., 16:45, в 16:45, Dimitri Komatitsch написал:п> >Hi Sevan, Hi all, > >This is very good news, it will be very interesting. > >I think Jessica Irving was also working on this with a student at >Princeton but I am not sure how far they went (and I do not know if it >was with SPECFEM; for sure they have not committed changes to SPECFEM). > >Are you based in Paris or in Berkeley? If you are in Paris we could >meet >at ENS on May 10 if you want to discuss that (it would be easier with a > >pen and paper + white board :-) , I will be in Paris for a meeting and >would have time to stop by. > >Thanks, >Best wishes, > >Dimitri. > >On 04/12/2017 03:26 PM, Sevan Adourian wrote: >> Hi Dimitri, >> >> Thanks for your answer! Yes, I'm continuing Yder's work on box >> tomography, and I'm trying to allow the stations to be outside of the >> box (which is why I need to compute the Green's functions from an >> "outside" receiver to each point of the mirror for a later >convolution >> with the wavefield). In my case I need to couple SPECFEM with a >slightly >> modified RegSEM. >> >> But reading what you describe, the problem might still come from me >> because the issues I'm facing are on the SPECFEM side only. So if you >> tell me it already worked, I might still do something wrong. >> >> Regards, >> Sévan >> >> On Wed, Apr 12, 2017 at 3:10 PM, Dimitri Komatitsch >> > >wrote: >> >> >> Hi Sevan, >> >> Yes, in earlier work >> (http://komatitsch.free.fr/preprints/GJI2_Vadim_2015.pdf >> and >> http://komatitsch.free.fr/preprints/Geology_Chevrot_2016.pdf >> ) >we >> have coupled SPECFEM3D with DSM and that worked fine, however >more >> recently we have tried to couple SPECFEM3D with AxiSEM to do FWI >> locally with our wavefield injection technique (also similar to >e.g. >> Masson and Romanowicz's box tomography approach) and so far we >get >> some weird extra phases, possibly something we are doing wrong in >> some rotations. Not sure if the problem is the same as yours, it >> could also be a small mistake we made in the representation >theorem >> coupling integral or something like that. >> >> Since you are at Berkeley, is this related to what Yder was >doing? >> (I think he was using finite differences, not SPECFEM?). >> >> Thanks, >> Best wishes, >> >> Dimitri. >> >> On 04/12/2017 03:01 PM, Sevan Adourian wrote: >> >> Hi Hom Nath and Dimitri, >> >> Alright then, I'll be waiting for the tests of your new >version >> then! >> Dimitri, may I ask you which problems you have faced in the >Green's >> functions computation you mentioned earlier? Do you also have >one >> component that is satisfying reciprocity? I'm asking because >I'm >> trying >> to couple specfem with another external code as well. >> >> Thanks and regards, >> Sévan >> >> On Wed, Apr 12, 2017 at 1:51 PM, Hom Nath Gharti >> >> >> >> wrote: >> >> Hi Sévan, >> >> New implementation is very similar, but the source time >function >> will have several options. I make it more general because >some >> people were interested on a moving source with sine >source time >> function, for exampme. >> >> Best, >> Hom Nath >> >> >> On Wed, Apr 12, 2017, 7:36 AM Sevan Adourian >> > >> > >> >> wrote: >> >> Hi Hom Nath and Dimitri, >> >> Thank you Dimitri and Hom Nath for the feedback. Hom >> Nath, will >> your new implementation be on the model of what's >done in >> specfem Cartesian (i.e. with the FORCESOLUTION file), >> and will >> it support a source that is not on a node of an >element >> without >> relocating this source? And will it still use a >ricker as a >> source time function? >> >> Thanks, >> Sévan >> >> >> On Wed, Apr 12, 2017 at 1:24 PM, Hom Nath Gharti >> >> >> >> wrote: >> >> Hi Dimitri, >> >> I will commit! >> >> Best, >> Hom Nath >> >> On Wed, Apr 12, 2017, 7:09 AM Dimitri Komatitsch >> > >> > >> wrote: >> >> >> Hi Hom Nath, >> >> Perfect, thanks! >> Then please remember to commit it to GitHub. >> >> Best regards, >> Dimitri. >> >> On 04/12/2017 01:06 PM, Hom Nath Gharti >wrote: >> > Hi Sévan, >> > >> > I am about to finish new point force >> implementation. I will let you know >> > once it is ready for testing. >> > >> > Best, >> > Hom Nath >> > >> > On Wed, Apr 12, 2017 at 6:36 AM, Dimitri >> Komatitsch >> > > >> > > >> > >> >> > >>> wrote: >> > >> > >> > Hi Sevan, >> > >> > I guess you can keep cc'ing the list >> because it is an issue Clément, >> > Vadim, Paul, Sébastien and I have been >> facing as well in our coupling >> > with external codes, thus we are >> interested in knowing more about the >> > potential problem. >> > >> > Thanks, >> > Best wishes, >> > Dimitri. >> > >> > On 04/12/2017 11:24 AM, Sevan Adourian >wrote: >> > > Hi Hom Nath, >> > > >> > > Thanks for your answer and concern. >> Since you suggest something that is >> > > not directly "specfem related", can I >> send you directly an email so that >> > > we can discuss it? And then I can >post >> the solution once we'll find it, >> > > in order to close the topic! >> > > >> > > Thanks again, >> > > Sévan >> > > >> > > On Tue, Apr 11, 2017 at 8:46 PM, Hom >> Nath Gharti > > > >> > >> > >> >> > > > > > >> > >> > >>>> wrote: >> > > >> > > Hi Sévan, >> > > >> > > Do you also switch the >orientations >> of source and receiver? >> > > >> > > Thanks, >> > > Hom Nath >> > > >> > > On Tue, Apr 11, 2017 at 1:39 PM, >> Sevan Adourian >> > > > >> > > >> > > >> > >> >> > > >> > > >> >> > > >> > >>>> >> > > wrote: >> > > >> > > Hi cig-seismo community, >> > > >> > > I'm trying to compute Green's >> functions >> using Specfem3D-Globe, >> > > using a single force point >source >> solution coupled with a >> > source >> > > time function that basically >> represents >> a filtered Dirac >> > delta. >> > > In order to test my >> implementation, I >> imposed a source at a >> > > point A and looked at the >obtained >> Green's function at a >> > > receiver B. Then, I switched >the >> position of the source >> > and the >> > > receiver and compared the 2 >in >> order to >> check the >> > reciprocity of >> > > those Green's functions. >> > > >> > > It turned out that I have a >perfect >> reciprocity (in the >> > > frequency band that >correspond to my >> filtered Dirac delta) >> > only >> > > on the component in which I >> input the >> force, while the other >> > > components don't quite match. >> > > >> > > I strongly suspected a >reference >> frame >> rotation issue, so I >> > > tried to input the force in >the >> global >> xyz coordinate system >> > > (i.e. adding the force >directly >> in the >> accel_crust_mantle >> > array >> > > in the comp_add_sources >subroutine, >> without multiplying >> > with the >> > > nu_source matrix) and looked >at the >> Green's functions in this >> > > xyz coordinates system (ie, >directly >> writing in the >> > seismograms >> > > structure the values of ux, >uy >> and uz >> computed in the >> > > compute_seismogram >subroutine), >> but I >> still have this issue. >> > > Would anyone have an idea of >> where this >> probable rotation >> > issue >> > > could come from? >> > > >> > > I'm working on >specfem3D-Globe >> v6.0.0. >> > > >> > > Thanks in advance for your >help! >> > > Sévan >> > > >> > > -- >> > > Sevan Adourian >> > > Masters Student at Ecole >Normale >> Superieure de Paris >> > > sevan.adourian at ens.fr >> >> > > >> > >> > >> >> > > >> > > >> > >> > >>> >> > > >> > > >> > > >> > > >> _______________________________________________ >> > > CIG-SEISMO mailing list >> > > CIG-SEISMO at geodynamics.org >> >> > > >> > >> > >> >> > > >> > > >> > >> > >>> >> > > >> > >> >http://lists.geodynamics.org/cgi-bin/mailman/listinfo/cig-seismo >> > >> >> >> >> > >> >> > >> >> >>> >> > > >> > >> > >> >> >> >> > >> >> > >> >> >>>> >> > > >> > > >> > > >> > > >> > > -- >> > > Sevan Adourian >> > > Masters Student at Ecole Normale >> Superieure de Paris >> > > sevan.adourian at ens.fr >> > > >> > >> > >> >> > > >> > > >> > >> > >>> >> > > >> > > >> > > >> _______________________________________________ >> > > CIG-SEISMO mailing list >> > > CIG-SEISMO at geodynamics.org >> >> > > >> > >> > >> >> > > >> >http://lists.geodynamics.org/cgi-bin/mailman/listinfo/cig-seismo >> > >> >> >> >> > >> >> > >> >> >>> >> > > >> > >> > -- >> > Dimitri Komatitsch, CNRS Research >Director >> (DR CNRS) >> > Laboratory of Mechanics and Acoustics, >> Marseille, France >> > http://komatitsch.free.fr >> > >> _______________________________________________ >> > CIG-SEISMO mailing list >> > CIG-SEISMO at geodynamics.org >> >> > > >> > >> > >> >> > >> >http://lists.geodynamics.org/cgi-bin/mailman/listinfo/cig-seismo > >> >> >> >> > >> >> > >> >> >>> >> > >> > >> > >> > >> > >_______________________________________________ >> > CIG-SEISMO mailing list >> > CIG-SEISMO at geodynamics.org >> >> > > >> > >> >http://lists.geodynamics.org/cgi-bin/mailman/listinfo/cig-seismo >> > >> >> >> >> > >> >> -- >> Dimitri Komatitsch, CNRS Research Director >(DR CNRS) >> Laboratory of Mechanics and Acoustics, >> Marseille, France >> http://komatitsch.free.fr >> >_______________________________________________ >> CIG-SEISMO mailing list >> CIG-SEISMO at geodynamics.org >> >> > > >> >> >http://lists.geodynamics.org/cgi-bin/mailman/listinfo/cig-seismo >> > >> >> >> >> >> >> _______________________________________________ >> CIG-SEISMO mailing list >> CIG-SEISMO at geodynamics.org >> >> > > >> >> >http://lists.geodynamics.org/cgi-bin/mailman/listinfo/cig-seismo >> > >> >> >> >> >> >> >> >> -- >> Sevan Adourian >> Masters Student at Ecole Normale Superieure de Paris >> sevan.adourian at ens.fr >> > >> >> >> _______________________________________________ >> CIG-SEISMO mailing list >> CIG-SEISMO at geodynamics.org >> >> > > >> >> >http://lists.geodynamics.org/cgi-bin/mailman/listinfo/cig-seismo >> > >> >> >> >> >> >> >> >> -- >> Sevan Adourian >> Masters Student at Ecole Normale Superieure de Paris >> sevan.adourian at ens.fr >> > >> >> >> _______________________________________________ >> CIG-SEISMO mailing list >> CIG-SEISMO at geodynamics.org > >> >http://lists.geodynamics.org/cgi-bin/mailman/listinfo/cig-seismo >> > >> >> >> -- >> Dimitri Komatitsch, CNRS Research Director (DR CNRS) >> Laboratory of Mechanics and Acoustics, Marseille, France >> http://komatitsch.free.fr >> _______________________________________________ >> CIG-SEISMO mailing list >> CIG-SEISMO at geodynamics.org >> http://lists.geodynamics.org/cgi-bin/mailman/listinfo/cig-seismo >> > >> >> >> >> >> -- >> Sevan Adourian >> Masters Student at Ecole Normale Superieure de Paris >> sevan.adourian at ens.fr >> >> >> _______________________________________________ >> CIG-SEISMO mailing list >> CIG-SEISMO at geodynamics.org >> http://lists.geodynamics.org/cgi-bin/mailman/listinfo/cig-seismo >> > >-- >Dimitri Komatitsch, CNRS Research Director (DR CNRS) >Laboratory of Mechanics and Acoustics, Marseille, France >http://komatitsch.free.fr >_______________________________________________ >CIG-SEISMO mailing list >CIG-SEISMO at geodynamics.org >http://lists.geodynamics.org/cgi-bin/mailman/listinfo/cig-seismo -------------- next part -------------- An HTML attachment was scrubbed... URL: From komatitsch at lma.cnrs-mrs.fr Thu Apr 20 03:39:51 2017 From: komatitsch at lma.cnrs-mrs.fr (Dimitri Komatitsch) Date: Thu, 20 Apr 2017 12:39:51 +0200 Subject: [CIG-SEISMO] Specfem2D wave field dump bug In-Reply-To: <33ca3cc7-576c-9fc0-9253-818da46d2ea5@ntnu.no> References: <33ca3cc7-576c-9fc0-9253-818da46d2ea5@ntnu.no> Message-ID: <58c71443-e446-f096-5df7-fc86c1e26191@lma.cnrs-mrs.fr> Hi Michael, Thank you very much for your bug report. I have fixed the official version of the code, and I also suppressed NSTEP_BETWEEN_OUTPUT_WAVE_DUMPS, which as you say was unused apparently, I replaced it with NSTEP_BETWEEN_OUTPUT_IMAGES. (I did it in the "devel" branch only, as we always do) Regarding recombining the information into a single dump for all MPI partitions, yes, the information should exist (likely in src/specfem2D/assemble_MPI.F90 , look for subroutine assemble_MPI_scalar() for instance, in which we use the MPI partition edge information to assemble the mass matrix (or any other scalar). Let me cc Etienne Bachmann because he has been modifying the 2D code recently to simplify it and thus he may have other/better suggestions on how to do that. The best thing would probably be to completely get rid of utils/recombine_all_slices_from_binary_dump.f90 and recombine all the slices inside the code instead, we do that for instance in src/specfem2D/plot_post.F90 to create the PostScript snapshots. There is also this comment in src/specfem2D/write_movie_output.f90 : ! dump the full (local) wavefield to a file ! note: in the case of MPI, in the future it would be more convenient to output a single file ! rather than one for each myrank if (output_wavefield_dumps) then If you do it please let us know and we will be glad to commit the change to the official source code (or just submit a Git pull request, see e.g. https://github.com/geodynamics/specfem3d/wiki/Using-Git-for-SPECFEM ) Thanks again! Best wishes, Dimitri. On 04/12/2017 09:51 PM, Michael Gineste wrote: > Hi Specfem2D team, > > First I would like to thank you for making and maintaining such an > excellent piece of software. > > I encountered a bug when wanting to dump the entire wave field at some > rate but with no need of the postscript plot output (switching this off > as postscript export is rather slow). Running the code with > output_postscript_snapshot = .false. and output_wavefield_dumps = .true. > would result in a segmentation fault. The reason being that the variable > this_is_the_first_we_dump was not initialised correctly leading to other > variables not being allocated. > I have attached my simpel workaround as a diff file, which might not be > the optimal solution but it worked for my purpose. As a sidenote, the > wavefield dump input NSTEP_BETWEEN_OUTPUT_WAVE_DUMPS is not used > directly, so one must set NSTEP_BETWEEN_OUTPUT_IMAGES to the value one > wants. > > In continuation of this I have question that I hoped you could help me > with. > I have used the function utils/recombine_all_slices_from_binary_dump.f90 > as offset for making a slightly more flexible recombining routine. But I > have found that the search for duplicates is a really slow process when > the mesh is large (as is said in the printout message) and this becomes > quite the bottleneck for my use. > > So I was wandering if this information of which coordinates interfacing > partitions shares, were not already calculated in the preparation of a > simulation? So that I could export this information at same time of > dumping grid and load this later, saving me the (brute force) search for > duplicates. > > I have spent some time looking through the Specfem2D code, but I have > not managed to conclude whether this information on coordinate > duplicates between partitions is available or not. Could you help me > with confirming/refute that this information is being generated > somewhere in the code, and if this information really is there, point me > towards the area/variable names that I should look at? > > My code version is (git) commit > dd9cc6b798da8b4ff7137eb8d96fd2a6bc4685eb, I cloned the stable repo > fairly recently. > > Thank you. > > Best regards, > > Michael Gineste > > PhD Candidate > Department of Mathematical Sciences > Norwegian University of Science and Technology (NTNU) > > > > > _______________________________________________ > CIG-SEISMO mailing list > CIG-SEISMO at geodynamics.org > http://lists.geodynamics.org/cgi-bin/mailman/listinfo/cig-seismo > -- Dimitri Komatitsch, CNRS Research Director (DR CNRS) Laboratory of Mechanics and Acoustics, Marseille, France http://komatitsch.free.fr From phil.cummins at anu.edu.au Sat Apr 22 18:45:25 2017 From: phil.cummins at anu.edu.au (Phil Cummins) Date: Sun, 23 Apr 2017 11:45:25 +1000 Subject: [CIG-SEISMO] specfem3d_globe with CUDA Message-ID: <58FC0735.3030308@anu.edu.au> Hello, I'm would like to use specfem3d_globe with my university's GPU cluster. I am trying to compile specfem3d_globe-7.0.0 after having configured it using: ./configure --with-cuda=cuda6 I am using the ifort fortran compiler version 12.1.9.293, and the version of cuda I have loaded is 6.5. The "make all" command does a lot of compilation using ifort and nvcc, but eventually fails when trying to link xspecfem3D itself with the error: ifort: error #10236: File not found: './obj/cuda_device_obj.o' make: *** [bin/xspecfem3D] Error 1 Delving into what happened during the make (see below), I noticed that after using nvcc to compile crust_mantle_impl_kernel_adjoint.cu, it attempts to use the command DUSE_OLDER_CUDA4_GPU to compile cuda_device_obj, which of course doesn't work because DUSE_OLDER_CUDA4_GPU is a compiler option not a command. It seems that make has somehow lost the beginning of this particular compilation command. Has anyone else seen or know what to do about it? Thanks, - Phil P.S. Some of the hardware on our cluster can be used only with cuda version 8. Does anyone know if specfem3d will work with cuda8? nvcc -c src/gpu/kernels.gen/crust_mantle_impl_kernel_adjoint.cu -o obj/crust_ma\ ntle_impl_kernel_adjoint.cuda-kernel.o --cudart=shared -I/apps/openmpi/1.7.5/\ include -DUSE_OLDER_CUDA4_GPU -gencode=arch=compute_20,code=\"sm_20,compute_20\\ " -x cu -I./setup -I./src/gpu/kernels.gen -DUSE_CUDA -include src/gpu/mesh_con\ stants_gpu.h DUSE_OLDER_CUDA4_GPU -gencode=arch=compute_20,code=\"sm_20,compute_20\" -o obj/\ cuda_device_obj.o obj/assemble_MPI_scalar_gpu.cuda.o obj/assemble_MPI_vector_gp\ u.cuda.o obj/check_fields_gpu.cuda.o obj/compute_add_sources_elastic_gpu.cuda.o\ obj/compute_coupling_gpu.cuda.o obj/compute_forces_crust_mantle_gpu.cuda.o obj\ /compute_forces_inner_core_gpu.cuda.o obj/compute_forces_outer_core_gpu.cuda.o \ obj/compute_kernels_gpu.cuda.o obj/compute_stacey_acoustic_gpu.cuda.o obj/compu\ te_stacey_elastic_gpu.cuda.o obj/compute_strain_gpu.cuda.o obj/helper_functions\ _gpu.cuda.o obj/initialize_gpu.cuda.o obj/noise_tomography_gpu.cuda.o obj/prepa\ re_mesh_constants_gpu.cuda.o obj/transfer_fields_gpu.cuda.o obj/update_displace\ ment_gpu.cuda.o obj/write_seismograms_gpu.cuda.o obj/save_and_compare_cpu_vs_gp\ u.cuda.o obj/assemble_boundary_accel_on_device.cuda-kernel.o obj/assemble_bound\ ary_potential_on_device.cuda-kernel.o obj/prepare_boundary_potential_on_device.\ cuda-kernel.o obj/prepare_boundary_accel_on_device.cuda-kernel.o obj/get_maximu\ m_scalar_kernel.cuda-kernel.o obj/get_maximum_vector_kernel.cuda-kernel.o obj/c\ ompute_add_sources_adjoint_kernel.cuda-kernel.o obj/compute_add_sources_kernel.\ cuda-kernel.o nel.cuda-kernel.o obj/write_seismograms_transfer_strain_from_device_kernel.cuda\ -kernel.o obj/noise_transfer_surface_to_host_kernel.cuda-kernel.o obj/noise_add\ _source_master_rec_kernel.cuda-kernel.o obj/noise_add_surface_movie_kernel.cuda\ -kernel.o obj/compute_stacey_acoustic_kernel.cuda-kernel.o obj/compute_stacey_a\ coustic_backward_kernel.cuda-kernel.o obj/compute_stacey_elastic_kernel.cuda-ke\ rnel.o obj/compute_stacey_elastic_backward_kernel.cuda-kernel.o obj/update_disp\ _veloc_kernel.cuda-kernel.o obj/update_potential_kernel.cuda-kernel.o obj/updat\ e_accel_elastic_kernel.cuda-kernel.o obj/update_veloc_elastic_kernel.cuda-kerne\ l.o obj/update_accel_acoustic_kernel.cuda-kernel.o obj/update_veloc_acoustic_ke\ rnel.cuda-kernel.o obj/compute_rho_kernel.cuda-kernel.o obj/compute_iso_kernel.\ cuda-kernel.o obj/compute_ani_kernel.cuda-kernel.o obj/compute_hess_kernel.cuda\ -kernel.o obj/compute_acoustic_kernel.cuda-kernel.o obj/compute_strength_noise_\ kernel.cuda-kernel.o obj/compute_ani_undoatt_kernel.cuda-kernel.o obj/compute_i\ so_undoatt_kernel.cuda-kernel.o obj/compute_strain_kernel.cuda-kernel.o obj/out\ er_core_impl_kernel_forward.cuda-kernel.o obj/outer_core_impl_kernel_adjoint.cu\ da-kernel.o obj/inner_core_impl_kernel_forward.cuda-kernel.o obj/inner_core_imp\ l_kernel_adjoint.cuda-kernel.o obj/crust_mantle_impl_kernel_forward.cuda-kernel\ .o obj/crust_mantle_impl_kernel_adjoint.cuda-kernel.o make: DUSE_OLDER_CUDA4_GPU: Command not found make: [obj/cuda_device_obj.o] Error 127 (ignored) -- Phil Cummins Prof. Natural Hazards Research School of Earth Sciences Australian National University -------------- next part -------------- An HTML attachment was scrubbed... URL: From phil.cummins at anu.edu.au Tue Apr 25 18:05:16 2017 From: phil.cummins at anu.edu.au (Phil Cummins) Date: Wed, 26 Apr 2017 11:05:16 +1000 Subject: [CIG-SEISMO] specfem3d_globe with CUDA In-Reply-To: <58FC0735.3030308@anu.edu.au> References: <58FC0735.3030308@anu.edu.au> Message-ID: <58FFF24C.3050602@anu.edu.au> Hello again, Still trying to compile specfem3d_globe using CUDA. I have obtaiend advice from our HPC center regarding the appropriate modules to use, which are: intel-fc/16.0.3.210 intel-cc/16.0.3.210 openmpi/1.10.2 cuda/8.0 I hope they're going to be OK. But I just can't make sense of what the Makefiule is trying to do with cuda_device_obj. As an exampe, let's consider what's happening with the CUDA module check_fields_gpu. First it is compiled into object code: nvcc -c src/gpu/check_fields_gpu.c -o obj/check_fields_gpu.cuda.o --cudart=shared -I/apps/openmpi/1.10.2/include -DUSE_OLDER_CUDA4_GPU -gencode=arch=compute_20,code=\"sm_20,compute_20\" -x cu -I./setup -I./src/gpu/kernels.gen -DUSE_CUDA That's fine (it does produce a deprecation warning). Then, for some reason that I can't figure out, the makefile is trying to link this in some way to the module cuda_device_obj, for which no source code seems to exist and for which the command appears to have omitted the nvcc compiler (so of course it doesn't work): DUSE_OLDER_CUDA4_GPU -gencode=arch=compute_20,code=\"sm_20,compute_20\" -o obj/cuda_device_obj.o obj/assemble_MPI_scalar_gpu.cuda.o obj/assemble_MPI_vector_gpu.cuda.o obj/check_fields_gpu.cuda.o obj/compute_add_sources_elastic_gpu.cuda.o obj/compute_coupling_gpu.cuda.o obj/compute_forces_crust_mantle_gpu.cuda.o obj/compute_forces_inner_core_gpu.cuda.o obj/compute_forces_outer_core_gpu.cuda.o obj/compute_kernels_gpu.cuda.o obj/compute_stacey_acoustic_gpu.cuda.o obj/compute_stacey_elastic_gpu.cuda.o obj/compute_strain_gpu.cuda.o obj/helper_functions_gpu.cuda.o obj/initialize_gpu.cuda.o obj/noise_tomography_gpu.cuda.o obj/prepare_mesh_constants_gpu.cuda.o obj/transfer_fields_gpu.cuda.o obj/update_displacement_gpu.cuda.o obj/write_seismograms_gpu.cuda.o obj/save_and_compare_cpu_vs_gpu.cuda.o obj/assemble_boundary_accel_on_device.cuda-kernel.o obj/assemble_boundary_potential_on_device.cuda-kernel.o obj/prepare_boundary_potential_on_device.cuda-kernel.o obj/prepare_boundary_accel_on_device.cuda-kernel.o obj/get_maximum_scalar_kernel.cuda-kernel.o obj/get_maximum_vector_kernel.cuda-kernel.o obj/compute_add_sources_adjoint_kernel.cuda-kernel.o obj/compute_add_sources_kernel.cuda-kernel.o Then, the commands issued by the Makefile try to create the main program xpsecfem3d by linking not only with cuda_device_obj.o, but also with check_fields_gpu.cuda.o. So even if I could get the above command to work somehow, the mpif90 command would almost certainly fail because of multiple definitions. In any case, I am struggling a bit because I just don't understand how or why cuda_device_obj.o is meant to be used. Can anyone help? mpif90 -g -xHost -fpe0 -ftz -assume buffered_io -assume byterecl -align sequence -vec-report0 -std03 -diag-disable 6477 -implicitnone -gen-interfaces -warn all -O3 -check nobounds -DFORCE_VECTORIZATION -o ./bin/xspecfem3D ./obj/assemble_MPI_scalar.solver.o ./obj/assemble_MPI_vector.solver.o ./obj/comp_source_spectrum.solver.o ./obj/compute_adj_source_frechet.solver.o ./obj/convert_time.solver.o ./obj/define_derivation_matrices.solver.o ./obj/file_io_threads.cc.o ./obj/force_ftz.cc.o ./obj/get_backazimuth.solver.o ./obj/get_cmt.solver.o ./obj/get_event_info.solver.o ./obj/make_gravity.solver.o ./obj/netlib_specfun_erf.solver.o ./obj/asdf_data.solverstatic_module.o ./obj/comp_source_time_function.solverstatic.o ./obj/specfem3D_par.solverstatic_module.o ./obj/write_seismograms.solverstatic.o ./obj/check_stability.solverstatic.o ./obj/compute_add_sources.solverstatic.o ./obj/compute_arrays_source.solverstatic.o ./obj/compute_boundary_kernel.solverstatic.o ./obj/compute_coupling.solverstatic.o ./obj/compute_element.solverstatic.o ./obj/compute_element_att_memory.solverstatic.o ./obj/compute_element_strain.solverstatic.o ./obj/compute_forces_acoustic_calling_routine.solverstatic.o ./obj/compute_forces_viscoelastic_calling_routine.solverstatic.o ./obj/compute_forces_crust_mantle_noDev.solverstatic.o ./obj/compute_forces_crust_mantle_Dev.solverstatic.o ./obj/compute_forces_inner_core_noDev.solverstatic.o ./obj/compute_forces_inner_core_Dev.solverstatic.o ./obj/compute_forces_outer_core_noDev.solverstatic.o ./obj/compute_forces_outer_core_Dev.solverstatic.o ./obj/compute_kernels.solverstatic.o ./obj/compute_seismograms.solverstatic.o ./obj/compute_stacey_crust_mantle.solverstatic.o ./obj/compute_stacey_outer_core.solverstatic.o ./obj/finali\ ze_simulation.solverstatic.o ./obj/get_attenuation.solverstatic.o ./obj/initialize_simulation.solverstatic.o ./obj/iterate_time.solverstatic.o ./obj/iterate_time_undoatt.solverstatic.o ./obj/locate_receivers.solverstatic.o ./obj/locate_regular_points.s\ olverstatic.o ./obj/locate_sources.solverstatic.o ./obj/multiply_arrays_source.solverstatic.o ./obj/noise_tomography.solverstatic.o ./obj/prepare_timerun.solverstatic.o ./obj/read_adjoint_sources.solverstatic.o ./obj/read_arrays_solver.solverstatic.o ./obj/read_forward_arrays.solverstatic.o ./obj/read_mesh_databases.solverstatic.o ./obj/read_topography_bathymetry.solverstatic.o ./obj/save_forward_arrays.solverstatic.o ./obj/save_kernels.solverstatic.o ./obj/save_regular_kernels.solverstatic.o ./obj/setup_GLL_points.solverstatic.o ./obj/setup_sources_receivers.solverstatic.o ./obj/specfem3D.solverstatic.o ./obj/update_displ\ acement_LDDRK.solverstatic.o ./obj/update_displacement_Newmark.solverstatic.o ./obj/write_movie_output.solverstatic.o ./obj/write_movie_volume.solverstatic.o ./obj/write_movie_surface.solverstatic.o ./obj/write_output_ASCII.solverstatic.o ./obj/write_output_SAC.solverstatic.o ./obj/assemble_MPI_scalar_gpu.cuda.o ./obj/assemble_MPI_vector_gpu.cuda.o ./obj/check_fields_gpu.cuda.o ./obj/compute_add_sources_elastic_gpu.cuda.o ./obj/compute_coupling_gpu.cuda.o ./obj/compute_forces_crust_mantle_gpu.cuda.o ./obj/compute_forces_inner_core_gpu.cuda.o ./obj/compute_forces_outer_core_gpu.cuda.o ./obj/compute_kernels_gpu.cuda.o ./obj/compute_stacey_acoustic_gpu.cuda.o ./obj/compute_stacey_elastic_gpu.cuda.o ./obj/compute_strain_gpu.cuda.o ./obj/helper_functions_gpu.cuda.o ./obj/initialize_gpu.cuda.o ./obj/noise_tomography_gpu.cuda.o ./obj/prepare_mesh_constants_gpu.cuda.o ./obj/transfer_fields_gpu.cuda.o ./obj/update_displacement_gpu.cuda.o ./obj/write_seismograms_gpu.cuda.o ./obj/save_and_compare_cpu_vs_gpu.cuda.o ./obj/cuda_device_obj.o ./obj/assemble_boundary_accel_on_device.cuda-kernel.o ./obj/assemble_boundary_potential_on_device.cuda-kernel.o ./obj/prepare_boundary_potential_on_device.cuda-kernel.o ./obj/prepare_boundary_accel_on_device.cuda-kernel.o ./obj/get_maximum_scalar_kernel.cuda-kernel.o ./obj/get_maximum_vector_kernel.cuda-kernel.o ./obj/compute_add_sources_adjoint_kernel.cuda-kernel.o ./obj/compute_add_sources_kernel.cuda-kernel.o ./obj/compute_coupling_fluid_CMB_kernel.cuda-kernel.o ./obj/compute_coupling_fluid_ICB_kernel.cuda-kernel.o ./obj/compute_coupling_CMB_fluid_kernel.cuda-kernel.o ./obj/compute_coupling_ICB_fluid_kernel.cuda-kernel.o ./obj/compute_coupling_ocean_kernel.cuda-kernel.o ./obj/write_seismograms_transfer_from_device_kernel.cuda-kernel.o ./obj/write_seismograms_transfer_strain_from_device_kernel.cuda-kernel.o ./obj/noise_transfer_surface_to_host_kernel.cuda-kernel.o ./obj/noise_add_source_master_rec_kernel.cuda-kernel.o ./obj/noise_add_surface_movie_kernel.cuda-kernel.o ./obj/compute_stacey_acoustic_kernel.cuda-kernel.o ./obj/compute_stacey_acoustic_backward_kernel.cuda-kernel.o \ ./obj/compute_stacey_elastic_kernel.cuda-kernel.o ./obj/compute_stacey_elastic_backward_kernel.cuda-kernel.o ./obj/update_disp_veloc_kernel.cuda-kernel.o ./obj/update_potential_kernel.cuda-kernel.o ./obj/update_accel_elastic_kernel.cuda-kernel.o ./obj/update_veloc_elastic_kernel.cuda-kernel.o ./obj/update_accel_acoustic_kernel.cuda-kernel.o ./obj/update_veloc_acoustic_kernel.cuda-kernel.o ./obj/compute_rho_kernel.cuda-kernel.o ./obj/compute_iso_kernel.cuda-kernel.o ./obj/compute_ani_kernel.cuda-kernel.o ./obj/compute_hess_kernel.cuda-kernel.o ./obj/compute_acoustic_kernel.cuda-kernel.o ./obj/compute_strength_noise_kernel.cuda-kernel.o ./obj/compute_ani_undoatt_kernel.cuda-kernel.o ./obj/compute_iso_undoatt_kernel.cuda-kernel.o ./obj/compute_strain_kernel.cuda-kernel.o ./obj/outer_core_impl_kernel_forward.cuda-kernel.o ./obj/outer_core_impl_kernel_adjoint.cudakernel.o ./obj/inner_core_impl_kernel_forward.cuda-kernel.o ./obj/inner_core_impl_kernel_adjoint.cuda-kernel.o ./obj/crust_mantle_impl_kernel_forward.cuda-kernel.o ./obj/crust_mantle_impl_kernel_adjoint.cuda-kernel.o ./obj/visual_vtk_stubs.visualc.o ./obj/shared_par.shared_module.o ./obj/auto_ner.shared.o ./obj/binary_c_io.cc.o ./obj/broadcast_computed_parameters.shared.o ./obj/calendar.shared.o ./obj/count_elements.shared.o ./obj/count_number_of_sources.shared.o ./obj/count_points.shared.o ./obj/create_name_database.shared.o ./obj/define_all_layers.shared.o ./obj/exit_mpi.shared.o ./obj/flush_system.shared.o ./obj/get_model_parameters.shared.o ./obj/get_timestep_and_layers.shared.o ./obj/gll_library.shared.o ./obj/hex_nodes.shared.o ./obj/intgrl.shared.o ./obj/lagrange_poly.shared.o ./obj/make_ellipticity.shared.o ./obj/model_prem.shared.o ./obj/model_topo_bathy.shared.o ./obj/parallel.sharedmpi.o ./obj/param_reader.cc.o ./obj/read_compute_parameters.shared.o ./obj/read_parameter_file.shared.o ./obj/read_value_parameters.shared.o ./obj/recompute_jacobian.shared.o ./obj/reduce.shared.o ./obj/rthetaphi_xyz.shared.o ./obj/spline_routines.shared.o ./obj/write_VTK_file.shared.o ./obj/adios_method_stubs.cc.o -lcudart Thanks, - Phil P.S. when i simply remove cuda_dev_obj.o from the above mpif90 command, the link fails with: ./obj/assemble_MPI_scalar_gpu.cuda.o:(.eh_frame+0x12): undefined reference to `__gxx_personality_v0' ./obj/assemble_MPI_vector_gpu.cuda.o:(.eh_frame+0x12): undefined reference to `__gxx_personality_v0' ./obj/check_fields_gpu.cuda.o:(.eh_frame+0x12): undefined reference to `__gxx_personality_v0' ./obj/compute_add_sources_elastic_gpu.cuda.o:(.eh_frame+0x12): undefined reference to `__gxx_personality_v0' ./obj/compute_coupling_gpu.cuda.o:(.eh_frame+0x12): undefined reference to `__gxx_personality_v0' ./obj/compute_forces_crust_mantle_gpu.cuda.o:(.eh_frame+0x12): more undefined references to `__gxx_personality_v0' follow Phil Cummins wrote: > Hello, > > I'm would like to use specfem3d_globe with my university's GPU > cluster. I am trying to compile specfem3d_globe-7.0.0 after having > configured it using: > ./configure --with-cuda=cuda6 > I am using the ifort fortran compiler version 12.1.9.293, and the > version of cuda I have loaded is 6.5. > The "make all" command does a lot of compilation using ifort and nvcc, > but eventually fails when trying to link xspecfem3D itself with the error: > > ifort: error #10236: File not found: './obj/cuda_device_obj.o' > make: *** [bin/xspecfem3D] Error 1 > > Delving into what happened during the make (see below), I noticed that > after using nvcc to compile crust_mantle_impl_kernel_adjoint.cu, it > attempts to use the command DUSE_OLDER_CUDA4_GPU to compile > cuda_device_obj, which of course doesn't work because > DUSE_OLDER_CUDA4_GPU is a compiler option not a command. It seems that > make has somehow lost the beginning of this particular compilation > command. > Has anyone else seen or know what to do about it? > Thanks, > > - Phil > > P.S. Some of the hardware on our cluster can be used only with cuda > version 8. Does anyone know if specfem3d will work with cuda8? > > nvcc -c src/gpu/kernels.gen/crust_mantle_impl_kernel_adjoint.cu -o > obj/crust_ma\ > ntle_impl_kernel_adjoint.cuda-kernel.o --cudart=shared > -I/apps/openmpi/1.7.5/\ > include -DUSE_OLDER_CUDA4_GPU > -gencode=arch=compute_20,code=\"sm_20,compute_20\\ > " -x cu -I./setup -I./src/gpu/kernels.gen -DUSE_CUDA -include > src/gpu/mesh_con\ > stants_gpu.h > DUSE_OLDER_CUDA4_GPU > -gencode=arch=compute_20,code=\"sm_20,compute_20\" -o obj/\ > cuda_device_obj.o obj/assemble_MPI_scalar_gpu.cuda.o > obj/assemble_MPI_vector_gp\ > u.cuda.o obj/check_fields_gpu.cuda.o > obj/compute_add_sources_elastic_gpu.cuda.o\ > obj/compute_coupling_gpu.cuda.o > obj/compute_forces_crust_mantle_gpu.cuda.o obj\ > /compute_forces_inner_core_gpu.cuda.o > obj/compute_forces_outer_core_gpu.cuda.o \ > obj/compute_kernels_gpu.cuda.o obj/compute_stacey_acoustic_gpu.cuda.o > obj/compu\ > te_stacey_elastic_gpu.cuda.o obj/compute_strain_gpu.cuda.o > obj/helper_functions\ > _gpu.cuda.o obj/initialize_gpu.cuda.o obj/noise_tomography_gpu.cuda.o > obj/prepa\ > re_mesh_constants_gpu.cuda.o obj/transfer_fields_gpu.cuda.o > obj/update_displace\ > ment_gpu.cuda.o obj/write_seismograms_gpu.cuda.o > obj/save_and_compare_cpu_vs_gp\ > u.cuda.o obj/assemble_boundary_accel_on_device.cuda-kernel.o > obj/assemble_bound\ > ary_potential_on_device.cuda-kernel.o > obj/prepare_boundary_potential_on_device.\ > cuda-kernel.o obj/prepare_boundary_accel_on_device.cuda-kernel.o > obj/get_maximu\ > m_scalar_kernel.cuda-kernel.o > obj/get_maximum_vector_kernel.cuda-kernel.o obj/c\ > ompute_add_sources_adjoint_kernel.cuda-kernel.o > obj/compute_add_sources_kernel.\ > cuda-kernel.o > nel.cuda-kernel.o > obj/write_seismograms_transfer_strain_from_device_kernel.cuda\ > -kernel.o obj/noise_transfer_surface_to_host_kernel.cuda-kernel.o > obj/noise_add\ > _source_master_rec_kernel.cuda-kernel.o > obj/noise_add_surface_movie_kernel.cuda\ > -kernel.o obj/compute_stacey_acoustic_kernel.cuda-kernel.o > obj/compute_stacey_a\ > coustic_backward_kernel.cuda-kernel.o > obj/compute_stacey_elastic_kernel.cuda-ke\ > rnel.o obj/compute_stacey_elastic_backward_kernel.cuda-kernel.o > obj/update_disp\ > _veloc_kernel.cuda-kernel.o obj/update_potential_kernel.cuda-kernel.o > obj/updat\ > e_accel_elastic_kernel.cuda-kernel.o > obj/update_veloc_elastic_kernel.cuda-kerne\ > l.o obj/update_accel_acoustic_kernel.cuda-kernel.o > obj/update_veloc_acoustic_ke\ > rnel.cuda-kernel.o obj/compute_rho_kernel.cuda-kernel.o > obj/compute_iso_kernel.\ > cuda-kernel.o obj/compute_ani_kernel.cuda-kernel.o > obj/compute_hess_kernel.cuda\ > -kernel.o obj/compute_acoustic_kernel.cuda-kernel.o > obj/compute_strength_noise_\ > kernel.cuda-kernel.o obj/compute_ani_undoatt_kernel.cuda-kernel.o > obj/compute_i\ > so_undoatt_kernel.cuda-kernel.o > obj/compute_strain_kernel.cuda-kernel.o obj/out\ > er_core_impl_kernel_forward.cuda-kernel.o > obj/outer_core_impl_kernel_adjoint.cu\ > da-kernel.o obj/inner_core_impl_kernel_forward.cuda-kernel.o > obj/inner_core_imp\ > l_kernel_adjoint.cuda-kernel.o > obj/crust_mantle_impl_kernel_forward.cuda-kernel\ > .o obj/crust_mantle_impl_kernel_adjoint.cuda-kernel.o > make: DUSE_OLDER_CUDA4_GPU: Command not found > make: [obj/cuda_device_obj.o] Error 127 (ignored) > > -- > Phil Cummins > Prof. Natural Hazards > Research School of Earth Sciences > Australian National University > _______________________________________________ > CIG-SEISMO mailing list > CIG-SEISMO at geodynamics.org > http://lists.geodynamics.org/cgi-bin/mailman/listinfo/cig-seismo -------------- next part -------------- An HTML attachment was scrubbed... URL: From phil.cummins at anu.edu.au Tue Apr 25 20:12:06 2017 From: phil.cummins at anu.edu.au (Phil Cummins) Date: Wed, 26 Apr 2017 13:12:06 +1000 Subject: [CIG-SEISMO] specfem3d_globe with CUDA In-Reply-To: <58FFF24C.3050602@anu.edu.au> References: <58FC0735.3030308@anu.edu.au> <58FFF24C.3050602@anu.edu.au> Message-ID: <59001006.5050309@anu.edu.au> Just a quick reply to my own post. I got xpsecfem3d to link by removing cuda_device_obj.o from the mpif90 link line, and I also had to add -lstdc++ to resolve the missing symbols from the c++ library. Phil Cummins wrote: > Hello again, > > Still trying to compile specfem3d_globe using CUDA. I have obtaiend > advice from our HPC center regarding the appropriate modules to use, > which are: > intel-fc/16.0.3.210 intel-cc/16.0.3.210 openmpi/1.10.2 cuda/8.0 > I hope they're going to be OK. But I just can't make sense of what the > Makefiule is trying to do with cuda_device_obj. As an exampe, let's > consider what's happening with the CUDA module check_fields_gpu. First > it is compiled into object code: > nvcc -c src/gpu/check_fields_gpu.c -o obj/check_fields_gpu.cuda.o > --cudart=shared -I/apps/openmpi/1.10.2/include -DUSE_OLDER_CUDA4_GPU > -gencode=arch=compute_20,code=\"sm_20,compute_20\" -x cu -I./setup > -I./src/gpu/kernels.gen -DUSE_CUDA > > That's fine (it does produce a deprecation warning). Then, for some > reason that I can't figure out, the makefile is trying to link this in > some way to the module cuda_device_obj, for which no source code seems > to exist and for which the command appears to have omitted the nvcc > compiler (so of course it doesn't work): > > DUSE_OLDER_CUDA4_GPU > -gencode=arch=compute_20,code=\"sm_20,compute_20\" -o > obj/cuda_device_obj.o obj/assemble_MPI_scalar_gpu.cuda.o > obj/assemble_MPI_vector_gpu.cuda.o obj/check_fields_gpu.cuda.o > obj/compute_add_sources_elastic_gpu.cuda.o > obj/compute_coupling_gpu.cuda.o > obj/compute_forces_crust_mantle_gpu.cuda.o > obj/compute_forces_inner_core_gpu.cuda.o > obj/compute_forces_outer_core_gpu.cuda.o > obj/compute_kernels_gpu.cuda.o obj/compute_stacey_acoustic_gpu.cuda.o > obj/compute_stacey_elastic_gpu.cuda.o obj/compute_strain_gpu.cuda.o > obj/helper_functions_gpu.cuda.o obj/initialize_gpu.cuda.o > obj/noise_tomography_gpu.cuda.o obj/prepare_mesh_constants_gpu.cuda.o > obj/transfer_fields_gpu.cuda.o obj/update_displacement_gpu.cuda.o > obj/write_seismograms_gpu.cuda.o > obj/save_and_compare_cpu_vs_gpu.cuda.o > obj/assemble_boundary_accel_on_device.cuda-kernel.o > obj/assemble_boundary_potential_on_device.cuda-kernel.o > obj/prepare_boundary_potential_on_device.cuda-kernel.o > obj/prepare_boundary_accel_on_device.cuda-kernel.o > obj/get_maximum_scalar_kernel.cuda-kernel.o > obj/get_maximum_vector_kernel.cuda-kernel.o > obj/compute_add_sources_adjoint_kernel.cuda-kernel.o > obj/compute_add_sources_kernel.cuda-kernel.o > > Then, the commands issued by the Makefile try to create the main > program xpsecfem3d by linking not only with cuda_device_obj.o, but > also with check_fields_gpu.cuda.o. So even if I could get the above > command to work somehow, the mpif90 command would almost certainly > fail because of multiple definitions. In any case, I am struggling a > bit because I just don't understand how or why cuda_device_obj.o is > meant to be used. Can anyone help? > > mpif90 -g -xHost -fpe0 -ftz -assume buffered_io -assume byterecl > -align sequence -vec-report0 -std03 -diag-disable 6477 -implicitnone > -gen-interfaces -warn all -O3 -check nobounds -DFORCE_VECTORIZATION -o > ./bin/xspecfem3D ./obj/assemble_MPI_scalar.solver.o > ./obj/assemble_MPI_vector.solver.o ./obj/comp_source_spectrum.solver.o > ./obj/compute_adj_source_frechet.solver.o ./obj/convert_time.solver.o > ./obj/define_derivation_matrices.solver.o ./obj/file_io_threads.cc.o > ./obj/force_ftz.cc.o ./obj/get_backazimuth.solver.o > ./obj/get_cmt.solver.o ./obj/get_event_info.solver.o > ./obj/make_gravity.solver.o ./obj/netlib_specfun_erf.solver.o > ./obj/asdf_data.solverstatic_module.o > ./obj/comp_source_time_function.solverstatic.o > ./obj/specfem3D_par.solverstatic_module.o > ./obj/write_seismograms.solverstatic.o > ./obj/check_stability.solverstatic.o > ./obj/compute_add_sources.solverstatic.o > ./obj/compute_arrays_source.solverstatic.o > ./obj/compute_boundary_kernel.solverstatic.o > ./obj/compute_coupling.solverstatic.o > ./obj/compute_element.solverstatic.o > ./obj/compute_element_att_memory.solverstatic.o > ./obj/compute_element_strain.solverstatic.o > ./obj/compute_forces_acoustic_calling_routine.solverstatic.o > ./obj/compute_forces_viscoelastic_calling_routine.solverstatic.o > ./obj/compute_forces_crust_mantle_noDev.solverstatic.o > ./obj/compute_forces_crust_mantle_Dev.solverstatic.o > ./obj/compute_forces_inner_core_noDev.solverstatic.o > ./obj/compute_forces_inner_core_Dev.solverstatic.o > ./obj/compute_forces_outer_core_noDev.solverstatic.o > ./obj/compute_forces_outer_core_Dev.solverstatic.o > ./obj/compute_kernels.solverstatic.o > ./obj/compute_seismograms.solverstatic.o > ./obj/compute_stacey_crust_mantle.solverstatic.o > ./obj/compute_stacey_outer_core.solverstatic.o ./obj/finali\ > ze_simulation.solverstatic.o ./obj/get_attenuation.solverstatic.o > ./obj/initialize_simulation.solverstatic.o > ./obj/iterate_time.solverstatic.o > ./obj/iterate_time_undoatt.solverstatic.o > ./obj/locate_receivers.solverstatic.o ./obj/locate_regular_points.s\ > olverstatic.o ./obj/locate_sources.solverstatic.o > ./obj/multiply_arrays_source.solverstatic.o > ./obj/noise_tomography.solverstatic.o > ./obj/prepare_timerun.solverstatic.o > ./obj/read_adjoint_sources.solverstatic.o > ./obj/read_arrays_solver.solverstatic.o > ./obj/read_forward_arrays.solverstatic.o > ./obj/read_mesh_databases.solverstatic.o > ./obj/read_topography_bathymetry.solverstatic.o > ./obj/save_forward_arrays.solverstatic.o > ./obj/save_kernels.solverstatic.o > ./obj/save_regular_kernels.solverstatic.o > ./obj/setup_GLL_points.solverstatic.o > ./obj/setup_sources_receivers.solverstatic.o > ./obj/specfem3D.solverstatic.o ./obj/update_displ\ > acement_LDDRK.solverstatic.o > ./obj/update_displacement_Newmark.solverstatic.o > ./obj/write_movie_output.solverstatic.o > ./obj/write_movie_volume.solverstatic.o > ./obj/write_movie_surface.solverstatic.o > ./obj/write_output_ASCII.solverstatic.o > ./obj/write_output_SAC.solverstatic.o > ./obj/assemble_MPI_scalar_gpu.cuda.o > ./obj/assemble_MPI_vector_gpu.cuda.o ./obj/check_fields_gpu.cuda.o > ./obj/compute_add_sources_elastic_gpu.cuda.o > ./obj/compute_coupling_gpu.cuda.o > ./obj/compute_forces_crust_mantle_gpu.cuda.o > ./obj/compute_forces_inner_core_gpu.cuda.o > ./obj/compute_forces_outer_core_gpu.cuda.o > ./obj/compute_kernels_gpu.cuda.o > ./obj/compute_stacey_acoustic_gpu.cuda.o > ./obj/compute_stacey_elastic_gpu.cuda.o > ./obj/compute_strain_gpu.cuda.o ./obj/helper_functions_gpu.cuda.o > ./obj/initialize_gpu.cuda.o ./obj/noise_tomography_gpu.cuda.o > ./obj/prepare_mesh_constants_gpu.cuda.o > ./obj/transfer_fields_gpu.cuda.o ./obj/update_displacement_gpu.cuda.o > ./obj/write_seismograms_gpu.cuda.o > ./obj/save_and_compare_cpu_vs_gpu.cuda.o ./obj/cuda_device_obj.o > ./obj/assemble_boundary_accel_on_device.cuda-kernel.o > ./obj/assemble_boundary_potential_on_device.cuda-kernel.o > ./obj/prepare_boundary_potential_on_device.cuda-kernel.o > ./obj/prepare_boundary_accel_on_device.cuda-kernel.o > ./obj/get_maximum_scalar_kernel.cuda-kernel.o > ./obj/get_maximum_vector_kernel.cuda-kernel.o > ./obj/compute_add_sources_adjoint_kernel.cuda-kernel.o > ./obj/compute_add_sources_kernel.cuda-kernel.o > ./obj/compute_coupling_fluid_CMB_kernel.cuda-kernel.o > ./obj/compute_coupling_fluid_ICB_kernel.cuda-kernel.o > ./obj/compute_coupling_CMB_fluid_kernel.cuda-kernel.o > ./obj/compute_coupling_ICB_fluid_kernel.cuda-kernel.o > ./obj/compute_coupling_ocean_kernel.cuda-kernel.o > ./obj/write_seismograms_transfer_from_device_kernel.cuda-kernel.o > ./obj/write_seismograms_transfer_strain_from_device_kernel.cuda-kernel.o > ./obj/noise_transfer_surface_to_host_kernel.cuda-kernel.o > ./obj/noise_add_source_master_rec_kernel.cuda-kernel.o > ./obj/noise_add_surface_movie_kernel.cuda-kernel.o > ./obj/compute_stacey_acoustic_kernel.cuda-kernel.o > ./obj/compute_stacey_acoustic_backward_kernel.cuda-kernel.o \ > ./obj/compute_stacey_elastic_kernel.cuda-kernel.o > ./obj/compute_stacey_elastic_backward_kernel.cuda-kernel.o > ./obj/update_disp_veloc_kernel.cuda-kernel.o > ./obj/update_potential_kernel.cuda-kernel.o > ./obj/update_accel_elastic_kernel.cuda-kernel.o > ./obj/update_veloc_elastic_kernel.cuda-kernel.o > ./obj/update_accel_acoustic_kernel.cuda-kernel.o > ./obj/update_veloc_acoustic_kernel.cuda-kernel.o > ./obj/compute_rho_kernel.cuda-kernel.o > ./obj/compute_iso_kernel.cuda-kernel.o > ./obj/compute_ani_kernel.cuda-kernel.o > ./obj/compute_hess_kernel.cuda-kernel.o > ./obj/compute_acoustic_kernel.cuda-kernel.o > ./obj/compute_strength_noise_kernel.cuda-kernel.o > ./obj/compute_ani_undoatt_kernel.cuda-kernel.o > ./obj/compute_iso_undoatt_kernel.cuda-kernel.o > ./obj/compute_strain_kernel.cuda-kernel.o > ./obj/outer_core_impl_kernel_forward.cuda-kernel.o > ./obj/outer_core_impl_kernel_adjoint.cudakernel.o > ./obj/inner_core_impl_kernel_forward.cuda-kernel.o > ./obj/inner_core_impl_kernel_adjoint.cuda-kernel.o > ./obj/crust_mantle_impl_kernel_forward.cuda-kernel.o > ./obj/crust_mantle_impl_kernel_adjoint.cuda-kernel.o > ./obj/visual_vtk_stubs.visualc.o ./obj/shared_par.shared_module.o > ./obj/auto_ner.shared.o ./obj/binary_c_io.cc.o > ./obj/broadcast_computed_parameters.shared.o ./obj/calendar.shared.o > ./obj/count_elements.shared.o ./obj/count_number_of_sources.shared.o > ./obj/count_points.shared.o ./obj/create_name_database.shared.o > ./obj/define_all_layers.shared.o ./obj/exit_mpi.shared.o > ./obj/flush_system.shared.o ./obj/get_model_parameters.shared.o > ./obj/get_timestep_and_layers.shared.o ./obj/gll_library.shared.o > ./obj/hex_nodes.shared.o ./obj/intgrl.shared.o > ./obj/lagrange_poly.shared.o ./obj/make_ellipticity.shared.o > ./obj/model_prem.shared.o ./obj/model_topo_bathy.shared.o > ./obj/parallel.sharedmpi.o ./obj/param_reader.cc.o > ./obj/read_compute_parameters.shared.o > ./obj/read_parameter_file.shared.o > ./obj/read_value_parameters.shared.o ./obj/recompute_jacobian.shared.o > ./obj/reduce.shared.o ./obj/rthetaphi_xyz.shared.o > ./obj/spline_routines.shared.o ./obj/write_VTK_file.shared.o > ./obj/adios_method_stubs.cc.o -lcudart > > > Thanks, > > - Phil > > P.S. when i simply remove cuda_dev_obj.o from the above mpif90 > command, the link fails with: > ./obj/assemble_MPI_scalar_gpu.cuda.o:(.eh_frame+0x12): undefined > reference to `__gxx_personality_v0' > ./obj/assemble_MPI_vector_gpu.cuda.o:(.eh_frame+0x12): undefined > reference to `__gxx_personality_v0' > ./obj/check_fields_gpu.cuda.o:(.eh_frame+0x12): undefined reference to > `__gxx_personality_v0' > ./obj/compute_add_sources_elastic_gpu.cuda.o:(.eh_frame+0x12): > undefined reference to `__gxx_personality_v0' > ./obj/compute_coupling_gpu.cuda.o:(.eh_frame+0x12): undefined > reference to `__gxx_personality_v0' > ./obj/compute_forces_crust_mantle_gpu.cuda.o:(.eh_frame+0x12): more > undefined references to `__gxx_personality_v0' follow > > Phil Cummins wrote: >> Hello, >> >> I'm would like to use specfem3d_globe with my university's GPU >> cluster. I am trying to compile specfem3d_globe-7.0.0 after having >> configured it using: >> ./configure --with-cuda=cuda6 >> I am using the ifort fortran compiler version 12.1.9.293, and the >> version of cuda I have loaded is 6.5. >> The "make all" command does a lot of compilation using ifort and >> nvcc, but eventually fails when trying to link xspecfem3D itself with >> the error: >> >> ifort: error #10236: File not found: './obj/cuda_device_obj.o' >> make: *** [bin/xspecfem3D] Error 1 >> >> Delving into what happened during the make (see below), I noticed >> that after using nvcc to compile crust_mantle_impl_kernel_adjoint.cu, >> it attempts to use the command DUSE_OLDER_CUDA4_GPU to compile >> cuda_device_obj, which of course doesn't work because >> DUSE_OLDER_CUDA4_GPU is a compiler option not a command. It seems >> that make has somehow lost the beginning of this particular >> compilation command. >> Has anyone else seen or know what to do about it? >> Thanks, >> >> - Phil >> >> P.S. Some of the hardware on our cluster can be used only with cuda >> version 8. Does anyone know if specfem3d will work with cuda8? >> >> nvcc -c src/gpu/kernels.gen/crust_mantle_impl_kernel_adjoint.cu -o >> obj/crust_ma\ >> ntle_impl_kernel_adjoint.cuda-kernel.o --cudart=shared >> -I/apps/openmpi/1.7.5/\ >> include -DUSE_OLDER_CUDA4_GPU >> -gencode=arch=compute_20,code=\"sm_20,compute_20\\ >> " -x cu -I./setup -I./src/gpu/kernels.gen -DUSE_CUDA -include >> src/gpu/mesh_con\ >> stants_gpu.h >> DUSE_OLDER_CUDA4_GPU >> -gencode=arch=compute_20,code=\"sm_20,compute_20\" -o obj/\ >> cuda_device_obj.o obj/assemble_MPI_scalar_gpu.cuda.o >> obj/assemble_MPI_vector_gp\ >> u.cuda.o obj/check_fields_gpu.cuda.o >> obj/compute_add_sources_elastic_gpu.cuda.o\ >> obj/compute_coupling_gpu.cuda.o >> obj/compute_forces_crust_mantle_gpu.cuda.o obj\ >> /compute_forces_inner_core_gpu.cuda.o >> obj/compute_forces_outer_core_gpu.cuda.o \ >> obj/compute_kernels_gpu.cuda.o obj/compute_stacey_acoustic_gpu.cuda.o >> obj/compu\ >> te_stacey_elastic_gpu.cuda.o obj/compute_strain_gpu.cuda.o >> obj/helper_functions\ >> _gpu.cuda.o obj/initialize_gpu.cuda.o obj/noise_tomography_gpu.cuda.o >> obj/prepa\ >> re_mesh_constants_gpu.cuda.o obj/transfer_fields_gpu.cuda.o >> obj/update_displace\ >> ment_gpu.cuda.o obj/write_seismograms_gpu.cuda.o >> obj/save_and_compare_cpu_vs_gp\ >> u.cuda.o obj/assemble_boundary_accel_on_device.cuda-kernel.o >> obj/assemble_bound\ >> ary_potential_on_device.cuda-kernel.o >> obj/prepare_boundary_potential_on_device.\ >> cuda-kernel.o obj/prepare_boundary_accel_on_device.cuda-kernel.o >> obj/get_maximu\ >> m_scalar_kernel.cuda-kernel.o >> obj/get_maximum_vector_kernel.cuda-kernel.o obj/c\ >> ompute_add_sources_adjoint_kernel.cuda-kernel.o >> obj/compute_add_sources_kernel.\ >> cuda-kernel.o >> nel.cuda-kernel.o >> obj/write_seismograms_transfer_strain_from_device_kernel.cuda\ >> -kernel.o obj/noise_transfer_surface_to_host_kernel.cuda-kernel.o >> obj/noise_add\ >> _source_master_rec_kernel.cuda-kernel.o >> obj/noise_add_surface_movie_kernel.cuda\ >> -kernel.o obj/compute_stacey_acoustic_kernel.cuda-kernel.o >> obj/compute_stacey_a\ >> coustic_backward_kernel.cuda-kernel.o >> obj/compute_stacey_elastic_kernel.cuda-ke\ >> rnel.o obj/compute_stacey_elastic_backward_kernel.cuda-kernel.o >> obj/update_disp\ >> _veloc_kernel.cuda-kernel.o obj/update_potential_kernel.cuda-kernel.o >> obj/updat\ >> e_accel_elastic_kernel.cuda-kernel.o >> obj/update_veloc_elastic_kernel.cuda-kerne\ >> l.o obj/update_accel_acoustic_kernel.cuda-kernel.o >> obj/update_veloc_acoustic_ke\ >> rnel.cuda-kernel.o obj/compute_rho_kernel.cuda-kernel.o >> obj/compute_iso_kernel.\ >> cuda-kernel.o obj/compute_ani_kernel.cuda-kernel.o >> obj/compute_hess_kernel.cuda\ >> -kernel.o obj/compute_acoustic_kernel.cuda-kernel.o >> obj/compute_strength_noise_\ >> kernel.cuda-kernel.o obj/compute_ani_undoatt_kernel.cuda-kernel.o >> obj/compute_i\ >> so_undoatt_kernel.cuda-kernel.o >> obj/compute_strain_kernel.cuda-kernel.o obj/out\ >> er_core_impl_kernel_forward.cuda-kernel.o >> obj/outer_core_impl_kernel_adjoint.cu\ >> da-kernel.o obj/inner_core_impl_kernel_forward.cuda-kernel.o >> obj/inner_core_imp\ >> l_kernel_adjoint.cuda-kernel.o >> obj/crust_mantle_impl_kernel_forward.cuda-kernel\ >> .o obj/crust_mantle_impl_kernel_adjoint.cuda-kernel.o >> make: DUSE_OLDER_CUDA4_GPU: Command not found >> make: [obj/cuda_device_obj.o] Error 127 (ignored) >> >> -- >> Phil Cummins >> Prof. Natural Hazards >> Research School of Earth Sciences >> Australian National University >> _______________________________________________ >> CIG-SEISMO mailing list >> CIG-SEISMO at geodynamics.org >> http://lists.geodynamics.org/cgi-bin/mailman/listinfo/cig-seismo > > _______________________________________________ > CIG-SEISMO mailing list > CIG-SEISMO at geodynamics.org > http://lists.geodynamics.org/cgi-bin/mailman/listinfo/cig-seismo -------------- next part -------------- An HTML attachment was scrubbed... URL: From komatitsch at lma.cnrs-mrs.fr Wed Apr 26 03:54:51 2017 From: komatitsch at lma.cnrs-mrs.fr (Dimitri Komatitsch) Date: Wed, 26 Apr 2017 12:54:51 +0200 Subject: [CIG-SEISMO] specfem3d_globe with CUDA In-Reply-To: <59001006.5050309@anu.edu.au> References: <58FC0735.3030308@anu.edu.au> <58FFF24C.3050602@anu.edu.au> <59001006.5050309@anu.edu.au> Message-ID: <57db055a-4a50-0749-4f6c-4873f01eb052@lma.cnrs-mrs.fr> Hi Phil, Hi all, Thanks! Let me cc Daniel Peter and Peter Messmer, who are very familiar with the CUDA version. Daniel, would you know how to commit Phil's fix to the official Git version of the code? (and I guess also for SPECFEM3D_Cartesian and for SPECFEM2D?). Thanks, Best wishes, Dimitri. On 04/26/2017 05:12 AM, Phil Cummins wrote: > Just a quick reply to my own post. I got xpsecfem3d to link by removing > cuda_device_obj.o from the mpif90 link line, and I also had to add > -lstdc++ to resolve the missing symbols from the c++ library. > > Phil Cummins wrote: >> Hello again, >> >> Still trying to compile specfem3d_globe using CUDA. I have obtaiend >> advice from our HPC center regarding the appropriate modules to use, >> which are: >> intel-fc/16.0.3.210 intel-cc/16.0.3.210 openmpi/1.10.2 cuda/8.0 >> I hope they're going to be OK. But I just can't make sense of what the >> Makefiule is trying to do with cuda_device_obj. As an exampe, let's >> consider what's happening with the CUDA module check_fields_gpu. First >> it is compiled into object code: >> nvcc -c src/gpu/check_fields_gpu.c -o obj/check_fields_gpu.cuda.o >> --cudart=shared -I/apps/openmpi/1.10.2/include -DUSE_OLDER_CUDA4_GPU >> -gencode=arch=compute_20,code=\"sm_20,compute_20\" -x cu -I./setup >> -I./src/gpu/kernels.gen -DUSE_CUDA >> >> That's fine (it does produce a deprecation warning). Then, for some >> reason that I can't figure out, the makefile is trying to link this in >> some way to the module cuda_device_obj, for which no source code seems >> to exist and for which the command appears to have omitted the nvcc >> compiler (so of course it doesn't work): >> >> DUSE_OLDER_CUDA4_GPU >> -gencode=arch=compute_20,code=\"sm_20,compute_20\" -o >> obj/cuda_device_obj.o obj/assemble_MPI_scalar_gpu.cuda.o >> obj/assemble_MPI_vector_gpu.cuda.o obj/check_fields_gpu.cuda.o >> obj/compute_add_sources_elastic_gpu.cuda.o >> obj/compute_coupling_gpu.cuda.o >> obj/compute_forces_crust_mantle_gpu.cuda.o >> obj/compute_forces_inner_core_gpu.cuda.o >> obj/compute_forces_outer_core_gpu.cuda.o >> obj/compute_kernels_gpu.cuda.o obj/compute_stacey_acoustic_gpu.cuda.o >> obj/compute_stacey_elastic_gpu.cuda.o obj/compute_strain_gpu.cuda.o >> obj/helper_functions_gpu.cuda.o obj/initialize_gpu.cuda.o >> obj/noise_tomography_gpu.cuda.o obj/prepare_mesh_constants_gpu.cuda.o >> obj/transfer_fields_gpu.cuda.o obj/update_displacement_gpu.cuda.o >> obj/write_seismograms_gpu.cuda.o >> obj/save_and_compare_cpu_vs_gpu.cuda.o >> obj/assemble_boundary_accel_on_device.cuda-kernel.o >> obj/assemble_boundary_potential_on_device.cuda-kernel.o >> obj/prepare_boundary_potential_on_device.cuda-kernel.o >> obj/prepare_boundary_accel_on_device.cuda-kernel.o >> obj/get_maximum_scalar_kernel.cuda-kernel.o >> obj/get_maximum_vector_kernel.cuda-kernel.o >> obj/compute_add_sources_adjoint_kernel.cuda-kernel.o >> obj/compute_add_sources_kernel.cuda-kernel.o >> >> Then, the commands issued by the Makefile try to create the main >> program xpsecfem3d by linking not only with cuda_device_obj.o, but >> also with check_fields_gpu.cuda.o. So even if I could get the above >> command to work somehow, the mpif90 command would almost certainly >> fail because of multiple definitions. In any case, I am struggling a >> bit because I just don't understand how or why cuda_device_obj.o is >> meant to be used. Can anyone help? >> >> mpif90 -g -xHost -fpe0 -ftz -assume buffered_io -assume byterecl >> -align sequence -vec-report0 -std03 -diag-disable 6477 -implicitnone >> -gen-interfaces -warn all -O3 -check nobounds -DFORCE_VECTORIZATION -o >> ./bin/xspecfem3D ./obj/assemble_MPI_scalar.solver.o >> ./obj/assemble_MPI_vector.solver.o ./obj/comp_source_spectrum.solver.o >> ./obj/compute_adj_source_frechet.solver.o ./obj/convert_time.solver.o >> ./obj/define_derivation_matrices.solver.o ./obj/file_io_threads.cc.o >> ./obj/force_ftz.cc.o ./obj/get_backazimuth.solver.o >> ./obj/get_cmt.solver.o ./obj/get_event_info.solver.o >> ./obj/make_gravity.solver.o ./obj/netlib_specfun_erf.solver.o >> ./obj/asdf_data.solverstatic_module.o >> ./obj/comp_source_time_function.solverstatic.o >> ./obj/specfem3D_par.solverstatic_module.o >> ./obj/write_seismograms.solverstatic.o >> ./obj/check_stability.solverstatic.o >> ./obj/compute_add_sources.solverstatic.o >> ./obj/compute_arrays_source.solverstatic.o >> ./obj/compute_boundary_kernel.solverstatic.o >> ./obj/compute_coupling.solverstatic.o >> ./obj/compute_element.solverstatic.o >> ./obj/compute_element_att_memory.solverstatic.o >> ./obj/compute_element_strain.solverstatic.o >> ./obj/compute_forces_acoustic_calling_routine.solverstatic.o >> ./obj/compute_forces_viscoelastic_calling_routine.solverstatic.o >> ./obj/compute_forces_crust_mantle_noDev.solverstatic.o >> ./obj/compute_forces_crust_mantle_Dev.solverstatic.o >> ./obj/compute_forces_inner_core_noDev.solverstatic.o >> ./obj/compute_forces_inner_core_Dev.solverstatic.o >> ./obj/compute_forces_outer_core_noDev.solverstatic.o >> ./obj/compute_forces_outer_core_Dev.solverstatic.o >> ./obj/compute_kernels.solverstatic.o >> ./obj/compute_seismograms.solverstatic.o >> ./obj/compute_stacey_crust_mantle.solverstatic.o >> ./obj/compute_stacey_outer_core.solverstatic.o ./obj/finali\ >> ze_simulation.solverstatic.o ./obj/get_attenuation.solverstatic.o >> ./obj/initialize_simulation.solverstatic.o >> ./obj/iterate_time.solverstatic.o >> ./obj/iterate_time_undoatt.solverstatic.o >> ./obj/locate_receivers.solverstatic.o ./obj/locate_regular_points.s\ >> olverstatic.o ./obj/locate_sources.solverstatic.o >> ./obj/multiply_arrays_source.solverstatic.o >> ./obj/noise_tomography.solverstatic.o >> ./obj/prepare_timerun.solverstatic.o >> ./obj/read_adjoint_sources.solverstatic.o >> ./obj/read_arrays_solver.solverstatic.o >> ./obj/read_forward_arrays.solverstatic.o >> ./obj/read_mesh_databases.solverstatic.o >> ./obj/read_topography_bathymetry.solverstatic.o >> ./obj/save_forward_arrays.solverstatic.o >> ./obj/save_kernels.solverstatic.o >> ./obj/save_regular_kernels.solverstatic.o >> ./obj/setup_GLL_points.solverstatic.o >> ./obj/setup_sources_receivers.solverstatic.o >> ./obj/specfem3D.solverstatic.o ./obj/update_displ\ >> acement_LDDRK.solverstatic.o >> ./obj/update_displacement_Newmark.solverstatic.o >> ./obj/write_movie_output.solverstatic.o >> ./obj/write_movie_volume.solverstatic.o >> ./obj/write_movie_surface.solverstatic.o >> ./obj/write_output_ASCII.solverstatic.o >> ./obj/write_output_SAC.solverstatic.o >> ./obj/assemble_MPI_scalar_gpu.cuda.o >> ./obj/assemble_MPI_vector_gpu.cuda.o ./obj/check_fields_gpu.cuda.o >> ./obj/compute_add_sources_elastic_gpu.cuda.o >> ./obj/compute_coupling_gpu.cuda.o >> ./obj/compute_forces_crust_mantle_gpu.cuda.o >> ./obj/compute_forces_inner_core_gpu.cuda.o >> ./obj/compute_forces_outer_core_gpu.cuda.o >> ./obj/compute_kernels_gpu.cuda.o >> ./obj/compute_stacey_acoustic_gpu.cuda.o >> ./obj/compute_stacey_elastic_gpu.cuda.o >> ./obj/compute_strain_gpu.cuda.o ./obj/helper_functions_gpu.cuda.o >> ./obj/initialize_gpu.cuda.o ./obj/noise_tomography_gpu.cuda.o >> ./obj/prepare_mesh_constants_gpu.cuda.o >> ./obj/transfer_fields_gpu.cuda.o ./obj/update_displacement_gpu.cuda.o >> ./obj/write_seismograms_gpu.cuda.o >> ./obj/save_and_compare_cpu_vs_gpu.cuda.o ./obj/cuda_device_obj.o >> ./obj/assemble_boundary_accel_on_device.cuda-kernel.o >> ./obj/assemble_boundary_potential_on_device.cuda-kernel.o >> ./obj/prepare_boundary_potential_on_device.cuda-kernel.o >> ./obj/prepare_boundary_accel_on_device.cuda-kernel.o >> ./obj/get_maximum_scalar_kernel.cuda-kernel.o >> ./obj/get_maximum_vector_kernel.cuda-kernel.o >> ./obj/compute_add_sources_adjoint_kernel.cuda-kernel.o >> ./obj/compute_add_sources_kernel.cuda-kernel.o >> ./obj/compute_coupling_fluid_CMB_kernel.cuda-kernel.o >> ./obj/compute_coupling_fluid_ICB_kernel.cuda-kernel.o >> ./obj/compute_coupling_CMB_fluid_kernel.cuda-kernel.o >> ./obj/compute_coupling_ICB_fluid_kernel.cuda-kernel.o >> ./obj/compute_coupling_ocean_kernel.cuda-kernel.o >> ./obj/write_seismograms_transfer_from_device_kernel.cuda-kernel.o >> ./obj/write_seismograms_transfer_strain_from_device_kernel.cuda-kernel.o >> ./obj/noise_transfer_surface_to_host_kernel.cuda-kernel.o >> ./obj/noise_add_source_master_rec_kernel.cuda-kernel.o >> ./obj/noise_add_surface_movie_kernel.cuda-kernel.o >> ./obj/compute_stacey_acoustic_kernel.cuda-kernel.o >> ./obj/compute_stacey_acoustic_backward_kernel.cuda-kernel.o \ >> ./obj/compute_stacey_elastic_kernel.cuda-kernel.o >> ./obj/compute_stacey_elastic_backward_kernel.cuda-kernel.o >> ./obj/update_disp_veloc_kernel.cuda-kernel.o >> ./obj/update_potential_kernel.cuda-kernel.o >> ./obj/update_accel_elastic_kernel.cuda-kernel.o >> ./obj/update_veloc_elastic_kernel.cuda-kernel.o >> ./obj/update_accel_acoustic_kernel.cuda-kernel.o >> ./obj/update_veloc_acoustic_kernel.cuda-kernel.o >> ./obj/compute_rho_kernel.cuda-kernel.o >> ./obj/compute_iso_kernel.cuda-kernel.o >> ./obj/compute_ani_kernel.cuda-kernel.o >> ./obj/compute_hess_kernel.cuda-kernel.o >> ./obj/compute_acoustic_kernel.cuda-kernel.o >> ./obj/compute_strength_noise_kernel.cuda-kernel.o >> ./obj/compute_ani_undoatt_kernel.cuda-kernel.o >> ./obj/compute_iso_undoatt_kernel.cuda-kernel.o >> ./obj/compute_strain_kernel.cuda-kernel.o >> ./obj/outer_core_impl_kernel_forward.cuda-kernel.o >> ./obj/outer_core_impl_kernel_adjoint.cudakernel.o >> ./obj/inner_core_impl_kernel_forward.cuda-kernel.o >> ./obj/inner_core_impl_kernel_adjoint.cuda-kernel.o >> ./obj/crust_mantle_impl_kernel_forward.cuda-kernel.o >> ./obj/crust_mantle_impl_kernel_adjoint.cuda-kernel.o >> ./obj/visual_vtk_stubs.visualc.o ./obj/shared_par.shared_module.o >> ./obj/auto_ner.shared.o ./obj/binary_c_io.cc.o >> ./obj/broadcast_computed_parameters.shared.o ./obj/calendar.shared.o >> ./obj/count_elements.shared.o ./obj/count_number_of_sources.shared.o >> ./obj/count_points.shared.o ./obj/create_name_database.shared.o >> ./obj/define_all_layers.shared.o ./obj/exit_mpi.shared.o >> ./obj/flush_system.shared.o ./obj/get_model_parameters.shared.o >> ./obj/get_timestep_and_layers.shared.o ./obj/gll_library.shared.o >> ./obj/hex_nodes.shared.o ./obj/intgrl.shared.o >> ./obj/lagrange_poly.shared.o ./obj/make_ellipticity.shared.o >> ./obj/model_prem.shared.o ./obj/model_topo_bathy.shared.o >> ./obj/parallel.sharedmpi.o ./obj/param_reader.cc.o >> ./obj/read_compute_parameters.shared.o >> ./obj/read_parameter_file.shared.o >> ./obj/read_value_parameters.shared.o ./obj/recompute_jacobian.shared.o >> ./obj/reduce.shared.o ./obj/rthetaphi_xyz.shared.o >> ./obj/spline_routines.shared.o ./obj/write_VTK_file.shared.o >> ./obj/adios_method_stubs.cc.o -lcudart >> >> >> Thanks, >> >> - Phil >> >> P.S. when i simply remove cuda_dev_obj.o from the above mpif90 >> command, the link fails with: >> ./obj/assemble_MPI_scalar_gpu.cuda.o:(.eh_frame+0x12): undefined >> reference to `__gxx_personality_v0' >> ./obj/assemble_MPI_vector_gpu.cuda.o:(.eh_frame+0x12): undefined >> reference to `__gxx_personality_v0' >> ./obj/check_fields_gpu.cuda.o:(.eh_frame+0x12): undefined reference to >> `__gxx_personality_v0' >> ./obj/compute_add_sources_elastic_gpu.cuda.o:(.eh_frame+0x12): >> undefined reference to `__gxx_personality_v0' >> ./obj/compute_coupling_gpu.cuda.o:(.eh_frame+0x12): undefined >> reference to `__gxx_personality_v0' >> ./obj/compute_forces_crust_mantle_gpu.cuda.o:(.eh_frame+0x12): more >> undefined references to `__gxx_personality_v0' follow >> >> Phil Cummins wrote: >>> Hello, >>> >>> I'm would like to use specfem3d_globe with my university's GPU >>> cluster. I am trying to compile specfem3d_globe-7.0.0 after having >>> configured it using: >>> ./configure --with-cuda=cuda6 >>> I am using the ifort fortran compiler version 12.1.9.293, and the >>> version of cuda I have loaded is 6.5. >>> The "make all" command does a lot of compilation using ifort and >>> nvcc, but eventually fails when trying to link xspecfem3D itself with >>> the error: >>> >>> ifort: error #10236: File not found: './obj/cuda_device_obj.o' >>> make: *** [bin/xspecfem3D] Error 1 >>> >>> Delving into what happened during the make (see below), I noticed >>> that after using nvcc to compile crust_mantle_impl_kernel_adjoint.cu, >>> it attempts to use the command DUSE_OLDER_CUDA4_GPU to compile >>> cuda_device_obj, which of course doesn't work because >>> DUSE_OLDER_CUDA4_GPU is a compiler option not a command. It seems >>> that make has somehow lost the beginning of this particular >>> compilation command. >>> Has anyone else seen or know what to do about it? >>> Thanks, >>> >>> - Phil >>> >>> P.S. Some of the hardware on our cluster can be used only with cuda >>> version 8. Does anyone know if specfem3d will work with cuda8? >>> >>> nvcc -c src/gpu/kernels.gen/crust_mantle_impl_kernel_adjoint.cu -o >>> obj/crust_ma\ >>> ntle_impl_kernel_adjoint.cuda-kernel.o --cudart=shared >>> -I/apps/openmpi/1.7.5/\ >>> include -DUSE_OLDER_CUDA4_GPU >>> -gencode=arch=compute_20,code=\"sm_20,compute_20\\ >>> " -x cu -I./setup -I./src/gpu/kernels.gen -DUSE_CUDA -include >>> src/gpu/mesh_con\ >>> stants_gpu.h >>> DUSE_OLDER_CUDA4_GPU >>> -gencode=arch=compute_20,code=\"sm_20,compute_20\" -o obj/\ >>> cuda_device_obj.o obj/assemble_MPI_scalar_gpu.cuda.o >>> obj/assemble_MPI_vector_gp\ >>> u.cuda.o obj/check_fields_gpu.cuda.o >>> obj/compute_add_sources_elastic_gpu.cuda.o\ >>> obj/compute_coupling_gpu.cuda.o >>> obj/compute_forces_crust_mantle_gpu.cuda.o obj\ >>> /compute_forces_inner_core_gpu.cuda.o >>> obj/compute_forces_outer_core_gpu.cuda.o \ >>> obj/compute_kernels_gpu.cuda.o obj/compute_stacey_acoustic_gpu.cuda.o >>> obj/compu\ >>> te_stacey_elastic_gpu.cuda.o obj/compute_strain_gpu.cuda.o >>> obj/helper_functions\ >>> _gpu.cuda.o obj/initialize_gpu.cuda.o obj/noise_tomography_gpu.cuda.o >>> obj/prepa\ >>> re_mesh_constants_gpu.cuda.o obj/transfer_fields_gpu.cuda.o >>> obj/update_displace\ >>> ment_gpu.cuda.o obj/write_seismograms_gpu.cuda.o >>> obj/save_and_compare_cpu_vs_gp\ >>> u.cuda.o obj/assemble_boundary_accel_on_device.cuda-kernel.o >>> obj/assemble_bound\ >>> ary_potential_on_device.cuda-kernel.o >>> obj/prepare_boundary_potential_on_device.\ >>> cuda-kernel.o obj/prepare_boundary_accel_on_device.cuda-kernel.o >>> obj/get_maximu\ >>> m_scalar_kernel.cuda-kernel.o >>> obj/get_maximum_vector_kernel.cuda-kernel.o obj/c\ >>> ompute_add_sources_adjoint_kernel.cuda-kernel.o >>> obj/compute_add_sources_kernel.\ >>> cuda-kernel.o >>> nel.cuda-kernel.o >>> obj/write_seismograms_transfer_strain_from_device_kernel.cuda\ >>> -kernel.o obj/noise_transfer_surface_to_host_kernel.cuda-kernel.o >>> obj/noise_add\ >>> _source_master_rec_kernel.cuda-kernel.o >>> obj/noise_add_surface_movie_kernel.cuda\ >>> -kernel.o obj/compute_stacey_acoustic_kernel.cuda-kernel.o >>> obj/compute_stacey_a\ >>> coustic_backward_kernel.cuda-kernel.o >>> obj/compute_stacey_elastic_kernel.cuda-ke\ >>> rnel.o obj/compute_stacey_elastic_backward_kernel.cuda-kernel.o >>> obj/update_disp\ >>> _veloc_kernel.cuda-kernel.o obj/update_potential_kernel.cuda-kernel.o >>> obj/updat\ >>> e_accel_elastic_kernel.cuda-kernel.o >>> obj/update_veloc_elastic_kernel.cuda-kerne\ >>> l.o obj/update_accel_acoustic_kernel.cuda-kernel.o >>> obj/update_veloc_acoustic_ke\ >>> rnel.cuda-kernel.o obj/compute_rho_kernel.cuda-kernel.o >>> obj/compute_iso_kernel.\ >>> cuda-kernel.o obj/compute_ani_kernel.cuda-kernel.o >>> obj/compute_hess_kernel.cuda\ >>> -kernel.o obj/compute_acoustic_kernel.cuda-kernel.o >>> obj/compute_strength_noise_\ >>> kernel.cuda-kernel.o obj/compute_ani_undoatt_kernel.cuda-kernel.o >>> obj/compute_i\ >>> so_undoatt_kernel.cuda-kernel.o >>> obj/compute_strain_kernel.cuda-kernel.o obj/out\ >>> er_core_impl_kernel_forward.cuda-kernel.o >>> obj/outer_core_impl_kernel_adjoint.cu\ >>> da-kernel.o obj/inner_core_impl_kernel_forward.cuda-kernel.o >>> obj/inner_core_imp\ >>> l_kernel_adjoint.cuda-kernel.o >>> obj/crust_mantle_impl_kernel_forward.cuda-kernel\ >>> .o obj/crust_mantle_impl_kernel_adjoint.cuda-kernel.o >>> make: DUSE_OLDER_CUDA4_GPU: Command not found >>> make: [obj/cuda_device_obj.o] Error 127 (ignored) >>> >>> -- >>> Phil Cummins >>> Prof. Natural Hazards >>> Research School of Earth Sciences >>> Australian National University >>> >>> _______________________________________________ >>> CIG-SEISMO mailing list >>> CIG-SEISMO at geodynamics.org >>> http://lists.geodynamics.org/cgi-bin/mailman/listinfo/cig-seismo >> >> _______________________________________________ >> CIG-SEISMO mailing list >> CIG-SEISMO at geodynamics.org >> http://lists.geodynamics.org/cgi-bin/mailman/listinfo/cig-seismo > > > > _______________________________________________ > CIG-SEISMO mailing list > CIG-SEISMO at geodynamics.org > http://lists.geodynamics.org/cgi-bin/mailman/listinfo/cig-seismo > -- Dimitri Komatitsch, CNRS Research Director (DR CNRS) Laboratory of Mechanics and Acoustics, Marseille, France http://komatitsch.free.fr From jimma.tamirat at aau.edu.et Wed Apr 26 04:08:09 2017 From: jimma.tamirat at aau.edu.et (Tamirat Bekele) Date: Wed, 26 Apr 2017 14:08:09 +0300 Subject: [CIG-SEISMO] topography data preparation Message-ID: Dear All, I am new for Specfem3D geodynamical tool that I am bit confused how to prepare a tomography data and interfaces file for my domain. Is there any helpful links or wiki which describe about conversion of Digital Elevation Model data to ASCII format that is suitable for Specfem3D? Please could you kindly help me on that? Many many thanks! With regards, Tamirat -------------- next part -------------- An HTML attachment was scrubbed... URL: From phil.cummins at anu.edu.au Wed Apr 26 07:04:18 2017 From: phil.cummins at anu.edu.au (Phil Cummins) Date: Thu, 27 Apr 2017 00:04:18 +1000 Subject: [CIG-SEISMO] specfem3d_globe with CUDA In-Reply-To: <57db055a-4a50-0749-4f6c-4873f01eb052@lma.cnrs-mrs.fr> References: <58FC0735.3030308@anu.edu.au> <58FFF24C.3050602@anu.edu.au> <59001006.5050309@anu.edu.au> <57db055a-4a50-0749-4f6c-4873f01eb052@lma.cnrs-mrs.fr> Message-ID: <5900A8E2.7040809@anu.edu.au> Hi Dimitri, Thanks for the kind words, but my approach was just a workaround. I think someone smarter than me should try to fix the Makefile or configure script. I'm still not sure what was wrong with it. Good luck! - Phil > Dimitri Komatitsch > 26 April 2017 at 8:54 PM > > Hi Phil, Hi all, > > Thanks! Let me cc Daniel Peter and Peter Messmer, who are very > familiar with the CUDA version. Daniel, would you know how to commit > Phil's fix to the official Git version of the code? (and I guess also > for SPECFEM3D_Cartesian and for SPECFEM2D?). > > Thanks, > Best wishes, > > Dimitri. > > > > Phil Cummins > 23 April 2017 at 11:45 AM > Hello, > > I'm would like to use specfem3d_globe with my university's GPU > cluster. I am trying to compile specfem3d_globe-7.0.0 after having > configured it using: > ./configure --with-cuda=cuda6 > I am using the ifort fortran compiler version 12.1.9.293, and the > version of cuda I have loaded is 6.5. > The "make all" command does a lot of compilation using ifort and nvcc, > but eventually fails when trying to link xspecfem3D itself with the error: > > ifort: error #10236: File not found: './obj/cuda_device_obj.o' > make: *** [bin/xspecfem3D] Error 1 > > Delving into what happened during the make (see below), I noticed that > after using nvcc to compile crust_mantle_impl_kernel_adjoint.cu, it > attempts to use the command DUSE_OLDER_CUDA4_GPU to compile > cuda_device_obj, which of course doesn't work because > DUSE_OLDER_CUDA4_GPU is a compiler option not a command. It seems that > make has somehow lost the beginning of this particular compilation > command. > Has anyone else seen or know what to do about it? > Thanks, > > - Phil > > P.S. Some of the hardware on our cluster can be used only with cuda > version 8. Does anyone know if specfem3d will work with cuda8? > > nvcc -c src/gpu/kernels.gen/crust_mantle_impl_kernel_adjoint.cu -o > obj/crust_ma\ > ntle_impl_kernel_adjoint.cuda-kernel.o --cudart=shared > -I/apps/openmpi/1.7.5/\ > include -DUSE_OLDER_CUDA4_GPU > -gencode=arch=compute_20,code=\"sm_20,compute_20\\ > " -x cu -I./setup -I./src/gpu/kernels.gen -DUSE_CUDA -include > src/gpu/mesh_con\ > stants_gpu.h > DUSE_OLDER_CUDA4_GPU > -gencode=arch=compute_20,code=\"sm_20,compute_20\" -o obj/\ > cuda_device_obj.o obj/assemble_MPI_scalar_gpu.cuda.o > obj/assemble_MPI_vector_gp\ > u.cuda.o obj/check_fields_gpu.cuda.o > obj/compute_add_sources_elastic_gpu.cuda.o\ > obj/compute_coupling_gpu.cuda.o > obj/compute_forces_crust_mantle_gpu.cuda.o obj\ > /compute_forces_inner_core_gpu.cuda.o > obj/compute_forces_outer_core_gpu.cuda.o \ > obj/compute_kernels_gpu.cuda.o obj/compute_stacey_acoustic_gpu.cuda.o > obj/compu\ > te_stacey_elastic_gpu.cuda.o obj/compute_strain_gpu.cuda.o > obj/helper_functions\ > _gpu.cuda.o obj/initialize_gpu.cuda.o obj/noise_tomography_gpu.cuda.o > obj/prepa\ > re_mesh_constants_gpu.cuda.o obj/transfer_fields_gpu.cuda.o > obj/update_displace\ > ment_gpu.cuda.o obj/write_seismograms_gpu.cuda.o > obj/save_and_compare_cpu_vs_gp\ > u.cuda.o obj/assemble_boundary_accel_on_device.cuda-kernel.o > obj/assemble_bound\ > ary_potential_on_device.cuda-kernel.o > obj/prepare_boundary_potential_on_device.\ > cuda-kernel.o obj/prepare_boundary_accel_on_device.cuda-kernel.o > obj/get_maximu\ > m_scalar_kernel.cuda-kernel.o > obj/get_maximum_vector_kernel.cuda-kernel.o obj/c\ > ompute_add_sources_adjoint_kernel.cuda-kernel.o > obj/compute_add_sources_kernel.\ > cuda-kernel.o > nel.cuda-kernel.o > obj/write_seismograms_transfer_strain_from_device_kernel.cuda\ > -kernel.o obj/noise_transfer_surface_to_host_kernel.cuda-kernel.o > obj/noise_add\ > _source_master_rec_kernel.cuda-kernel.o > obj/noise_add_surface_movie_kernel.cuda\ > -kernel.o obj/compute_stacey_acoustic_kernel.cuda-kernel.o > obj/compute_stacey_a\ > coustic_backward_kernel.cuda-kernel.o > obj/compute_stacey_elastic_kernel.cuda-ke\ > rnel.o obj/compute_stacey_elastic_backward_kernel.cuda-kernel.o > obj/update_disp\ > _veloc_kernel.cuda-kernel.o obj/update_potential_kernel.cuda-kernel.o > obj/updat\ > e_accel_elastic_kernel.cuda-kernel.o > obj/update_veloc_elastic_kernel.cuda-kerne\ > l.o obj/update_accel_acoustic_kernel.cuda-kernel.o > obj/update_veloc_acoustic_ke\ > rnel.cuda-kernel.o obj/compute_rho_kernel.cuda-kernel.o > obj/compute_iso_kernel.\ > cuda-kernel.o obj/compute_ani_kernel.cuda-kernel.o > obj/compute_hess_kernel.cuda\ > -kernel.o obj/compute_acoustic_kernel.cuda-kernel.o > obj/compute_strength_noise_\ > kernel.cuda-kernel.o obj/compute_ani_undoatt_kernel.cuda-kernel.o > obj/compute_i\ > so_undoatt_kernel.cuda-kernel.o > obj/compute_strain_kernel.cuda-kernel.o obj/out\ > er_core_impl_kernel_forward.cuda-kernel.o > obj/outer_core_impl_kernel_adjoint.cu\ > da-kernel.o obj/inner_core_impl_kernel_forward.cuda-kernel.o > obj/inner_core_imp\ > l_kernel_adjoint.cuda-kernel.o > obj/crust_mantle_impl_kernel_forward.cuda-kernel\ > .o obj/crust_mantle_impl_kernel_adjoint.cuda-kernel.o > make: DUSE_OLDER_CUDA4_GPU: Command not found > make: [obj/cuda_device_obj.o] Error 127 (ignored) > > _______________________________________________ > CIG-SEISMO mailing list > CIG-SEISMO at geodynamics.org > http://lists.geodynamics.org/cgi-bin/mailman/listinfo/cig-seismo -- Phil Cummins Prof. Natural Hazards Research School of Earth Sciences Australian National University -------------- next part -------------- An HTML attachment was scrubbed... URL: From p.lanucara at cineca.it Wed Apr 26 07:26:19 2017 From: p.lanucara at cineca.it (Piero Lanucara) Date: Wed, 26 Apr 2017 16:26:19 +0200 Subject: [CIG-SEISMO] specfem3d_globe with CUDA In-Reply-To: <5900A8E2.7040809@anu.edu.au> References: <58FC0735.3030308@anu.edu.au> <58FFF24C.3050602@anu.edu.au> <59001006.5050309@anu.edu.au> <57db055a-4a50-0749-4f6c-4873f01eb052@lma.cnrs-mrs.fr> <5900A8E2.7040809@anu.edu.au> Message-ID: <63a9014d-80c0-5330-c826-0685cb4a1194@cineca.it> Hi all as far as I understand it should work with: configure.... --with-cuda=cuda5 regards Piero Il 26/04/2017 16:04, Phil Cummins ha scritto: > Hi Dimitri, > > Thanks for the kind words, but my approach was just a workaround. I > think someone smarter than me should try to fix the Makefile or > configure script. I'm still not sure what was wrong with it. > Good luck! > > - Phil > > >> Dimitri Komatitsch >> 26 April 2017 at 8:54 PM >> >> Hi Phil, Hi all, >> >> Thanks! Let me cc Daniel Peter and Peter Messmer, who are very >> familiar with the CUDA version. Daniel, would you know how to commit >> Phil's fix to the official Git version of the code? (and I guess also >> for SPECFEM3D_Cartesian and for SPECFEM2D?). >> >> Thanks, >> Best wishes, >> >> Dimitri. >> >> >> >> Phil Cummins >> 23 April 2017 at 11:45 AM >> Hello, >> >> I'm would like to use specfem3d_globe with my university's GPU >> cluster. I am trying to compile specfem3d_globe-7.0.0 after having >> configured it using: >> ./configure --with-cuda=cuda6 >> I am using the ifort fortran compiler version 12.1.9.293, and the >> version of cuda I have loaded is 6.5. >> The "make all" command does a lot of compilation using ifort and >> nvcc, but eventually fails when trying to link xspecfem3D itself with >> the error: >> >> ifort: error #10236: File not found: './obj/cuda_device_obj.o' >> make: *** [bin/xspecfem3D] Error 1 >> >> Delving into what happened during the make (see below), I noticed >> that after using nvcc to compile crust_mantle_impl_kernel_adjoint.cu, >> it attempts to use the command DUSE_OLDER_CUDA4_GPU to compile >> cuda_device_obj, which of course doesn't work because >> DUSE_OLDER_CUDA4_GPU is a compiler option not a command. It seems >> that make has somehow lost the beginning of this particular >> compilation command. >> Has anyone else seen or know what to do about it? >> Thanks, >> >> - Phil >> >> P.S. Some of the hardware on our cluster can be used only with cuda >> version 8. Does anyone know if specfem3d will work with cuda8? >> >> nvcc -c src/gpu/kernels.gen/crust_mantle_impl_kernel_adjoint.cu -o >> obj/crust_ma\ >> ntle_impl_kernel_adjoint.cuda-kernel.o --cudart=shared >> -I/apps/openmpi/1.7.5/\ >> include -DUSE_OLDER_CUDA4_GPU >> -gencode=arch=compute_20,code=\"sm_20,compute_20\\ >> " -x cu -I./setup -I./src/gpu/kernels.gen -DUSE_CUDA -include >> src/gpu/mesh_con\ >> stants_gpu.h >> DUSE_OLDER_CUDA4_GPU >> -gencode=arch=compute_20,code=\"sm_20,compute_20\" -o obj/\ >> cuda_device_obj.o obj/assemble_MPI_scalar_gpu.cuda.o >> obj/assemble_MPI_vector_gp\ >> u.cuda.o obj/check_fields_gpu.cuda.o >> obj/compute_add_sources_elastic_gpu.cuda.o\ >> obj/compute_coupling_gpu.cuda.o >> obj/compute_forces_crust_mantle_gpu.cuda.o obj\ >> /compute_forces_inner_core_gpu.cuda.o >> obj/compute_forces_outer_core_gpu.cuda.o \ >> obj/compute_kernels_gpu.cuda.o obj/compute_stacey_acoustic_gpu.cuda.o >> obj/compu\ >> te_stacey_elastic_gpu.cuda.o obj/compute_strain_gpu.cuda.o >> obj/helper_functions\ >> _gpu.cuda.o obj/initialize_gpu.cuda.o obj/noise_tomography_gpu.cuda.o >> obj/prepa\ >> re_mesh_constants_gpu.cuda.o obj/transfer_fields_gpu.cuda.o >> obj/update_displace\ >> ment_gpu.cuda.o obj/write_seismograms_gpu.cuda.o >> obj/save_and_compare_cpu_vs_gp\ >> u.cuda.o obj/assemble_boundary_accel_on_device.cuda-kernel.o >> obj/assemble_bound\ >> ary_potential_on_device.cuda-kernel.o >> obj/prepare_boundary_potential_on_device.\ >> cuda-kernel.o obj/prepare_boundary_accel_on_device.cuda-kernel.o >> obj/get_maximu\ >> m_scalar_kernel.cuda-kernel.o >> obj/get_maximum_vector_kernel.cuda-kernel.o obj/c\ >> ompute_add_sources_adjoint_kernel.cuda-kernel.o >> obj/compute_add_sources_kernel.\ >> cuda-kernel.o >> nel.cuda-kernel.o >> obj/write_seismograms_transfer_strain_from_device_kernel.cuda\ >> -kernel.o obj/noise_transfer_surface_to_host_kernel.cuda-kernel.o >> obj/noise_add\ >> _source_master_rec_kernel.cuda-kernel.o >> obj/noise_add_surface_movie_kernel.cuda\ >> -kernel.o obj/compute_stacey_acoustic_kernel.cuda-kernel.o >> obj/compute_stacey_a\ >> coustic_backward_kernel.cuda-kernel.o >> obj/compute_stacey_elastic_kernel.cuda-ke\ >> rnel.o obj/compute_stacey_elastic_backward_kernel.cuda-kernel.o >> obj/update_disp\ >> _veloc_kernel.cuda-kernel.o obj/update_potential_kernel.cuda-kernel.o >> obj/updat\ >> e_accel_elastic_kernel.cuda-kernel.o >> obj/update_veloc_elastic_kernel.cuda-kerne\ >> l.o obj/update_accel_acoustic_kernel.cuda-kernel.o >> obj/update_veloc_acoustic_ke\ >> rnel.cuda-kernel.o obj/compute_rho_kernel.cuda-kernel.o >> obj/compute_iso_kernel.\ >> cuda-kernel.o obj/compute_ani_kernel.cuda-kernel.o >> obj/compute_hess_kernel.cuda\ >> -kernel.o obj/compute_acoustic_kernel.cuda-kernel.o >> obj/compute_strength_noise_\ >> kernel.cuda-kernel.o obj/compute_ani_undoatt_kernel.cuda-kernel.o >> obj/compute_i\ >> so_undoatt_kernel.cuda-kernel.o >> obj/compute_strain_kernel.cuda-kernel.o obj/out\ >> er_core_impl_kernel_forward.cuda-kernel.o >> obj/outer_core_impl_kernel_adjoint.cu\ >> da-kernel.o obj/inner_core_impl_kernel_forward.cuda-kernel.o >> obj/inner_core_imp\ >> l_kernel_adjoint.cuda-kernel.o >> obj/crust_mantle_impl_kernel_forward.cuda-kernel\ >> .o obj/crust_mantle_impl_kernel_adjoint.cuda-kernel.o >> make: DUSE_OLDER_CUDA4_GPU: Command not found >> make: [obj/cuda_device_obj.o] Error 127 (ignored) >> >> _______________________________________________ >> CIG-SEISMO mailing list >> CIG-SEISMO at geodynamics.org >> http://lists.geodynamics.org/cgi-bin/mailman/listinfo/cig-seismo > > -- > Phil Cummins > Prof. Natural Hazards > Research School of Earth Sciences > Australian National University > > > _______________________________________________ > CIG-SEISMO mailing list > CIG-SEISMO at geodynamics.org > http://lists.geodynamics.org/cgi-bin/mailman/listinfo/cig-seismo -- Piero Lanucara CINECA SuperComputing Applications and Innovation Department - SCAI Phone: +39 06 44486709 Fax: +39 06 4957083 Address: CINECA Sede di Roma Via dei Tizii,6/b - 00185 Roma - Italy E-mail: p.lanucara at cineca.it skype: lpeter175 site: http://www.hpc.cineca.it -------------- next part -------------- An HTML attachment was scrubbed... URL: From daniel.peter at kaust.edu.sa Wed Apr 26 15:24:37 2017 From: daniel.peter at kaust.edu.sa (Daniel B. Peter) Date: Wed, 26 Apr 2017 22:24:37 +0000 Subject: [CIG-SEISMO] topography data preparation In-Reply-To: References: Message-ID: <8853E131-65C3-4A0D-8420-98238D0F67D4@kaust.edu.sa> Dear Tamirat, you mean for SPECFEM3D_Cartesian? in this case, you could try to follow this wiki: https://wiki.geodynamics.org/software:specfem3d:start#tutorial_3mount_st_helens within the package, there are also more examples to give you an idea how to use it with the in-house mesher, look in: EXAMPLES/meshfem3D_examples/ best wishes, daniel > On Apr 26, 2017, at 2:08 PM, Tamirat Bekele wrote: > > Dear All, > > I am new for Specfem3D geodynamical tool that I am bit confused how to prepare a tomography data and interfaces file for my domain. Is there any helpful links or wiki which describe about conversion of Digital Elevation Model data to ASCII format that is suitable for Specfem3D? > Please could you kindly help me on that? Many many thanks! > > With regards, > Tamirat > > > _______________________________________________ > CIG-SEISMO mailing list > CIG-SEISMO at geodynamics.org > http://lists.geodynamics.org/cgi-bin/mailman/listinfo/cig-seismo ________________________________ This message and its contents including attachments are intended solely for the original recipient. If you are not the intended recipient or have received this message in error, please notify me immediately and delete this message from your computer system. Any unauthorized use or distribution is prohibited. Please consider the environment before printing this email. From daniel.peter at kaust.edu.sa Wed Apr 26 15:10:14 2017 From: daniel.peter at kaust.edu.sa (Daniel B. Peter) Date: Wed, 26 Apr 2017 22:10:14 +0000 Subject: [CIG-SEISMO] specfem3d_globe with CUDA In-Reply-To: <63a9014d-80c0-5330-c826-0685cb4a1194@cineca.it> References: <58FC0735.3030308@anu.edu.au> <58FFF24C.3050602@anu.edu.au> <59001006.5050309@anu.edu.au> <57db055a-4a50-0749-4f6c-4873f01eb052@lma.cnrs-mrs.fr> <5900A8E2.7040809@anu.edu.au> <63a9014d-80c0-5330-c826-0685cb4a1194@cineca.it> Message-ID: Hi Phil, hi all, thanks for pointing out that the default —with-cuda=cuda which leads to CUDA4 compilation is probably obsolete by now and we should change this default. i’m going to make the changes and fix to the configuration script. in your case however, i’m not sure why the initial ./configure —with-cuda=cuda6 didn’t work. first, what are the GPU cards you want to run it on? then choose: -for Tesla K20: ./configure —with-cuda=cuda5 -for Tesla K80: ./configure —with-cuda=cuda6 -for Maxwell K2200: ./configure —with-cuda=cuda7 -for Pascal P100: ./configure —with-cuda=cuda8 we use the cuda*** specification as a short flag to point to the specific architecture and optimization. even if you have the cudatoolkit version 8, but want to compile kernels for a Tesla K20 card, you would choose —with-cuda=cuda5 best wishes, daniel On Apr 26, 2017, at 5:26 PM, Piero Lanucara > wrote: Hi all as far as I understand it should work with: configure.... --with-cuda=cuda5 regards Piero Il 26/04/2017 16:04, Phil Cummins ha scritto: Hi Dimitri, Thanks for the kind words, but my approach was just a workaround. I think someone smarter than me should try to fix the Makefile or configure script. I'm still not sure what was wrong with it. Good luck! - Phil Dimitri Komatitsch 26 April 2017 at 8:54 PM Hi Phil, Hi all, Thanks! Let me cc Daniel Peter and Peter Messmer, who are very familiar with the CUDA version. Daniel, would you know how to commit Phil's fix to the official Git version of the code? (and I guess also for SPECFEM3D_Cartesian and for SPECFEM2D?). Thanks, Best wishes, Dimitri. Phil Cummins 23 April 2017 at 11:45 AM Hello, I'm would like to use specfem3d_globe with my university's GPU cluster. I am trying to compile specfem3d_globe-7.0.0 after having configured it using: ./configure --with-cuda=cuda6 I am using the ifort fortran compiler version 12.1.9.293, and the version of cuda I have loaded is 6.5. The "make all" command does a lot of compilation using ifort and nvcc, but eventually fails when trying to link xspecfem3D itself with the error: ifort: error #10236: File not found: './obj/cuda_device_obj.o' make: *** [bin/xspecfem3D] Error 1 Delving into what happened during the make (see below), I noticed that after using nvcc to compile crust_mantle_impl_kernel_adjoint.cu, it attempts to use the command DUSE_OLDER_CUDA4_GPU to compile cuda_device_obj, which of course doesn't work because DUSE_OLDER_CUDA4_GPU is a compiler option not a command. It seems that make has somehow lost the beginning of this particular compilation command. Has anyone else seen or know what to do about it? Thanks, - Phil P.S. Some of the hardware on our cluster can be used only with cuda version 8. Does anyone know if specfem3d will work with cuda8? nvcc -c src/gpu/kernels.gen/crust_mantle_impl_kernel_adjoint.cu -o obj/crust_ma\ ntle_impl_kernel_adjoint.cuda-kernel.o --cudart=shared -I/apps/openmpi/1.7.5/\ include -DUSE_OLDER_CUDA4_GPU -gencode=arch=compute_20,code=\"sm_20,compute_20\\ " -x cu -I./setup -I./src/gpu/kernels.gen -DUSE_CUDA -include src/gpu/mesh_con\ stants_gpu.h DUSE_OLDER_CUDA4_GPU -gencode=arch=compute_20,code=\"sm_20,compute_20\" -o obj/\ cuda_device_obj.o obj/assemble_MPI_scalar_gpu.cuda.o obj/assemble_MPI_vector_gp\ u.cuda.o obj/check_fields_gpu.cuda.o obj/compute_add_sources_elastic_gpu.cuda.o\ obj/compute_coupling_gpu.cuda.o obj/compute_forces_crust_mantle_gpu.cuda.o obj\ /compute_forces_inner_core_gpu.cuda.o obj/compute_forces_outer_core_gpu.cuda.o \ obj/compute_kernels_gpu.cuda.o obj/compute_stacey_acoustic_gpu.cuda.o obj/compu\ te_stacey_elastic_gpu.cuda.o obj/compute_strain_gpu.cuda.o obj/helper_functions\ _gpu.cuda.o obj/initialize_gpu.cuda.o obj/noise_tomography_gpu.cuda.o obj/prepa\ re_mesh_constants_gpu.cuda.o obj/transfer_fields_gpu.cuda.o obj/update_displace\ ment_gpu.cuda.o obj/write_seismograms_gpu.cuda.o obj/save_and_compare_cpu_vs_gp\ u.cuda.o obj/assemble_boundary_accel_on_device.cuda-kernel.o obj/assemble_bound\ ary_potential_on_device.cuda-kernel.o obj/prepare_boundary_potential_on_device.\ cuda-kernel.o obj/prepare_boundary_accel_on_device.cuda-kernel.o obj/get_maximu\ m_scalar_kernel.cuda-kernel.o obj/get_maximum_vector_kernel.cuda-kernel.o obj/c\ ompute_add_sources_adjoint_kernel.cuda-kernel.o obj/compute_add_sources_kernel.\ cuda-kernel.o nel.cuda-kernel.o obj/write_seismograms_transfer_strain_from_device_kernel.cuda\ -kernel.o obj/noise_transfer_surface_to_host_kernel.cuda-kernel.o obj/noise_add\ _source_master_rec_kernel.cuda-kernel.o obj/noise_add_surface_movie_kernel.cuda\ -kernel.o obj/compute_stacey_acoustic_kernel.cuda-kernel.o obj/compute_stacey_a\ coustic_backward_kernel.cuda-kernel.o obj/compute_stacey_elastic_kernel.cuda-ke\ rnel.o obj/compute_stacey_elastic_backward_kernel.cuda-kernel.o obj/update_disp\ _veloc_kernel.cuda-kernel.o obj/update_potential_kernel.cuda-kernel.o obj/updat\ e_accel_elastic_kernel.cuda-kernel.o obj/update_veloc_elastic_kernel.cuda-kerne\ l.o obj/update_accel_acoustic_kernel.cuda-kernel.o obj/update_veloc_acoustic_ke\ rnel.cuda-kernel.o obj/compute_rho_kernel.cuda-kernel.o obj/compute_iso_kernel.\ cuda-kernel.o obj/compute_ani_kernel.cuda-kernel.o obj/compute_hess_kernel.cuda\ -kernel.o obj/compute_acoustic_kernel.cuda-kernel.o obj/compute_strength_noise_\ kernel.cuda-kernel.o obj/compute_ani_undoatt_kernel.cuda-kernel.o obj/compute_i\ so_undoatt_kernel.cuda-kernel.o obj/compute_strain_kernel.cuda-kernel.o obj/out\ er_core_impl_kernel_forward.cuda-kernel.o obj/outer_core_impl_kernel_adjoint.cu\ da-kernel.o obj/inner_core_impl_kernel_forward.cuda-kernel.o obj/inner_core_imp\ l_kernel_adjoint.cuda-kernel.o obj/crust_mantle_impl_kernel_forward.cuda-kernel\ .o obj/crust_mantle_impl_kernel_adjoint.cuda-kernel.o make: DUSE_OLDER_CUDA4_GPU: Command not found make: [obj/cuda_device_obj.o] Error 127 (ignored) _______________________________________________ CIG-SEISMO mailing list CIG-SEISMO at geodynamics.org http://lists.geodynamics.org/cgi-bin/mailman/listinfo/cig-seismo -- Phil Cummins Prof. Natural Hazards Research School of Earth Sciences Australian National University _______________________________________________ CIG-SEISMO mailing list CIG-SEISMO at geodynamics.org http://lists.geodynamics.org/cgi-bin/mailman/listinfo/cig-seismo -- Piero Lanucara CINECA SuperComputing Applications and Innovation Department - SCAI Phone: +39 06 44486709 Fax: +39 06 4957083 Address: CINECA Sede di Roma Via dei Tizii,6/b - 00185 Roma - Italy E-mail: p.lanucara at cineca.it skype: lpeter175 site: http://www.hpc.cineca.it _______________________________________________ CIG-SEISMO mailing list CIG-SEISMO at geodynamics.org http://lists.geodynamics.org/cgi-bin/mailman/listinfo/cig-seismo ________________________________ This message and its contents including attachments are intended solely for the original recipient. If you are not the intended recipient or have received this message in error, please notify me immediately and delete this message from your computer system. Any unauthorized use or distribution is prohibited. Please consider the environment before printing this email. -------------- next part -------------- An HTML attachment was scrubbed... URL: From baagaard at usgs.gov Wed Apr 26 17:18:05 2017 From: baagaard at usgs.gov (Brad Aagaard) Date: Wed, 26 Apr 2017 17:18:05 -0700 Subject: [CIG-SEISMO] specfem3d_globe with CUDA In-Reply-To: References: <58FC0735.3030308@anu.edu.au> <58FFF24C.3050602@anu.edu.au> <59001006.5050309@anu.edu.au> <57db055a-4a50-0749-4f6c-4873f01eb052@lma.cnrs-mrs.fr> <5900A8E2.7040809@anu.edu.au> <63a9014d-80c0-5330-c826-0685cb4a1194@cineca.it> Message-ID: <0ff51484-d337-64fa-c129-b7083cab711a@usgs.gov> On 04/26/2017 03:10 PM, Daniel B. Peter wrote: > Hi Phil, hi all, > > thanks for pointing out that the default —with-cuda=cuda which leads to > CUDA4 compilation is probably obsolete by now and we should change this > default. i’m going to make the changes and fix to the configuration script. > > in your case however, i’m not sure why the initial > ./configure —with-cuda=cuda6 > didn’t work. > > first, what are the GPU cards you want to run it on? then choose: > -for Tesla K20: ./configure —with-cuda=cuda5 > -for Tesla K80: ./configure —with-cuda=cuda6 > -for Maxwell K2200: ./configure —with-cuda=cuda7 > -for Pascal P100: ./configure —with-cuda=cuda8 > > we use the cuda*** specification as a short flag to point to the > specific architecture and optimization. even if you have the cudatoolkit > version 8, but want to compile kernels for a Tesla K20 card, you would > choose —with-cuda=cuda5 > > best wishes, > daniel If --with-cuda=cudaX is tied to a card rather than the CUDA version, why not be explicit and use --with-cuda=K20 --with-cuda=K80 etc. This would make it much more transparent to the user. Regards, Brad From phil.cummins at anu.edu.au Wed Apr 26 19:25:31 2017 From: phil.cummins at anu.edu.au (Phil Cummins) Date: Thu, 27 Apr 2017 12:25:31 +1000 Subject: [CIG-SEISMO] specfem3d_globe with CUDA In-Reply-To: <0ff51484-d337-64fa-c129-b7083cab711a@usgs.gov> References: <58FC0735.3030308@anu.edu.au> <58FFF24C.3050602@anu.edu.au> <59001006.5050309@anu.edu.au> <57db055a-4a50-0749-4f6c-4873f01eb052@lma.cnrs-mrs.fr> <5900A8E2.7040809@anu.edu.au> <63a9014d-80c0-5330-c826-0685cb4a1194@cineca.it> <0ff51484-d337-64fa-c129-b7083cab711a@usgs.gov> Message-ID: <5901569B.9070200@anu.edu.au> Hi again, I agree with Brad. I had assumed the number following cuda was for the version of the CUDA software, and I used cuda6 because it was the highest number I had thought was available (our sysadmin told me I need to compile with CUDA version 8.0). I have now used the cuda8 option because that is the one for my card, but I had exactly the same behavior with "make all" - I still had to remove cuda_dev_obj.o and add -lstdc++ to the final mpif90 link command. (still trying to run some of the examples ...) Regards, - Phil > Brad Aagaard > 27 April 2017 at 10:18 AM > > > If --with-cuda=cudaX is tied to a card rather than the CUDA version, > why not be explicit and use > > --with-cuda=K20 > --with-cuda=K80 > etc. > > This would make it much more transparent to the user. > > Regards, > Brad > > _______________________________________________ > CIG-SEISMO mailing list > CIG-SEISMO at geodynamics.org > http://lists.geodynamics.org/cgi-bin/mailman/listinfo/cig-seismo > Daniel B. Peter > 27 April 2017 at 8:10 AM > Hi Phil, hi all, > > thanks for pointing out that the default —with-cuda=cuda which leads > to CUDA4 compilation is probably obsolete by now and we should change > this default. i’m going to make the changes and fix to the > configuration script. > > in your case however, i’m not sure why the initial > ./configure —with-cuda=cuda6 > didn’t work. > > first, what are the GPU cards you want to run it on? then choose: > -for Tesla K20: ./configure —with-cuda=cuda5 > -for Tesla K80: ./configure —with-cuda=cuda6 > -for Maxwell K2200: ./configure —with-cuda=cuda7 > -for Pascal P100: ./configure —with-cuda=cuda8 > > we use the cuda*** specification as a short flag to point to the > specific architecture and optimization. even if you have the > cudatoolkit version 8, but want to compile kernels for a Tesla K20 > card, you would choose —with-cuda=cuda5 > > best wishes, > daniel > > > > > > > > > ------------------------------------------------------------------------ > This message and its contents including attachments are intended > solely for the original recipient. If you are not the intended > recipient or have received this message in error, please notify me > immediately and delete this message from your computer system. Any > unauthorized use or distribution is prohibited. Please consider the > environment before printing this email. > _______________________________________________ > CIG-SEISMO mailing list > CIG-SEISMO at geodynamics.org > http://lists.geodynamics.org/cgi-bin/mailman/listinfo/cig-seismo > Piero Lanucara > 27 April 2017 at 12:26 AM > > Hi all > > as far as I understand it should work with: > > configure.... --with-cuda=cuda5 > > regards > > Piero > > > Il 26/04/2017 16:04, Phil Cummins ha scritto: > > -- > Piero Lanucara > CINECA > SuperComputing Applications and Innovation Department - SCAI > Phone: +39 06 44486709 > Fax: +39 06 4957083 > Address: CINECA Sede di Roma Via dei Tizii,6/b - 00185 Roma - Italy > E-mail: p.lanucara at cineca.it > skype: lpeter175 > site: http://www.hpc.cineca.it > _______________________________________________ > CIG-SEISMO mailing list > CIG-SEISMO at geodynamics.org > http://lists.geodynamics.org/cgi-bin/mailman/listinfo/cig-seismo > Phil Cummins > 27 April 2017 at 12:04 AM > Hi Dimitri, > > Thanks for the kind words, but my approach was just a workaround. I > think someone smarter than me should try to fix the Makefile or > configure script. I'm still not sure what was wrong with it. > Good luck! > > - Phil > > > > _______________________________________________ > CIG-SEISMO mailing list > CIG-SEISMO at geodynamics.org > http://lists.geodynamics.org/cgi-bin/mailman/listinfo/cig-seismo > Dimitri Komatitsch > 26 April 2017 at 8:54 PM > > Hi Phil, Hi all, > > Thanks! Let me cc Daniel Peter and Peter Messmer, who are very > familiar with the CUDA version. Daniel, would you know how to commit > Phil's fix to the official Git version of the code? (and I guess also > for SPECFEM3D_Cartesian and for SPECFEM2D?). > > Thanks, > Best wishes, > > Dimitri. > > > -- Phil Cummins Prof. Natural Hazards Research School of Earth Sciences Australian National University -------------- next part -------------- An HTML attachment was scrubbed... URL: From p.lanucara at cineca.it Thu Apr 27 01:12:24 2017 From: p.lanucara at cineca.it (Piero Lanucara) Date: Thu, 27 Apr 2017 10:12:24 +0200 Subject: [CIG-SEISMO] specfem3d_globe with CUDA In-Reply-To: References: <58FC0735.3030308@anu.edu.au> <58FFF24C.3050602@anu.edu.au> <59001006.5050309@anu.edu.au> <57db055a-4a50-0749-4f6c-4873f01eb052@lma.cnrs-mrs.fr> <5900A8E2.7040809@anu.edu.au> <63a9014d-80c0-5330-c826-0685cb4a1194@cineca.it> Message-ID: <619c8dd2-f12e-0984-4856-57cd5907f746@cineca.it> Hi Peter I agree but, in principle at least, it should the configure able to point to the system recovered CUDA libs and runtime? so, something like --with-cuda should work fine and should point to the correct version of CUDA runtime? Thanks Piero Il 27/04/2017 00:10, Daniel B. Peter ha scritto: > Hi Phil, hi all, > > thanks for pointing out that the default —with-cuda=cuda which leads > to CUDA4 compilation is probably obsolete by now and we should change > this default. i’m going to make the changes and fix to the > configuration script. > > in your case however, i’m not sure why the initial > ./configure —with-cuda=cuda6 > didn’t work. > > first, what are the GPU cards you want to run it on? then choose: > -for Tesla K20: ./configure —with-cuda=cuda5 > -for Tesla K80: ./configure —with-cuda=cuda6 > -for Maxwell K2200: ./configure —with-cuda=cuda7 > -for Pascal P100: ./configure —with-cuda=cuda8 > > we use the cuda*** specification as a short flag to point to the > specific architecture and optimization. even if you have the > cudatoolkit version 8, but want to compile kernels for a Tesla K20 > card, you would choose —with-cuda=cuda5 > > best wishes, > daniel > > > > > > >> On Apr 26, 2017, at 5:26 PM, Piero Lanucara > > wrote: >> >> Hi all >> >> as far as I understand it should work with: >> >> configure.... --with-cuda=cuda5 >> >> regards >> >> Piero >> >> >> Il 26/04/2017 16:04, Phil Cummins ha scritto: >>> Hi Dimitri, >>> >>> Thanks for the kind words, but my approach was just a workaround. I >>> think someone smarter than me should try to fix the Makefile or >>> configure script. I'm still not sure what was wrong with it. >>> Good luck! >>> >>> - Phil >>> >>> >>>> Dimitri Komatitsch >>>> 26 April 2017 at 8:54 PM >>>> >>>> Hi Phil, Hi all, >>>> >>>> Thanks! Let me cc Daniel Peter and Peter Messmer, who are very >>>> familiar with the CUDA version. Daniel, would you know how to >>>> commit Phil's fix to the official Git version of the code? (and I >>>> guess also for SPECFEM3D_Cartesian and for SPECFEM2D?). >>>> >>>> Thanks, >>>> Best wishes, >>>> >>>> Dimitri. >>>> >>>> >>>> >>>> Phil Cummins >>>> 23 April 2017 at 11:45 AM >>>> Hello, >>>> >>>> I'm would like to use specfem3d_globe with my university's GPU >>>> cluster. I am trying to compile specfem3d_globe-7.0.0 after having >>>> configured it using: >>>> ./configure --with-cuda=cuda6 >>>> I am using the ifort fortran compiler version 12.1.9.293, and the >>>> version of cuda I have loaded is 6.5. >>>> The "make all" command does a lot of compilation using ifort and >>>> nvcc, but eventually fails when trying to link xspecfem3D itself >>>> with the error: >>>> >>>> ifort: error #10236: File not found: './obj/cuda_device_obj.o' >>>> make: *** [bin/xspecfem3D] Error 1 >>>> >>>> Delving into what happened during the make (see below), I noticed >>>> that after using nvcc to compile >>>> crust_mantle_impl_kernel_adjoint.cu, it attempts to use the command >>>> DUSE_OLDER_CUDA4_GPU to compile cuda_device_obj, which of course >>>> doesn't work because DUSE_OLDER_CUDA4_GPU is a compiler option not >>>> a command. It seems that make has somehow lost the beginning of >>>> this particular compilation command. >>>> Has anyone else seen or know what to do about it? >>>> Thanks, >>>> >>>> - Phil >>>> >>>> P.S. Some of the hardware on our cluster can be used only with cuda >>>> version 8. Does anyone know if specfem3d will work with cuda8? >>>> >>>> nvcc -c src/gpu/kernels.gen/crust_mantle_impl_kernel_adjoint.cu -o >>>> obj/crust_ma\ >>>> ntle_impl_kernel_adjoint.cuda-kernel.o --cudart=shared >>>> -I/apps/openmpi/1.7.5/\ >>>> include -DUSE_OLDER_CUDA4_GPU >>>> -gencode=arch=compute_20,code=\"sm_20,compute_20\\ >>>> " -x cu -I./setup -I./src/gpu/kernels.gen -DUSE_CUDA -include >>>> src/gpu/mesh_con\ >>>> stants_gpu.h >>>> DUSE_OLDER_CUDA4_GPU >>>> -gencode=arch=compute_20,code=\"sm_20,compute_20\" -o obj/\ >>>> cuda_device_obj.o obj/assemble_MPI_scalar_gpu.cuda.o >>>> obj/assemble_MPI_vector_gp\ >>>> u.cuda.o obj/check_fields_gpu.cuda.o >>>> obj/compute_add_sources_elastic_gpu.cuda.o\ >>>> obj/compute_coupling_gpu.cuda.o >>>> obj/compute_forces_crust_mantle_gpu.cuda.o obj\ >>>> /compute_forces_inner_core_gpu.cuda.o >>>> obj/compute_forces_outer_core_gpu.cuda.o \ >>>> obj/compute_kernels_gpu.cuda.o >>>> obj/compute_stacey_acoustic_gpu.cuda.o obj/compu\ >>>> te_stacey_elastic_gpu.cuda.o obj/compute_strain_gpu.cuda.o >>>> obj/helper_functions\ >>>> _gpu.cuda.o obj/initialize_gpu.cuda.o >>>> obj/noise_tomography_gpu.cuda.o obj/prepa\ >>>> re_mesh_constants_gpu.cuda.o obj/transfer_fields_gpu.cuda.o >>>> obj/update_displace\ >>>> ment_gpu.cuda.o obj/write_seismograms_gpu.cuda.o >>>> obj/save_and_compare_cpu_vs_gp\ >>>> u.cuda.o obj/assemble_boundary_accel_on_device.cuda-kernel.o >>>> obj/assemble_bound\ >>>> ary_potential_on_device.cuda-kernel.o >>>> obj/prepare_boundary_potential_on_device.\ >>>> cuda-kernel.o obj/prepare_boundary_accel_on_device.cuda-kernel.o >>>> obj/get_maximu\ >>>> m_scalar_kernel.cuda-kernel.o >>>> obj/get_maximum_vector_kernel.cuda-kernel.o obj/c\ >>>> ompute_add_sources_adjoint_kernel.cuda-kernel.o >>>> obj/compute_add_sources_kernel.\ >>>> cuda-kernel.o >>>> nel.cuda-kernel.o >>>> obj/write_seismograms_transfer_strain_from_device_kernel.cuda\ >>>> -kernel.o obj/noise_transfer_surface_to_host_kernel.cuda-kernel.o >>>> obj/noise_add\ >>>> _source_master_rec_kernel.cuda-kernel.o >>>> obj/noise_add_surface_movie_kernel.cuda\ >>>> -kernel.o obj/compute_stacey_acoustic_kernel.cuda-kernel.o >>>> obj/compute_stacey_a\ >>>> coustic_backward_kernel.cuda-kernel.o >>>> obj/compute_stacey_elastic_kernel.cuda-ke\ >>>> rnel.o obj/compute_stacey_elastic_backward_kernel.cuda-kernel.o >>>> obj/update_disp\ >>>> _veloc_kernel.cuda-kernel.o >>>> obj/update_potential_kernel.cuda-kernel.o obj/updat\ >>>> e_accel_elastic_kernel.cuda-kernel.o >>>> obj/update_veloc_elastic_kernel.cuda-kerne\ >>>> l.o obj/update_accel_acoustic_kernel.cuda-kernel.o >>>> obj/update_veloc_acoustic_ke\ >>>> rnel.cuda-kernel.o obj/compute_rho_kernel.cuda-kernel.o >>>> obj/compute_iso_kernel.\ >>>> cuda-kernel.o obj/compute_ani_kernel.cuda-kernel.o >>>> obj/compute_hess_kernel.cuda\ >>>> -kernel.o obj/compute_acoustic_kernel.cuda-kernel.o >>>> obj/compute_strength_noise_\ >>>> kernel.cuda-kernel.o obj/compute_ani_undoatt_kernel.cuda-kernel.o >>>> obj/compute_i\ >>>> so_undoatt_kernel.cuda-kernel.o >>>> obj/compute_strain_kernel.cuda-kernel.o obj/out\ >>>> er_core_impl_kernel_forward.cuda-kernel.o >>>> obj/outer_core_impl_kernel_adjoint.cu\ >>>> da-kernel.o obj/inner_core_impl_kernel_forward.cuda-kernel.o >>>> obj/inner_core_imp\ >>>> l_kernel_adjoint.cuda-kernel.o >>>> obj/crust_mantle_impl_kernel_forward.cuda-kernel\ >>>> .o obj/crust_mantle_impl_kernel_adjoint.cuda-kernel.o >>>> make: DUSE_OLDER_CUDA4_GPU: Command not found >>>> make: [obj/cuda_device_obj.o] Error 127 (ignored) >>>> >>>> _______________________________________________ >>>> CIG-SEISMO mailing list >>>> CIG-SEISMO at geodynamics.org >>>> http://lists.geodynamics.org/cgi-bin/mailman/listinfo/cig-seismo >>> >>> -- >>> Phil Cummins >>> Prof. Natural Hazards >>> Research School of Earth Sciences >>> Australian National University >>> >>> >>> _______________________________________________ >>> CIG-SEISMO mailing list >>> CIG-SEISMO at geodynamics.org >>> http://lists.geodynamics.org/cgi-bin/mailman/listinfo/cig-seismo >> >> -- >> Piero Lanucara >> CINECA >> SuperComputing Applications and Innovation Department - SCAI >> Phone: +39 06 44486709 >> Fax: +39 06 4957083 >> Address: CINECA Sede di Roma Via dei Tizii,6/b - 00185 Roma - Italy >> E-mail:p.lanucara at cineca.it >> skype: lpeter175 >> site:http://www.hpc.cineca.it >> _______________________________________________ >> CIG-SEISMO mailing list >> CIG-SEISMO at geodynamics.org >> http://lists.geodynamics.org/cgi-bin/mailman/listinfo/cig-seismo > > > ------------------------------------------------------------------------ > This message and its contents including attachments are intended > solely for the original recipient. If you are not the intended > recipient or have received this message in error, please notify me > immediately and delete this message from your computer system. Any > unauthorized use or distribution is prohibited. Please consider the > environment before printing this email. > > > _______________________________________________ > CIG-SEISMO mailing list > CIG-SEISMO at geodynamics.org > http://lists.geodynamics.org/cgi-bin/mailman/listinfo/cig-seismo -- Piero Lanucara CINECA SuperComputing Applications and Innovation Department - SCAI Phone: +39 06 44486709 Fax: +39 06 4957083 Address: CINECA Sede di Roma Via dei Tizii,6/b - 00185 Roma - Italy E-mail: p.lanucara at cineca.it skype: lpeter175 site: http://www.hpc.cineca.it -------------- next part -------------- An HTML attachment was scrubbed... URL: From daniel.peter at kaust.edu.sa Thu Apr 27 02:27:10 2017 From: daniel.peter at kaust.edu.sa (Daniel B. Peter) Date: Thu, 27 Apr 2017 09:27:10 +0000 Subject: [CIG-SEISMO] specfem3d_globe with CUDA In-Reply-To: <619c8dd2-f12e-0984-4856-57cd5907f746@cineca.it> References: <58FC0735.3030308@anu.edu.au> <58FFF24C.3050602@anu.edu.au> <59001006.5050309@anu.edu.au> <57db055a-4a50-0749-4f6c-4873f01eb052@lma.cnrs-mrs.fr> <5900A8E2.7040809@anu.edu.au> <63a9014d-80c0-5330-c826-0685cb4a1194@cineca.it> , <619c8dd2-f12e-0984-4856-57cd5907f746@cineca.it> Message-ID: <4DDF31B6-D0B0-44D9-8095-90E2300B1C04@kaust.edu.sa> Phil, could you attach the config.log and original Makefile? for some reason the cuda option is not properly recognized in your configuration step. @Brad: for the funny cuda flagging scheme, it is mainly due to historic reasons. we had to distinguish between cuda versions 4 or older (which was default at the time) and the newer version 5 and higher. so we introduced the cuda5 flag option. then came newer hardware and with each new hardware architecture a new cuda version appeared. so basically each major cuda version introduced a new type of hardware. i assume that a user doesn't know the architecture number for his card (do you know what the sm_** code is for your P100 card?), but the cudatoolkit version he needs for the compilation (you will need cuda version 8 for the new pascal architecture). also, having a separate flag for each card is not convenient, as the optimization depends on the architecture. the cuda flag option thus gives you a brief history of GPU time. we could change it and use the architecture number, maybe. anyway, if you need a specific cuda flag, you can always modify the Makefile directly. best, daniel On Apr 27, 2017, at 10:12, Piero Lanucara > wrote: Hi Peter I agree but, in principle at least, it should the configure able to point to the system recovered CUDA libs and runtime? so, something like --with-cuda should work fine and should point to the correct version of CUDA runtime? Thanks Piero Il 27/04/2017 00:10, Daniel B. Peter ha scritto: Hi Phil, hi all, thanks for pointing out that the default —with-cuda=cuda which leads to CUDA4 compilation is probably obsolete by now and we should change this default. i’m going to make the changes and fix to the configuration script. in your case however, i’m not sure why the initial ./configure —with-cuda=cuda6 didn’t work. first, what are the GPU cards you want to run it on? then choose: -for Tesla K20: ./configure —with-cuda=cuda5 -for Tesla K80: ./configure —with-cuda=cuda6 -for Maxwell K2200: ./configure —with-cuda=cuda7 -for Pascal P100: ./configure —with-cuda=cuda8 we use the cuda*** specification as a short flag to point to the specific architecture and optimization. even if you have the cudatoolkit version 8, but want to compile kernels for a Tesla K20 card, you would choose —with-cuda=cuda5 best wishes, daniel On Apr 26, 2017, at 5:26 PM, Piero Lanucara > wrote: Hi all as far as I understand it should work with: configure.... --with-cuda=cuda5 regards Piero Il 26/04/2017 16:04, Phil Cummins ha scritto: Hi Dimitri, Thanks for the kind words, but my approach was just a workaround. I think someone smarter than me should try to fix the Makefile or configure script. I'm still not sure what was wrong with it. Good luck! - Phil Dimitri Komatitsch 26 April 2017 at 8:54 PM Hi Phil, Hi all, Thanks! Let me cc Daniel Peter and Peter Messmer, who are very familiar with the CUDA version. Daniel, would you know how to commit Phil's fix to the official Git version of the code? (and I guess also for SPECFEM3D_Cartesian and for SPECFEM2D?). Thanks, Best wishes, Dimitri. Phil Cummins 23 April 2017 at 11:45 AM Hello, I'm would like to use specfem3d_globe with my university's GPU cluster. I am trying to compile specfem3d_globe-7.0.0 after having configured it using: ./configure --with-cuda=cuda6 I am using the ifort fortran compiler version 12.1.9.293, and the version of cuda I have loaded is 6.5. The "make all" command does a lot of compilation using ifort and nvcc, but eventually fails when trying to link xspecfem3D itself with the error: ifort: error #10236: File not found: './obj/cuda_device_obj.o' make: *** [bin/xspecfem3D] Error 1 Delving into what happened during the make (see below), I noticed that after using nvcc to compile crust_mantle_impl_kernel_adjoint.cu, it attempts to use the command DUSE_OLDER_CUDA4_GPU to compile cuda_device_obj, which of course doesn't work because DUSE_OLDER_CUDA4_GPU is a compiler option not a command. It seems that make has somehow lost the beginning of this particular compilation command. Has anyone else seen or know what to do about it? Thanks, - Phil P.S. Some of the hardware on our cluster can be used only with cuda version 8. Does anyone know if specfem3d will work with cuda8? nvcc -c src/gpu/kernels.gen/crust_mantle_impl_kernel_adjoint.cu -o obj/crust_ma\ ntle_impl_kernel_adjoint.cuda-kernel.o --cudart=shared -I/apps/openmpi/1.7.5/\ include -DUSE_OLDER_CUDA4_GPU -gencode=arch=compute_20,code=\"sm_20,compute_20\\ " -x cu -I./setup -I./src/gpu/kernels.gen -DUSE_CUDA -include src/gpu/mesh_con\ stants_gpu.h DUSE_OLDER_CUDA4_GPU -gencode=arch=compute_20,code=\"sm_20,compute_20\" -o obj/\ cuda_device_obj.o obj/assemble_MPI_scalar_gpu.cuda.o obj/assemble_MPI_vector_gp\ u.cuda.o obj/check_fields_gpu.cuda.o obj/compute_add_sources_elastic_gpu.cuda.o\ obj/compute_coupling_gpu.cuda.o obj/compute_forces_crust_mantle_gpu.cuda.o obj\ /compute_forces_inner_core_gpu.cuda.o obj/compute_forces_outer_core_gpu.cuda.o \ obj/compute_kernels_gpu.cuda.o obj/compute_stacey_acoustic_gpu.cuda.o obj/compu\ te_stacey_elastic_gpu.cuda.o obj/compute_strain_gpu.cuda.o obj/helper_functions\ _gpu.cuda.o obj/initialize_gpu.cuda.o obj/noise_tomography_gpu.cuda.o obj/prepa\ re_mesh_constants_gpu.cuda.o obj/transfer_fields_gpu.cuda.o obj/update_displace\ ment_gpu.cuda.o obj/write_seismograms_gpu.cuda.o obj/save_and_compare_cpu_vs_gp\ u.cuda.o obj/assemble_boundary_accel_on_device.cuda-kernel.o obj/assemble_bound\ ary_potential_on_device.cuda-kernel.o obj/prepare_boundary_potential_on_device.\ cuda-kernel.o obj/prepare_boundary_accel_on_device.cuda-kernel.o obj/get_maximu\ m_scalar_kernel.cuda-kernel.o obj/get_maximum_vector_kernel.cuda-kernel.o obj/c\ ompute_add_sources_adjoint_kernel.cuda-kernel.o obj/compute_add_sources_kernel.\ cuda-kernel.o nel.cuda-kernel.o obj/write_seismograms_transfer_strain_from_device_kernel.cuda\ -kernel.o obj/noise_transfer_surface_to_host_kernel.cuda-kernel.o obj/noise_add\ _source_master_rec_kernel.cuda-kernel.o obj/noise_add_surface_movie_kernel.cuda\ -kernel.o obj/compute_stacey_acoustic_kernel.cuda-kernel.o obj/compute_stacey_a\ coustic_backward_kernel.cuda-kernel.o obj/compute_stacey_elastic_kernel.cuda-ke\ rnel.o obj/compute_stacey_elastic_backward_kernel.cuda-kernel.o obj/update_disp\ _veloc_kernel.cuda-kernel.o obj/update_potential_kernel.cuda-kernel.o obj/updat\ e_accel_elastic_kernel.cuda-kernel.o obj/update_veloc_elastic_kernel.cuda-kerne\ l.o obj/update_accel_acoustic_kernel.cuda-kernel.o obj/update_veloc_acoustic_ke\ rnel.cuda-kernel.o obj/compute_rho_kernel.cuda-kernel.o obj/compute_iso_kernel.\ cuda-kernel.o obj/compute_ani_kernel.cuda-kernel.o obj/compute_hess_kernel.cuda\ -kernel.o obj/compute_acoustic_kernel.cuda-kernel.o obj/compute_strength_noise_\ kernel.cuda-kernel.o obj/compute_ani_undoatt_kernel.cuda-kernel.o obj/compute_i\ so_undoatt_kernel.cuda-kernel.o obj/compute_strain_kernel.cuda-kernel.o obj/out\ er_core_impl_kernel_forward.cuda-kernel.o obj/outer_core_impl_kernel_adjoint.cu\ da-kernel.o obj/inner_core_impl_kernel_forward.cuda-kernel.o obj/inner_core_imp\ l_kernel_adjoint.cuda-kernel.o obj/crust_mantle_impl_kernel_forward.cuda-kernel\ .o obj/crust_mantle_impl_kernel_adjoint.cuda-kernel.o make: DUSE_OLDER_CUDA4_GPU: Command not found make: [obj/cuda_device_obj.o] Error 127 (ignored) _______________________________________________ CIG-SEISMO mailing list CIG-SEISMO at geodynamics.org http://lists.geodynamics.org/cgi-bin/mailman/listinfo/cig-seismo -- Phil Cummins Prof. Natural Hazards Research School of Earth Sciences Australian National University _______________________________________________ CIG-SEISMO mailing list CIG-SEISMO at geodynamics.org http://lists.geodynamics.org/cgi-bin/mailman/listinfo/cig-seismo -- Piero Lanucara CINECA SuperComputing Applications and Innovation Department - SCAI Phone: +39 06 44486709 Fax: +39 06 4957083 Address: CINECA Sede di Roma Via dei Tizii,6/b - 00185 Roma - Italy E-mail: p.lanucara at cineca.it skype: lpeter175 site: http://www.hpc.cineca.it _______________________________________________ CIG-SEISMO mailing list CIG-SEISMO at geodynamics.org http://lists.geodynamics.org/cgi-bin/mailman/listinfo/cig-seismo ________________________________ This message and its contents including attachments are intended solely for the original recipient. If you are not the intended recipient or have received this message in error, please notify me immediately and delete this message from your computer system. Any unauthorized use or distribution is prohibited. Please consider the environment before printing this email. _______________________________________________ CIG-SEISMO mailing list CIG-SEISMO at geodynamics.org http://lists.geodynamics.org/cgi-bin/mailman/listinfo/cig-seismo -- Piero Lanucara CINECA SuperComputing Applications and Innovation Department - SCAI Phone: +39 06 44486709 Fax: +39 06 4957083 Address: CINECA Sede di Roma Via dei Tizii,6/b - 00185 Roma - Italy E-mail: p.lanucara at cineca.it skype: lpeter175 site: http://www.hpc.cineca.it _______________________________________________ CIG-SEISMO mailing list CIG-SEISMO at geodynamics.org http://lists.geodynamics.org/cgi-bin/mailman/listinfo/cig-seismo -------------- next part -------------- An HTML attachment was scrubbed... URL: From jimma.tamirat at aau.edu.et Sun Apr 30 07:44:42 2017 From: jimma.tamirat at aau.edu.et (Tamirat Bekele) Date: Sun, 30 Apr 2017 17:44:42 +0300 Subject: [CIG-SEISMO] =?utf-8?q?=28no_subject=29?= Message-ID: Dear All, I am new to Specfem3d and working hard to be proficient. Since CUBIT is a commercial package, I prefer to use Gmsh to create mesh. After creating mesh I tried to convert that mesh to Specfem format using gmsh2specfem3d.py script provided with Specfem3d package. I have encountered series of problem, the one which I couldn't be able to solve is the following: Traceback (most recent call last): File "gmsh2specfem3d.py", line 395, in OuvreGmsh('',Fic); File "gmsh2specfem3d.py", line 103, in OuvreGmsh NbPhysNames = int(string.split(lignes[PosPhys+1])[0]) UnboundLocalError: local variable 'PosPhys' referenced before assignment After googling I declared PosPhys as global variable in 'OuvreGmsh' function and it worked, but the following error happend: Traceback (most recent call last): File "gmsh2specfem3d.py", line 394, in OuvreGmsh('',Fic); File "gmsh2specfem3d.py", line 102, in OuvreGmsh NbPhysNames = int(string.split(lignes[PosPhys+1])[0]) ValueError: invalid literal for int() with base 10: '2.2' Please could you kindly help me on that. Thanks! With regards, Tamirat -------------- next part -------------- An HTML attachment was scrubbed... URL: From jimma.tamirat at aau.edu.et Sun Apr 30 07:45:44 2017 From: jimma.tamirat at aau.edu.et (Tamirat Bekele) Date: Sun, 30 Apr 2017 17:45:44 +0300 Subject: [CIG-SEISMO] gmsh2specfem3d.py error Message-ID: Dear All, I am new to Specfem3d and working hard to be proficient. Since CUBIT is a commercial package, I prefer to use Gmsh to create mesh. After creating mesh I tried to convert that mesh to Specfem format using gmsh2specfem3d.py script provided with Specfem3d package. I have encountered series of problem, the one which I couldn't be able to solve is the following: Traceback (most recent call last): File "gmsh2specfem3d.py", line 395, in OuvreGmsh('',Fic); File "gmsh2specfem3d.py", line 103, in OuvreGmsh NbPhysNames = int(string.split(lignes[PosPhys+1])[0]) UnboundLocalError: local variable 'PosPhys' referenced before assignment After googling I declared PosPhys as global variable in 'OuvreGmsh' function and it worked, but the following error happend: Traceback (most recent call last): File "gmsh2specfem3d.py", line 394, in OuvreGmsh('',Fic); File "gmsh2specfem3d.py", line 102, in OuvreGmsh NbPhysNames = int(string.split(lignes[PosPhys+1])[0]) ValueError: invalid literal for int() with base 10: '2.2' Please could you kindly help me on that. Thanks! With regards, Tamirat -------------- next part -------------- An HTML attachment was scrubbed... URL: