[CIG-SHORT] Rate and State and Initial Traction
Matthew Knepley
knepley at mcs.anl.gov
Thu Aug 30 14:19:45 PDT 2012
On Thu, Aug 30, 2012 at 4:11 PM, Romain Jolivet <rjolivet at caltech.edu>wrote:
> I did just run:
>
> pylith step01.cfg --petsc.fp_trap=0
>
> Is that what you meant? If it is, I still have the same problem :-) and it
> does not change what I see on the screen. Should it change something?\
>
Then it is not underflow.
> However, the norm computation problem does not appear when I set the
> zero_tolerance to 1.0 (it does not converge though, but I don't have this
> Floating point exception...)
>
As Brad says, change the relative tolerance, but leave the absolute
tolerance intact.
Matt
> Romain
>
>
> Romain Jolivet
> Postdoctoral Scholar
> Geological and Planetary Sciences
> California Institute of Technology, Pasadena, CA
> rjolivet at caltech.edu
> +1 (626) 560 6356
>
> On Aug 30, 2012, at 2:00 PM, Matthew Knepley wrote:
>
> On Thu, Aug 30, 2012 at 3:38 PM, Romain Jolivet <rjolivet at caltech.edu>wrote:
>
>> Thanks for the advice. I changed the preconditioner settings to what you
>> suggested. I don't see that jump in the linear norm anymore.
>> However, I still have the floating point exception problem. This arise
>> after the first round of ksp.
>>
>> In the log file (attached), I see to messages saying the ksp solver for
>> friction does converge (twice), then the SNES residual norm is printed,
>> then KSP starts, and when it has to compute the new SNES residual norm, it
>> says that VecNorm() encoutered a Inf or NAN value:
>>
>
> It could still be underflow. Try running with --petsc.fp_trap=0
>
> Thanks,
>
> Matt
>
>
>> 141 KSP Residual norm 1.185626397065e-09
>> 142 KSP Residual norm 8.558244459183e-10
>> 143 KSP Residual norm 4.537120666370e-10
>> 144 KSP Residual norm 2.695590025796e-10
>> 145 KSP Residual norm 2.263153111449e-10
>> 146 KSP Residual norm 2.156010427246e-10
>> Linear solve converged due to CONVERGED_RTOL iterations 146
>> [0]PETSC ERROR: --------------------- Error Message
>> ------------------------------------
>> [0]PETSC ERROR: Floating point exception!
>> [0]PETSC ERROR: Infinite or not-a-number generated in norm!
>> [0]PETSC ERROR:
>> ------------------------------------------------------------------------
>> [0]PETSC ERROR: Petsc Development HG revision:
>> 3bede28f1195e406d4257c9fbeebefe827793801 HG Date: Mon Jun 11 22:14:12 2012
>> -0400
>> [0]PETSC ERROR: See docs/changes/index.html for recent updates.
>> [0]PETSC ERROR: See docs/faq.html for hints about trouble shooting.
>> [0]PETSC ERROR: See docs/index.html for manual pages.
>> [0]PETSC ERROR:
>> ------------------------------------------------------------------------
>> [0]PETSC ERROR:
>> /Users/jolivetinsar/MYBIN/pylith/pylith-1.7.1-darwin-10.6.8/bin/mpinemesis
>> on a pylith-da named dhcp-74-229.caltech.edu by jolivetinsar Thu Aug 30
>> 13:34:55 2012
>> [0]PETSC ERROR: Libraries linked from
>> /Users/buildbot/install/pylith_darwin_10.6_binbot/lib
>> [0]PETSC ERROR: Configure run at Tue Jun 12 06:09:00 2012
>> [0]PETSC ERROR: Configure options
>> --prefix=/Users/buildbot/install/pylith_darwin_10.6_binbot
>> --download-f-blas-lapack=1 --with-sieve=1 --with-c2html=0
>> --CXXFLAGS=-DMPICH_IGNORE_CXX_SEEK --with-clanguage=C++
>> --with-mpicompilers=1 --with-debugging=0 --download-chaco=1
>> --download-boost=1 --download-ml=1 --with-hdf5=1
>> --with-hdf5-dir=/Users/buildbot/install/pylith_darwin_10.6_binbot
>> --with-x=0 --with-shared-libraries=1
>> [0]PETSC ERROR:
>> ------------------------------------------------------------------------
>> [0]PETSC ERROR: VecNorm() line 169 in src/vec/vec/interface/rvector.c
>> [0]PETSC ERROR: VecNormalize() line 263 in src/vec/vec/interface/rvector.c
>> [0]PETSC ERROR: KSPGMRESCycle() line 127 in
>> src/ksp/ksp/impls/gmres/gmres.c
>> [0]PETSC ERROR: KSPSolve_GMRES() line 231 in
>> src/ksp/ksp/impls/gmres/gmres.c
>> [0]PETSC ERROR: KSPSolve() line 446 in src/ksp/ksp/interface/itfunc.c
>> [0]PETSC ERROR: _sensitivitySolve() line 2172 in
>> "unknowndirectory/"faults/FaultCohesiveDyn.cc
>> Traceback (most recent call last):
>> File
>> "/Users/jolivetinsar/MYBIN/pylith/pylith-1.7.1-darwin-10.6.8/lib/python2.7/site-packages/pylith/apps/PetscApplication.py",
>> line 65, in onComputeNodes
>> Fatal error. Calling MPI_Abort() to abort PyLith application.
>> self.main(*args, **kwds)
>> File
>> "/Users/jolivetinsar/MYBIN/pylith/pylith-1.7.1-darwin-10.6.8/lib/python2.7/site-packages/pylith/apps/PyLithApp.py",
>> line 119, in main
>> self.problem.run(self)
>> File
>> "/Users/jolivetinsar/MYBIN/pylith/pylith-1.7.1-darwin-10.6.8/lib/python2.7/site-packages/pylith/problems/TimeDependent.py",
>> line 154, in run
>> self.formulation.step(t, dt)
>> File
>> "/Users/jolivetinsar/MYBIN/pylith/pylith-1.7.1-darwin-10.6.8/lib/python2.7/site-packages/pylith/problems/Implicit.py",
>> line 219, in step
>> self.solver.solve(dispIncr, self.jacobian, residual)
>> File
>> "/Users/jolivetinsar/MYBIN/pylith/pylith-1.7.1-darwin-10.6.8/lib/python2.7/site-packages/pylith/problems/problems.py",
>> line 180, in solve
>> def solve(self, *args): return _problems.SolverNonlinear_solve(self,
>> *args)
>> RuntimeError: std::exception
>> application called MPI_Abort(MPI_COMM_WORLD, -1) - process 0
>> /Users/jolivetinsar/MYBIN/pylith/pylith-1.7.1-darwin-10.6.8/bin/nemesis:
>> mpirun: exit 255
>> /Users/jolivetinsar/MYBIN/pylith/pylith-1.7.1-darwin-10.6.8/bin/pylith:
>> /Users/jolivetinsar/MYBIN/pylith/pylith-1.7.1-darwin-10.6.8/bin/nemesis:
>> exit 1
>>
>> Any idea?
>> Romain
>>
>>
>>
>> Romain Jolivet
>> Postdoctoral Scholar
>> Geological and Planetary Sciences
>> California Institute of Technology, Pasadena, CA
>> rjolivet at caltech.edu
>> +1 (626) 560 6356
>>
>> On Aug 30, 2012, at 12:00 PM, cig-short-request at geodynamics.org wrote:
>>
>> > Send CIG-SHORT mailing list submissions to
>> > cig-short at geodynamics.org
>> >
>> > To subscribe or unsubscribe via the World Wide Web, visit
>> > http://geodynamics.org/cgi-bin/mailman/listinfo/cig-short
>> > or, via email, send a message with subject or body 'help' to
>> > cig-short-request at geodynamics.org
>> >
>> > You can reach the person managing the list at
>> > cig-short-owner at geodynamics.org
>> >
>> > When replying, please edit your Subject line so it is more specific
>> > than "Re: Contents of CIG-SHORT digest..."
>> >
>> >
>> > Today's Topics:
>> >
>> > 1. Re: Rate and State and Initial Traction (Brad Aagaard)
>> > 2. Fwd: Rate and State and Initial Traction (Matthew Knepley)
>> >
>> >
>> > ----------------------------------------------------------------------
>> >
>> > Message: 1
>> > Date: Wed, 29 Aug 2012 12:36:08 -0700
>> > From: Brad Aagaard <baagaard at usgs.gov>
>> > Subject: Re: [CIG-SHORT] Rate and State and Initial Traction
>> > To: cig-short at geodynamics.org
>> > Message-ID: <503E6F28.6050001 at usgs.gov>
>> > Content-Type: text/plain; charset=ISO-8859-1; format=flowed
>> >
>> > Romain,
>> >
>> > It looks like you are using Schur complement preconditioner settings.
>> > The jump in the KSP residual at iteration 10 is odd and suggests
>> > something is not quite right in the linear solve (probably related to
>> > the preconditioner). Try using the AMG preconditioner. It may not be
>> > quite as fast but it is more robust. Matt can probably help you tune the
>> > Schur complement solver conditions some more if necessary.
>> >
>> > Your SNES tolerances are quite large. I suggest using tolerances similar
>> > to those in examples/3d/hex8/step14.cfg.
>> >
>> > SOLVER TOLERANCES
>> >
>> > ksp_rtol = 1.0e-16
>> > ksp_atol = 1.0e-12
>> >
>> > snes_rtol = 1.0e-14
>> > snes_atol = 1.0e-10
>> >
>> >
>> > AMG PRECONDITIONER SETTINGS
>> >
>> > [pylithapp.timedependent.formulation]
>> > split_fields = True
>> > matrix_type = aij
>> >
>> > [pylithapp.petsc]
>> > ksp_gmres_restart = 100
>> > fs_pc_type = fieldsplit
>> > fs_pc_fieldsplit_real_diagonal = true
>> > fs_pc_fieldsplit_type = multiplicative
>> > fs_fieldsplit_0_pc_type = ml
>> > fs_fieldsplit_1_pc_type = jacobi
>> > fs_fieldsplit_0_ksp_type = preonly
>> > fs_fieldsplit_1_ksp_type = preonly
>> >
>> > Brad
>> >
>> >
>> > On 08/29/2012 11:42 AM, Romain Jolivet wrote:
>> >> Hello Brad, Matt, Charles and the pylith team,
>> >>
>> >> Following your advice from the other day, I am experimenting how to
>> create earthquakes on a frictional interface by applying a traction change
>> history. It works fine with the SlipWeakening law in 2D. I'm trying to move
>> on to the RateStateAgeing behavior and I have problems applying the initial
>> traction conditions:
>> >>
>> >> I use a SimpleDB to apply the frictional properties on the fault: one
>> domain is rate-strengthening, the other is rate-weakening with a linear
>> transition between the two domains. If I do not try to turn on any
>> traction_perturbation option, the code runs great, and I have a slow slip
>> all the along the fault at any time.
>> >>
>> >> As I want the rate-weakening domain to be stuck sometimes, I want to
>> turn on traction_perturbation.db_initial. I use a SimpleDB to put a
>> negative traction all along the fault, and a shear traction compatible with
>> the friction coefficients I impose. However, when I run this, I have the
>> following error message from PETSc (I attach the log):
>> >>
>> >>>>
>> /Users/jolivetinsar/MYBIN/pylith/pylith-1.7.1-darwin-10.6.8/lib/python2.7/site-packages/pylith/problems/Implicit.py:215:step
>> >> -- implicit(info)
>> >> -- Solving equations.
>> >> 0 SNES Function norm 2.679589636063e+01
>> >> 0 KSP Residual norm 9.637565795900e-01
>> >> 1 KSP Residual norm 3.564937135471e-08
>> >> 2 KSP Residual norm 4.313963401911e-16
>> >> 3 KSP Residual norm 3.050427683646e-16
>> >> 4 KSP Residual norm 2.490663081181e-16
>> >> 5 KSP Residual norm 2.156976950523e-16
>> >> 6 KSP Residual norm 1.929258577738e-16
>> >> 7 KSP Residual norm 1.761163907914e-16
>> >> 8 KSP Residual norm 1.630520838230e-16
>> >> 9 KSP Residual norm 1.525212509706e-16
>> >> 10 KSP Residual norm 1.833325155789e-11
>> >> 11 KSP Residual norm 7.039784754868e-23
>> >> Linear solve converged due to CONVERGED_ATOL iterations 11
>> >> [0]PETSC ERROR: --------------------- Error Message
>> ------------------------------------
>> >> [0]PETSC ERROR: Floating point exception!
>> >> [0]PETSC ERROR: Infinite or not-a-number generated in norm!
>> >>
>> >> If anyone has an idea about that error message....
>> >>
>> >> I think it has to do with some slip vector. If I set
>> friction.min_slip_rate=1e-1, this problem does not happen (the nonlinear
>> solver does not converge, but at least, no crash). If I set the normal
>> traction to be positive, it does not crash as well (and it does not
>> converge as well). I join to this message the cfg file I am using for that
>> experiment.
>> >>
>> >> Thank you
>> >> Romain
>> >>
>> >>
>> >>
>> >>
>> >> Romain Jolivet
>> >> Postdoctoral Scholar
>> >> Geological and Planetary Sciences
>> >> California Institute of Technology, Pasadena, CA
>> >> rjolivet at caltech.edu
>> >> +1 (626) 560 6356
>> >>
>> >>
>> >>
>> >> _______________________________________________
>> >> CIG-SHORT mailing list
>> >> CIG-SHORT at geodynamics.org
>> >> http://geodynamics.org/cgi-bin/mailman/listinfo/cig-short
>> >>
>> >
>> >
>> >
>> > ------------------------------
>> >
>> > Message: 2
>> > Date: Wed, 29 Aug 2012 22:09:25 -0500
>> > From: Matthew Knepley <knepley at mcs.anl.gov>
>> > Subject: [CIG-SHORT] Fwd: Rate and State and Initial Traction
>> > To: cig-short at geodynamics.org
>> > Message-ID:
>> > <CAMYG4GkMcUqRjhMaYZ8Ooj-x=
>> 06QKFrc7fiwW1dStmDSDO1qWA at mail.gmail.com>
>> > Content-Type: text/plain; charset="iso-8859-1"
>> >
>> > On Wed, Aug 29, 2012 at 2:36 PM, Brad Aagaard <baagaard at usgs.gov>
>> wrote:
>> >
>> >> Romain,
>> >>
>> >> It looks like you are using Schur complement preconditioner settings.
>> >> The jump in the KSP residual at iteration 10 is odd and suggests
>> >> something is not quite right in the linear solve (probably related to
>> >>
>> >
>> > Brad is right that the tolerances here will produce some garbage. Going
>> > below
>> > machine epsilon (about 10^-16) means you get randomness and ultimately
>> > underflow which I think you got in a norm calculation since things get
>> > squared.
>> > The solver should have converged on iterate 2, and will with the
>> tolerances
>> > that
>> > Brad suggested.
>> >
>> > Thanks,
>> >
>> > Matt
>> >
>> >
>> >> the preconditioner). Try using the AMG preconditioner. It may not be
>> >> quite as fast but it is more robust. Matt can probably help you tune
>> the
>> >> Schur complement solver conditions some more if necessary.
>> >>
>> >> Your SNES tolerances are quite large. I suggest using tolerances
>> similar
>> >> to those in examples/3d/hex8/step14.cfg.
>> >>
>> >> SOLVER TOLERANCES
>> >>
>> >> ksp_rtol = 1.0e-16
>> >> ksp_atol = 1.0e-12
>> >>
>> >> snes_rtol = 1.0e-14
>> >> snes_atol = 1.0e-10
>> >>
>> >>
>> >> AMG PRECONDITIONER SETTINGS
>> >>
>> >> [pylithapp.timedependent.formulation]
>> >> split_fields = True
>> >> matrix_type = aij
>> >>
>> >> [pylithapp.petsc]
>> >> ksp_gmres_restart = 100
>> >> fs_pc_type = fieldsplit
>> >> fs_pc_fieldsplit_real_diagonal = true
>> >> fs_pc_fieldsplit_type = multiplicative
>> >> fs_fieldsplit_0_pc_type = ml
>> >> fs_fieldsplit_1_pc_type = jacobi
>> >> fs_fieldsplit_0_ksp_type = preonly
>> >> fs_fieldsplit_1_ksp_type = preonly
>> >>
>> >> Brad
>> >>
>> >>
>> >> On 08/29/2012 11:42 AM, Romain Jolivet wrote:
>> >>> Hello Brad, Matt, Charles and the pylith team,
>> >>>
>> >>> Following your advice from the other day, I am experimenting how to
>> >> create earthquakes on a frictional interface by applying a traction
>> change
>> >> history. It works fine with the SlipWeakening law in 2D. I'm trying to
>> move
>> >> on to the RateStateAgeing behavior and I have problems applying the
>> initial
>> >> traction conditions:
>> >>>
>> >>> I use a SimpleDB to apply the frictional properties on the fault: one
>> >> domain is rate-strengthening, the other is rate-weakening with a linear
>> >> transition between the two domains. If I do not try to turn on any
>> >> traction_perturbation option, the code runs great, and I have a slow
>> slip
>> >> all the along the fault at any time.
>> >>>
>> >>> As I want the rate-weakening domain to be stuck sometimes, I want to
>> >> turn on traction_perturbation.db_initial. I use a SimpleDB to put a
>> >> negative traction all along the fault, and a shear traction compatible
>> with
>> >> the friction coefficients I impose. However, when I run this, I have
>> the
>> >> following error message from PETSc (I attach the log):
>> >>>
>> >>>>>
>> >>
>> /Users/jolivetinsar/MYBIN/pylith/pylith-1.7.1-darwin-10.6.8/lib/python2.7/site-packages/pylith/problems/Implicit.py:215:step
>> >>> -- implicit(info)
>> >>> -- Solving equations.
>> >>> 0 SNES Function norm 2.679589636063e+01
>> >>> 0 KSP Residual norm 9.637565795900e-01
>> >>> 1 KSP Residual norm 3.564937135471e-08
>> >>> 2 KSP Residual norm 4.313963401911e-16
>> >>> 3 KSP Residual norm 3.050427683646e-16
>> >>> 4 KSP Residual norm 2.490663081181e-16
>> >>> 5 KSP Residual norm 2.156976950523e-16
>> >>> 6 KSP Residual norm 1.929258577738e-16
>> >>> 7 KSP Residual norm 1.761163907914e-16
>> >>> 8 KSP Residual norm 1.630520838230e-16
>> >>> 9 KSP Residual norm 1.525212509706e-16
>> >>> 10 KSP Residual norm 1.833325155789e-11
>> >>> 11 KSP Residual norm 7.039784754868e-23
>> >>> Linear solve converged due to CONVERGED_ATOL iterations 11
>> >>> [0]PETSC ERROR: --------------------- Error Message
>> >> ------------------------------------
>> >>> [0]PETSC ERROR: Floating point exception!
>> >>> [0]PETSC ERROR: Infinite or not-a-number generated in norm!
>> >>>
>> >>> If anyone has an idea about that error message....
>> >>>
>> >>> I think it has to do with some slip vector. If I set
>> >> friction.min_slip_rate=1e-1, this problem does not happen (the
>> nonlinear
>> >> solver does not converge, but at least, no crash). If I set the normal
>> >> traction to be positive, it does not crash as well (and it does not
>> >> converge as well). I join to this message the cfg file I am using for
>> that
>> >> experiment.
>> >>>
>> >>> Thank you
>> >>> Romain
>> >>>
>> >>>
>> >>>
>> >>>
>> >>> Romain Jolivet
>> >>> Postdoctoral Scholar
>> >>> Geological and Planetary Sciences
>> >>> California Institute of Technology, Pasadena, CA
>> >>> rjolivet at caltech.edu
>> >>> +1 (626) 560 6356
>> >>>
>> >>>
>> >>>
>> >>> _______________________________________________
>> >>> CIG-SHORT mailing list
>> >>> CIG-SHORT at geodynamics.org
>> >>> http://geodynamics.org/cgi-bin/mailman/listinfo/cig-short
>> >>>
>> >>
>> >> _______________________________________________
>> >> CIG-SHORT mailing list
>> >> CIG-SHORT at geodynamics.org
>> >> http://geodynamics.org/cgi-bin/mailman/listinfo/cig-short
>> >>
>> >
>> >
>> >
>> > --
>> > What most experimenters take for granted before they begin their
>> > experiments is infinitely more interesting than any results to which
>> their
>> > experiments lead.
>> > -- Norbert Wiener
>> >
>> >
>> >
>> > --
>> > What most experimenters take for granted before they begin their
>> > experiments is infinitely more interesting than any results to which
>> their
>> > experiments lead.
>> > -- Norbert Wiener
>> > -------------- next part --------------
>> > An HTML attachment was scrubbed...
>> > URL:
>> http://geodynamics.org/pipermail/cig-short/attachments/20120829/9e91899c/attachment.html
>> >
>> > ------------------------------
>> >
>> > _______________________________________________
>> > CIG-SHORT mailing list
>> > CIG-SHORT at geodynamics.org
>> > http://geodynamics.org/cgi-bin/mailman/listinfo/cig-short
>> >
>> >
>> > End of CIG-SHORT Digest, Vol 72, Issue 21
>> > *****************************************
>>
>> _______________________________________________
>> CIG-SHORT mailing list
>> CIG-SHORT at geodynamics.org
>> http://geodynamics.org/cgi-bin/mailman/listinfo/cig-short
>>
>
>
>
> --
> What most experimenters take for granted before they begin their
> experiments is infinitely more interesting than any results to which their
> experiments lead.
> -- Norbert Wiener
>
>
>
--
What most experimenters take for granted before they begin their
experiments is infinitely more interesting than any results to which their
experiments lead.
-- Norbert Wiener
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://geodynamics.org/pipermail/cig-short/attachments/20120830/964a474f/attachment-0001.htm
More information about the CIG-SHORT
mailing list