[CIG-SHORT] Rate and State and Initial Traction
Matthew Knepley
knepley at mcs.anl.gov
Thu Aug 30 14:00:58 PDT 2012
On Thu, Aug 30, 2012 at 3:38 PM, Romain Jolivet <rjolivet at caltech.edu>wrote:
> Thanks for the advice. I changed the preconditioner settings to what you
> suggested. I don't see that jump in the linear norm anymore.
> However, I still have the floating point exception problem. This arise
> after the first round of ksp.
>
> In the log file (attached), I see to messages saying the ksp solver for
> friction does converge (twice), then the SNES residual norm is printed,
> then KSP starts, and when it has to compute the new SNES residual norm, it
> says that VecNorm() encoutered a Inf or NAN value:
>
It could still be underflow. Try running with --petsc.fp_trap=0
Thanks,
Matt
> 141 KSP Residual norm 1.185626397065e-09
> 142 KSP Residual norm 8.558244459183e-10
> 143 KSP Residual norm 4.537120666370e-10
> 144 KSP Residual norm 2.695590025796e-10
> 145 KSP Residual norm 2.263153111449e-10
> 146 KSP Residual norm 2.156010427246e-10
> Linear solve converged due to CONVERGED_RTOL iterations 146
> [0]PETSC ERROR: --------------------- Error Message
> ------------------------------------
> [0]PETSC ERROR: Floating point exception!
> [0]PETSC ERROR: Infinite or not-a-number generated in norm!
> [0]PETSC ERROR:
> ------------------------------------------------------------------------
> [0]PETSC ERROR: Petsc Development HG revision:
> 3bede28f1195e406d4257c9fbeebefe827793801 HG Date: Mon Jun 11 22:14:12 2012
> -0400
> [0]PETSC ERROR: See docs/changes/index.html for recent updates.
> [0]PETSC ERROR: See docs/faq.html for hints about trouble shooting.
> [0]PETSC ERROR: See docs/index.html for manual pages.
> [0]PETSC ERROR:
> ------------------------------------------------------------------------
> [0]PETSC ERROR:
> /Users/jolivetinsar/MYBIN/pylith/pylith-1.7.1-darwin-10.6.8/bin/mpinemesis
> on a pylith-da named dhcp-74-229.caltech.edu by jolivetinsar Thu Aug 30
> 13:34:55 2012
> [0]PETSC ERROR: Libraries linked from
> /Users/buildbot/install/pylith_darwin_10.6_binbot/lib
> [0]PETSC ERROR: Configure run at Tue Jun 12 06:09:00 2012
> [0]PETSC ERROR: Configure options
> --prefix=/Users/buildbot/install/pylith_darwin_10.6_binbot
> --download-f-blas-lapack=1 --with-sieve=1 --with-c2html=0
> --CXXFLAGS=-DMPICH_IGNORE_CXX_SEEK --with-clanguage=C++
> --with-mpicompilers=1 --with-debugging=0 --download-chaco=1
> --download-boost=1 --download-ml=1 --with-hdf5=1
> --with-hdf5-dir=/Users/buildbot/install/pylith_darwin_10.6_binbot
> --with-x=0 --with-shared-libraries=1
> [0]PETSC ERROR:
> ------------------------------------------------------------------------
> [0]PETSC ERROR: VecNorm() line 169 in src/vec/vec/interface/rvector.c
> [0]PETSC ERROR: VecNormalize() line 263 in src/vec/vec/interface/rvector.c
> [0]PETSC ERROR: KSPGMRESCycle() line 127 in src/ksp/ksp/impls/gmres/gmres.c
> [0]PETSC ERROR: KSPSolve_GMRES() line 231 in
> src/ksp/ksp/impls/gmres/gmres.c
> [0]PETSC ERROR: KSPSolve() line 446 in src/ksp/ksp/interface/itfunc.c
> [0]PETSC ERROR: _sensitivitySolve() line 2172 in
> "unknowndirectory/"faults/FaultCohesiveDyn.cc
> Traceback (most recent call last):
> File
> "/Users/jolivetinsar/MYBIN/pylith/pylith-1.7.1-darwin-10.6.8/lib/python2.7/site-packages/pylith/apps/PetscApplication.py",
> line 65, in onComputeNodes
> Fatal error. Calling MPI_Abort() to abort PyLith application.
> self.main(*args, **kwds)
> File
> "/Users/jolivetinsar/MYBIN/pylith/pylith-1.7.1-darwin-10.6.8/lib/python2.7/site-packages/pylith/apps/PyLithApp.py",
> line 119, in main
> self.problem.run(self)
> File
> "/Users/jolivetinsar/MYBIN/pylith/pylith-1.7.1-darwin-10.6.8/lib/python2.7/site-packages/pylith/problems/TimeDependent.py",
> line 154, in run
> self.formulation.step(t, dt)
> File
> "/Users/jolivetinsar/MYBIN/pylith/pylith-1.7.1-darwin-10.6.8/lib/python2.7/site-packages/pylith/problems/Implicit.py",
> line 219, in step
> self.solver.solve(dispIncr, self.jacobian, residual)
> File
> "/Users/jolivetinsar/MYBIN/pylith/pylith-1.7.1-darwin-10.6.8/lib/python2.7/site-packages/pylith/problems/problems.py",
> line 180, in solve
> def solve(self, *args): return _problems.SolverNonlinear_solve(self,
> *args)
> RuntimeError: std::exception
> application called MPI_Abort(MPI_COMM_WORLD, -1) - process 0
> /Users/jolivetinsar/MYBIN/pylith/pylith-1.7.1-darwin-10.6.8/bin/nemesis:
> mpirun: exit 255
> /Users/jolivetinsar/MYBIN/pylith/pylith-1.7.1-darwin-10.6.8/bin/pylith:
> /Users/jolivetinsar/MYBIN/pylith/pylith-1.7.1-darwin-10.6.8/bin/nemesis:
> exit 1
>
> Any idea?
> Romain
>
>
>
> Romain Jolivet
> Postdoctoral Scholar
> Geological and Planetary Sciences
> California Institute of Technology, Pasadena, CA
> rjolivet at caltech.edu
> +1 (626) 560 6356
>
> On Aug 30, 2012, at 12:00 PM, cig-short-request at geodynamics.org wrote:
>
> > Send CIG-SHORT mailing list submissions to
> > cig-short at geodynamics.org
> >
> > To subscribe or unsubscribe via the World Wide Web, visit
> > http://geodynamics.org/cgi-bin/mailman/listinfo/cig-short
> > or, via email, send a message with subject or body 'help' to
> > cig-short-request at geodynamics.org
> >
> > You can reach the person managing the list at
> > cig-short-owner at geodynamics.org
> >
> > When replying, please edit your Subject line so it is more specific
> > than "Re: Contents of CIG-SHORT digest..."
> >
> >
> > Today's Topics:
> >
> > 1. Re: Rate and State and Initial Traction (Brad Aagaard)
> > 2. Fwd: Rate and State and Initial Traction (Matthew Knepley)
> >
> >
> > ----------------------------------------------------------------------
> >
> > Message: 1
> > Date: Wed, 29 Aug 2012 12:36:08 -0700
> > From: Brad Aagaard <baagaard at usgs.gov>
> > Subject: Re: [CIG-SHORT] Rate and State and Initial Traction
> > To: cig-short at geodynamics.org
> > Message-ID: <503E6F28.6050001 at usgs.gov>
> > Content-Type: text/plain; charset=ISO-8859-1; format=flowed
> >
> > Romain,
> >
> > It looks like you are using Schur complement preconditioner settings.
> > The jump in the KSP residual at iteration 10 is odd and suggests
> > something is not quite right in the linear solve (probably related to
> > the preconditioner). Try using the AMG preconditioner. It may not be
> > quite as fast but it is more robust. Matt can probably help you tune the
> > Schur complement solver conditions some more if necessary.
> >
> > Your SNES tolerances are quite large. I suggest using tolerances similar
> > to those in examples/3d/hex8/step14.cfg.
> >
> > SOLVER TOLERANCES
> >
> > ksp_rtol = 1.0e-16
> > ksp_atol = 1.0e-12
> >
> > snes_rtol = 1.0e-14
> > snes_atol = 1.0e-10
> >
> >
> > AMG PRECONDITIONER SETTINGS
> >
> > [pylithapp.timedependent.formulation]
> > split_fields = True
> > matrix_type = aij
> >
> > [pylithapp.petsc]
> > ksp_gmres_restart = 100
> > fs_pc_type = fieldsplit
> > fs_pc_fieldsplit_real_diagonal = true
> > fs_pc_fieldsplit_type = multiplicative
> > fs_fieldsplit_0_pc_type = ml
> > fs_fieldsplit_1_pc_type = jacobi
> > fs_fieldsplit_0_ksp_type = preonly
> > fs_fieldsplit_1_ksp_type = preonly
> >
> > Brad
> >
> >
> > On 08/29/2012 11:42 AM, Romain Jolivet wrote:
> >> Hello Brad, Matt, Charles and the pylith team,
> >>
> >> Following your advice from the other day, I am experimenting how to
> create earthquakes on a frictional interface by applying a traction change
> history. It works fine with the SlipWeakening law in 2D. I'm trying to move
> on to the RateStateAgeing behavior and I have problems applying the initial
> traction conditions:
> >>
> >> I use a SimpleDB to apply the frictional properties on the fault: one
> domain is rate-strengthening, the other is rate-weakening with a linear
> transition between the two domains. If I do not try to turn on any
> traction_perturbation option, the code runs great, and I have a slow slip
> all the along the fault at any time.
> >>
> >> As I want the rate-weakening domain to be stuck sometimes, I want to
> turn on traction_perturbation.db_initial. I use a SimpleDB to put a
> negative traction all along the fault, and a shear traction compatible with
> the friction coefficients I impose. However, when I run this, I have the
> following error message from PETSc (I attach the log):
> >>
> >>>>
> /Users/jolivetinsar/MYBIN/pylith/pylith-1.7.1-darwin-10.6.8/lib/python2.7/site-packages/pylith/problems/Implicit.py:215:step
> >> -- implicit(info)
> >> -- Solving equations.
> >> 0 SNES Function norm 2.679589636063e+01
> >> 0 KSP Residual norm 9.637565795900e-01
> >> 1 KSP Residual norm 3.564937135471e-08
> >> 2 KSP Residual norm 4.313963401911e-16
> >> 3 KSP Residual norm 3.050427683646e-16
> >> 4 KSP Residual norm 2.490663081181e-16
> >> 5 KSP Residual norm 2.156976950523e-16
> >> 6 KSP Residual norm 1.929258577738e-16
> >> 7 KSP Residual norm 1.761163907914e-16
> >> 8 KSP Residual norm 1.630520838230e-16
> >> 9 KSP Residual norm 1.525212509706e-16
> >> 10 KSP Residual norm 1.833325155789e-11
> >> 11 KSP Residual norm 7.039784754868e-23
> >> Linear solve converged due to CONVERGED_ATOL iterations 11
> >> [0]PETSC ERROR: --------------------- Error Message
> ------------------------------------
> >> [0]PETSC ERROR: Floating point exception!
> >> [0]PETSC ERROR: Infinite or not-a-number generated in norm!
> >>
> >> If anyone has an idea about that error message....
> >>
> >> I think it has to do with some slip vector. If I set
> friction.min_slip_rate=1e-1, this problem does not happen (the nonlinear
> solver does not converge, but at least, no crash). If I set the normal
> traction to be positive, it does not crash as well (and it does not
> converge as well). I join to this message the cfg file I am using for that
> experiment.
> >>
> >> Thank you
> >> Romain
> >>
> >>
> >>
> >>
> >> Romain Jolivet
> >> Postdoctoral Scholar
> >> Geological and Planetary Sciences
> >> California Institute of Technology, Pasadena, CA
> >> rjolivet at caltech.edu
> >> +1 (626) 560 6356
> >>
> >>
> >>
> >> _______________________________________________
> >> CIG-SHORT mailing list
> >> CIG-SHORT at geodynamics.org
> >> http://geodynamics.org/cgi-bin/mailman/listinfo/cig-short
> >>
> >
> >
> >
> > ------------------------------
> >
> > Message: 2
> > Date: Wed, 29 Aug 2012 22:09:25 -0500
> > From: Matthew Knepley <knepley at mcs.anl.gov>
> > Subject: [CIG-SHORT] Fwd: Rate and State and Initial Traction
> > To: cig-short at geodynamics.org
> > Message-ID:
> > <CAMYG4GkMcUqRjhMaYZ8Ooj-x=
> 06QKFrc7fiwW1dStmDSDO1qWA at mail.gmail.com>
> > Content-Type: text/plain; charset="iso-8859-1"
> >
> > On Wed, Aug 29, 2012 at 2:36 PM, Brad Aagaard <baagaard at usgs.gov> wrote:
> >
> >> Romain,
> >>
> >> It looks like you are using Schur complement preconditioner settings.
> >> The jump in the KSP residual at iteration 10 is odd and suggests
> >> something is not quite right in the linear solve (probably related to
> >>
> >
> > Brad is right that the tolerances here will produce some garbage. Going
> > below
> > machine epsilon (about 10^-16) means you get randomness and ultimately
> > underflow which I think you got in a norm calculation since things get
> > squared.
> > The solver should have converged on iterate 2, and will with the
> tolerances
> > that
> > Brad suggested.
> >
> > Thanks,
> >
> > Matt
> >
> >
> >> the preconditioner). Try using the AMG preconditioner. It may not be
> >> quite as fast but it is more robust. Matt can probably help you tune the
> >> Schur complement solver conditions some more if necessary.
> >>
> >> Your SNES tolerances are quite large. I suggest using tolerances similar
> >> to those in examples/3d/hex8/step14.cfg.
> >>
> >> SOLVER TOLERANCES
> >>
> >> ksp_rtol = 1.0e-16
> >> ksp_atol = 1.0e-12
> >>
> >> snes_rtol = 1.0e-14
> >> snes_atol = 1.0e-10
> >>
> >>
> >> AMG PRECONDITIONER SETTINGS
> >>
> >> [pylithapp.timedependent.formulation]
> >> split_fields = True
> >> matrix_type = aij
> >>
> >> [pylithapp.petsc]
> >> ksp_gmres_restart = 100
> >> fs_pc_type = fieldsplit
> >> fs_pc_fieldsplit_real_diagonal = true
> >> fs_pc_fieldsplit_type = multiplicative
> >> fs_fieldsplit_0_pc_type = ml
> >> fs_fieldsplit_1_pc_type = jacobi
> >> fs_fieldsplit_0_ksp_type = preonly
> >> fs_fieldsplit_1_ksp_type = preonly
> >>
> >> Brad
> >>
> >>
> >> On 08/29/2012 11:42 AM, Romain Jolivet wrote:
> >>> Hello Brad, Matt, Charles and the pylith team,
> >>>
> >>> Following your advice from the other day, I am experimenting how to
> >> create earthquakes on a frictional interface by applying a traction
> change
> >> history. It works fine with the SlipWeakening law in 2D. I'm trying to
> move
> >> on to the RateStateAgeing behavior and I have problems applying the
> initial
> >> traction conditions:
> >>>
> >>> I use a SimpleDB to apply the frictional properties on the fault: one
> >> domain is rate-strengthening, the other is rate-weakening with a linear
> >> transition between the two domains. If I do not try to turn on any
> >> traction_perturbation option, the code runs great, and I have a slow
> slip
> >> all the along the fault at any time.
> >>>
> >>> As I want the rate-weakening domain to be stuck sometimes, I want to
> >> turn on traction_perturbation.db_initial. I use a SimpleDB to put a
> >> negative traction all along the fault, and a shear traction compatible
> with
> >> the friction coefficients I impose. However, when I run this, I have the
> >> following error message from PETSc (I attach the log):
> >>>
> >>>>>
> >>
> /Users/jolivetinsar/MYBIN/pylith/pylith-1.7.1-darwin-10.6.8/lib/python2.7/site-packages/pylith/problems/Implicit.py:215:step
> >>> -- implicit(info)
> >>> -- Solving equations.
> >>> 0 SNES Function norm 2.679589636063e+01
> >>> 0 KSP Residual norm 9.637565795900e-01
> >>> 1 KSP Residual norm 3.564937135471e-08
> >>> 2 KSP Residual norm 4.313963401911e-16
> >>> 3 KSP Residual norm 3.050427683646e-16
> >>> 4 KSP Residual norm 2.490663081181e-16
> >>> 5 KSP Residual norm 2.156976950523e-16
> >>> 6 KSP Residual norm 1.929258577738e-16
> >>> 7 KSP Residual norm 1.761163907914e-16
> >>> 8 KSP Residual norm 1.630520838230e-16
> >>> 9 KSP Residual norm 1.525212509706e-16
> >>> 10 KSP Residual norm 1.833325155789e-11
> >>> 11 KSP Residual norm 7.039784754868e-23
> >>> Linear solve converged due to CONVERGED_ATOL iterations 11
> >>> [0]PETSC ERROR: --------------------- Error Message
> >> ------------------------------------
> >>> [0]PETSC ERROR: Floating point exception!
> >>> [0]PETSC ERROR: Infinite or not-a-number generated in norm!
> >>>
> >>> If anyone has an idea about that error message....
> >>>
> >>> I think it has to do with some slip vector. If I set
> >> friction.min_slip_rate=1e-1, this problem does not happen (the nonlinear
> >> solver does not converge, but at least, no crash). If I set the normal
> >> traction to be positive, it does not crash as well (and it does not
> >> converge as well). I join to this message the cfg file I am using for
> that
> >> experiment.
> >>>
> >>> Thank you
> >>> Romain
> >>>
> >>>
> >>>
> >>>
> >>> Romain Jolivet
> >>> Postdoctoral Scholar
> >>> Geological and Planetary Sciences
> >>> California Institute of Technology, Pasadena, CA
> >>> rjolivet at caltech.edu
> >>> +1 (626) 560 6356
> >>>
> >>>
> >>>
> >>> _______________________________________________
> >>> CIG-SHORT mailing list
> >>> CIG-SHORT at geodynamics.org
> >>> http://geodynamics.org/cgi-bin/mailman/listinfo/cig-short
> >>>
> >>
> >> _______________________________________________
> >> CIG-SHORT mailing list
> >> CIG-SHORT at geodynamics.org
> >> http://geodynamics.org/cgi-bin/mailman/listinfo/cig-short
> >>
> >
> >
> >
> > --
> > What most experimenters take for granted before they begin their
> > experiments is infinitely more interesting than any results to which
> their
> > experiments lead.
> > -- Norbert Wiener
> >
> >
> >
> > --
> > What most experimenters take for granted before they begin their
> > experiments is infinitely more interesting than any results to which
> their
> > experiments lead.
> > -- Norbert Wiener
> > -------------- next part --------------
> > An HTML attachment was scrubbed...
> > URL:
> http://geodynamics.org/pipermail/cig-short/attachments/20120829/9e91899c/attachment.html
> >
> > ------------------------------
> >
> > _______________________________________________
> > CIG-SHORT mailing list
> > CIG-SHORT at geodynamics.org
> > http://geodynamics.org/cgi-bin/mailman/listinfo/cig-short
> >
> >
> > End of CIG-SHORT Digest, Vol 72, Issue 21
> > *****************************************
>
> _______________________________________________
> CIG-SHORT mailing list
> CIG-SHORT at geodynamics.org
> http://geodynamics.org/cgi-bin/mailman/listinfo/cig-short
>
--
What most experimenters take for granted before they begin their
experiments is infinitely more interesting than any results to which their
experiments lead.
-- Norbert Wiener
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://geodynamics.org/pipermail/cig-short/attachments/20120830/f394f2bf/attachment-0001.htm
More information about the CIG-SHORT
mailing list