[CIG-SHORT] Rate and State and Initial Traction
Romain Jolivet
rjolivet at caltech.edu
Thu Aug 30 14:11:25 PDT 2012
I did just run:
pylith step01.cfg --petsc.fp_trap=0
Is that what you meant? If it is, I still have the same problem :-) and it does not change what I see on the screen. Should it change something?
However, the norm computation problem does not appear when I set the zero_tolerance to 1.0 (it does not converge though, but I don't have this Floating point exception...)
Romain
Romain Jolivet
Postdoctoral Scholar
Geological and Planetary Sciences
California Institute of Technology, Pasadena, CA
rjolivet at caltech.edu
+1 (626) 560 6356
On Aug 30, 2012, at 2:00 PM, Matthew Knepley wrote:
> On Thu, Aug 30, 2012 at 3:38 PM, Romain Jolivet <rjolivet at caltech.edu> wrote:
> Thanks for the advice. I changed the preconditioner settings to what you suggested. I don't see that jump in the linear norm anymore.
> However, I still have the floating point exception problem. This arise after the first round of ksp.
>
> In the log file (attached), I see to messages saying the ksp solver for friction does converge (twice), then the SNES residual norm is printed, then KSP starts, and when it has to compute the new SNES residual norm, it says that VecNorm() encoutered a Inf or NAN value:
>
> It could still be underflow. Try running with --petsc.fp_trap=0
>
> Thanks,
>
> Matt
>
> 141 KSP Residual norm 1.185626397065e-09
> 142 KSP Residual norm 8.558244459183e-10
> 143 KSP Residual norm 4.537120666370e-10
> 144 KSP Residual norm 2.695590025796e-10
> 145 KSP Residual norm 2.263153111449e-10
> 146 KSP Residual norm 2.156010427246e-10
> Linear solve converged due to CONVERGED_RTOL iterations 146
> [0]PETSC ERROR: --------------------- Error Message ------------------------------------
> [0]PETSC ERROR: Floating point exception!
> [0]PETSC ERROR: Infinite or not-a-number generated in norm!
> [0]PETSC ERROR: ------------------------------------------------------------------------
> [0]PETSC ERROR: Petsc Development HG revision: 3bede28f1195e406d4257c9fbeebefe827793801 HG Date: Mon Jun 11 22:14:12 2012 -0400
> [0]PETSC ERROR: See docs/changes/index.html for recent updates.
> [0]PETSC ERROR: See docs/faq.html for hints about trouble shooting.
> [0]PETSC ERROR: See docs/index.html for manual pages.
> [0]PETSC ERROR: ------------------------------------------------------------------------
> [0]PETSC ERROR: /Users/jolivetinsar/MYBIN/pylith/pylith-1.7.1-darwin-10.6.8/bin/mpinemesis on a pylith-da named dhcp-74-229.caltech.edu by jolivetinsar Thu Aug 30 13:34:55 2012
> [0]PETSC ERROR: Libraries linked from /Users/buildbot/install/pylith_darwin_10.6_binbot/lib
> [0]PETSC ERROR: Configure run at Tue Jun 12 06:09:00 2012
> [0]PETSC ERROR: Configure options --prefix=/Users/buildbot/install/pylith_darwin_10.6_binbot --download-f-blas-lapack=1 --with-sieve=1 --with-c2html=0 --CXXFLAGS=-DMPICH_IGNORE_CXX_SEEK --with-clanguage=C++ --with-mpicompilers=1 --with-debugging=0 --download-chaco=1 --download-boost=1 --download-ml=1 --with-hdf5=1 --with-hdf5-dir=/Users/buildbot/install/pylith_darwin_10.6_binbot --with-x=0 --with-shared-libraries=1
> [0]PETSC ERROR: ------------------------------------------------------------------------
> [0]PETSC ERROR: VecNorm() line 169 in src/vec/vec/interface/rvector.c
> [0]PETSC ERROR: VecNormalize() line 263 in src/vec/vec/interface/rvector.c
> [0]PETSC ERROR: KSPGMRESCycle() line 127 in src/ksp/ksp/impls/gmres/gmres.c
> [0]PETSC ERROR: KSPSolve_GMRES() line 231 in src/ksp/ksp/impls/gmres/gmres.c
> [0]PETSC ERROR: KSPSolve() line 446 in src/ksp/ksp/interface/itfunc.c
> [0]PETSC ERROR: _sensitivitySolve() line 2172 in "unknowndirectory/"faults/FaultCohesiveDyn.cc
> Traceback (most recent call last):
> File "/Users/jolivetinsar/MYBIN/pylith/pylith-1.7.1-darwin-10.6.8/lib/python2.7/site-packages/pylith/apps/PetscApplication.py", line 65, in onComputeNodes
> Fatal error. Calling MPI_Abort() to abort PyLith application.
> self.main(*args, **kwds)
> File "/Users/jolivetinsar/MYBIN/pylith/pylith-1.7.1-darwin-10.6.8/lib/python2.7/site-packages/pylith/apps/PyLithApp.py", line 119, in main
> self.problem.run(self)
> File "/Users/jolivetinsar/MYBIN/pylith/pylith-1.7.1-darwin-10.6.8/lib/python2.7/site-packages/pylith/problems/TimeDependent.py", line 154, in run
> self.formulation.step(t, dt)
> File "/Users/jolivetinsar/MYBIN/pylith/pylith-1.7.1-darwin-10.6.8/lib/python2.7/site-packages/pylith/problems/Implicit.py", line 219, in step
> self.solver.solve(dispIncr, self.jacobian, residual)
> File "/Users/jolivetinsar/MYBIN/pylith/pylith-1.7.1-darwin-10.6.8/lib/python2.7/site-packages/pylith/problems/problems.py", line 180, in solve
> def solve(self, *args): return _problems.SolverNonlinear_solve(self, *args)
> RuntimeError: std::exception
> application called MPI_Abort(MPI_COMM_WORLD, -1) - process 0
> /Users/jolivetinsar/MYBIN/pylith/pylith-1.7.1-darwin-10.6.8/bin/nemesis: mpirun: exit 255
> /Users/jolivetinsar/MYBIN/pylith/pylith-1.7.1-darwin-10.6.8/bin/pylith: /Users/jolivetinsar/MYBIN/pylith/pylith-1.7.1-darwin-10.6.8/bin/nemesis: exit 1
>
> Any idea?
> Romain
>
>
>
> Romain Jolivet
> Postdoctoral Scholar
> Geological and Planetary Sciences
> California Institute of Technology, Pasadena, CA
> rjolivet at caltech.edu
> +1 (626) 560 6356
>
> On Aug 30, 2012, at 12:00 PM, cig-short-request at geodynamics.org wrote:
>
> > Send CIG-SHORT mailing list submissions to
> > cig-short at geodynamics.org
> >
> > To subscribe or unsubscribe via the World Wide Web, visit
> > http://geodynamics.org/cgi-bin/mailman/listinfo/cig-short
> > or, via email, send a message with subject or body 'help' to
> > cig-short-request at geodynamics.org
> >
> > You can reach the person managing the list at
> > cig-short-owner at geodynamics.org
> >
> > When replying, please edit your Subject line so it is more specific
> > than "Re: Contents of CIG-SHORT digest..."
> >
> >
> > Today's Topics:
> >
> > 1. Re: Rate and State and Initial Traction (Brad Aagaard)
> > 2. Fwd: Rate and State and Initial Traction (Matthew Knepley)
> >
> >
> > ----------------------------------------------------------------------
> >
> > Message: 1
> > Date: Wed, 29 Aug 2012 12:36:08 -0700
> > From: Brad Aagaard <baagaard at usgs.gov>
> > Subject: Re: [CIG-SHORT] Rate and State and Initial Traction
> > To: cig-short at geodynamics.org
> > Message-ID: <503E6F28.6050001 at usgs.gov>
> > Content-Type: text/plain; charset=ISO-8859-1; format=flowed
> >
> > Romain,
> >
> > It looks like you are using Schur complement preconditioner settings.
> > The jump in the KSP residual at iteration 10 is odd and suggests
> > something is not quite right in the linear solve (probably related to
> > the preconditioner). Try using the AMG preconditioner. It may not be
> > quite as fast but it is more robust. Matt can probably help you tune the
> > Schur complement solver conditions some more if necessary.
> >
> > Your SNES tolerances are quite large. I suggest using tolerances similar
> > to those in examples/3d/hex8/step14.cfg.
> >
> > SOLVER TOLERANCES
> >
> > ksp_rtol = 1.0e-16
> > ksp_atol = 1.0e-12
> >
> > snes_rtol = 1.0e-14
> > snes_atol = 1.0e-10
> >
> >
> > AMG PRECONDITIONER SETTINGS
> >
> > [pylithapp.timedependent.formulation]
> > split_fields = True
> > matrix_type = aij
> >
> > [pylithapp.petsc]
> > ksp_gmres_restart = 100
> > fs_pc_type = fieldsplit
> > fs_pc_fieldsplit_real_diagonal = true
> > fs_pc_fieldsplit_type = multiplicative
> > fs_fieldsplit_0_pc_type = ml
> > fs_fieldsplit_1_pc_type = jacobi
> > fs_fieldsplit_0_ksp_type = preonly
> > fs_fieldsplit_1_ksp_type = preonly
> >
> > Brad
> >
> >
> > On 08/29/2012 11:42 AM, Romain Jolivet wrote:
> >> Hello Brad, Matt, Charles and the pylith team,
> >>
> >> Following your advice from the other day, I am experimenting how to create earthquakes on a frictional interface by applying a traction change history. It works fine with the SlipWeakening law in 2D. I'm trying to move on to the RateStateAgeing behavior and I have problems applying the initial traction conditions:
> >>
> >> I use a SimpleDB to apply the frictional properties on the fault: one domain is rate-strengthening, the other is rate-weakening with a linear transition between the two domains. If I do not try to turn on any traction_perturbation option, the code runs great, and I have a slow slip all the along the fault at any time.
> >>
> >> As I want the rate-weakening domain to be stuck sometimes, I want to turn on traction_perturbation.db_initial. I use a SimpleDB to put a negative traction all along the fault, and a shear traction compatible with the friction coefficients I impose. However, when I run this, I have the following error message from PETSc (I attach the log):
> >>
> >>>> /Users/jolivetinsar/MYBIN/pylith/pylith-1.7.1-darwin-10.6.8/lib/python2.7/site-packages/pylith/problems/Implicit.py:215:step
> >> -- implicit(info)
> >> -- Solving equations.
> >> 0 SNES Function norm 2.679589636063e+01
> >> 0 KSP Residual norm 9.637565795900e-01
> >> 1 KSP Residual norm 3.564937135471e-08
> >> 2 KSP Residual norm 4.313963401911e-16
> >> 3 KSP Residual norm 3.050427683646e-16
> >> 4 KSP Residual norm 2.490663081181e-16
> >> 5 KSP Residual norm 2.156976950523e-16
> >> 6 KSP Residual norm 1.929258577738e-16
> >> 7 KSP Residual norm 1.761163907914e-16
> >> 8 KSP Residual norm 1.630520838230e-16
> >> 9 KSP Residual norm 1.525212509706e-16
> >> 10 KSP Residual norm 1.833325155789e-11
> >> 11 KSP Residual norm 7.039784754868e-23
> >> Linear solve converged due to CONVERGED_ATOL iterations 11
> >> [0]PETSC ERROR: --------------------- Error Message ------------------------------------
> >> [0]PETSC ERROR: Floating point exception!
> >> [0]PETSC ERROR: Infinite or not-a-number generated in norm!
> >>
> >> If anyone has an idea about that error message....
> >>
> >> I think it has to do with some slip vector. If I set friction.min_slip_rate=1e-1, this problem does not happen (the nonlinear solver does not converge, but at least, no crash). If I set the normal traction to be positive, it does not crash as well (and it does not converge as well). I join to this message the cfg file I am using for that experiment.
> >>
> >> Thank you
> >> Romain
> >>
> >>
> >>
> >>
> >> Romain Jolivet
> >> Postdoctoral Scholar
> >> Geological and Planetary Sciences
> >> California Institute of Technology, Pasadena, CA
> >> rjolivet at caltech.edu
> >> +1 (626) 560 6356
> >>
> >>
> >>
> >> _______________________________________________
> >> CIG-SHORT mailing list
> >> CIG-SHORT at geodynamics.org
> >> http://geodynamics.org/cgi-bin/mailman/listinfo/cig-short
> >>
> >
> >
> >
> > ------------------------------
> >
> > Message: 2
> > Date: Wed, 29 Aug 2012 22:09:25 -0500
> > From: Matthew Knepley <knepley at mcs.anl.gov>
> > Subject: [CIG-SHORT] Fwd: Rate and State and Initial Traction
> > To: cig-short at geodynamics.org
> > Message-ID:
> > <CAMYG4GkMcUqRjhMaYZ8Ooj-x=06QKFrc7fiwW1dStmDSDO1qWA at mail.gmail.com>
> > Content-Type: text/plain; charset="iso-8859-1"
> >
> > On Wed, Aug 29, 2012 at 2:36 PM, Brad Aagaard <baagaard at usgs.gov> wrote:
> >
> >> Romain,
> >>
> >> It looks like you are using Schur complement preconditioner settings.
> >> The jump in the KSP residual at iteration 10 is odd and suggests
> >> something is not quite right in the linear solve (probably related to
> >>
> >
> > Brad is right that the tolerances here will produce some garbage. Going
> > below
> > machine epsilon (about 10^-16) means you get randomness and ultimately
> > underflow which I think you got in a norm calculation since things get
> > squared.
> > The solver should have converged on iterate 2, and will with the tolerances
> > that
> > Brad suggested.
> >
> > Thanks,
> >
> > Matt
> >
> >
> >> the preconditioner). Try using the AMG preconditioner. It may not be
> >> quite as fast but it is more robust. Matt can probably help you tune the
> >> Schur complement solver conditions some more if necessary.
> >>
> >> Your SNES tolerances are quite large. I suggest using tolerances similar
> >> to those in examples/3d/hex8/step14.cfg.
> >>
> >> SOLVER TOLERANCES
> >>
> >> ksp_rtol = 1.0e-16
> >> ksp_atol = 1.0e-12
> >>
> >> snes_rtol = 1.0e-14
> >> snes_atol = 1.0e-10
> >>
> >>
> >> AMG PRECONDITIONER SETTINGS
> >>
> >> [pylithapp.timedependent.formulation]
> >> split_fields = True
> >> matrix_type = aij
> >>
> >> [pylithapp.petsc]
> >> ksp_gmres_restart = 100
> >> fs_pc_type = fieldsplit
> >> fs_pc_fieldsplit_real_diagonal = true
> >> fs_pc_fieldsplit_type = multiplicative
> >> fs_fieldsplit_0_pc_type = ml
> >> fs_fieldsplit_1_pc_type = jacobi
> >> fs_fieldsplit_0_ksp_type = preonly
> >> fs_fieldsplit_1_ksp_type = preonly
> >>
> >> Brad
> >>
> >>
> >> On 08/29/2012 11:42 AM, Romain Jolivet wrote:
> >>> Hello Brad, Matt, Charles and the pylith team,
> >>>
> >>> Following your advice from the other day, I am experimenting how to
> >> create earthquakes on a frictional interface by applying a traction change
> >> history. It works fine with the SlipWeakening law in 2D. I'm trying to move
> >> on to the RateStateAgeing behavior and I have problems applying the initial
> >> traction conditions:
> >>>
> >>> I use a SimpleDB to apply the frictional properties on the fault: one
> >> domain is rate-strengthening, the other is rate-weakening with a linear
> >> transition between the two domains. If I do not try to turn on any
> >> traction_perturbation option, the code runs great, and I have a slow slip
> >> all the along the fault at any time.
> >>>
> >>> As I want the rate-weakening domain to be stuck sometimes, I want to
> >> turn on traction_perturbation.db_initial. I use a SimpleDB to put a
> >> negative traction all along the fault, and a shear traction compatible with
> >> the friction coefficients I impose. However, when I run this, I have the
> >> following error message from PETSc (I attach the log):
> >>>
> >>>>>
> >> /Users/jolivetinsar/MYBIN/pylith/pylith-1.7.1-darwin-10.6.8/lib/python2.7/site-packages/pylith/problems/Implicit.py:215:step
> >>> -- implicit(info)
> >>> -- Solving equations.
> >>> 0 SNES Function norm 2.679589636063e+01
> >>> 0 KSP Residual norm 9.637565795900e-01
> >>> 1 KSP Residual norm 3.564937135471e-08
> >>> 2 KSP Residual norm 4.313963401911e-16
> >>> 3 KSP Residual norm 3.050427683646e-16
> >>> 4 KSP Residual norm 2.490663081181e-16
> >>> 5 KSP Residual norm 2.156976950523e-16
> >>> 6 KSP Residual norm 1.929258577738e-16
> >>> 7 KSP Residual norm 1.761163907914e-16
> >>> 8 KSP Residual norm 1.630520838230e-16
> >>> 9 KSP Residual norm 1.525212509706e-16
> >>> 10 KSP Residual norm 1.833325155789e-11
> >>> 11 KSP Residual norm 7.039784754868e-23
> >>> Linear solve converged due to CONVERGED_ATOL iterations 11
> >>> [0]PETSC ERROR: --------------------- Error Message
> >> ------------------------------------
> >>> [0]PETSC ERROR: Floating point exception!
> >>> [0]PETSC ERROR: Infinite or not-a-number generated in norm!
> >>>
> >>> If anyone has an idea about that error message....
> >>>
> >>> I think it has to do with some slip vector. If I set
> >> friction.min_slip_rate=1e-1, this problem does not happen (the nonlinear
> >> solver does not converge, but at least, no crash). If I set the normal
> >> traction to be positive, it does not crash as well (and it does not
> >> converge as well). I join to this message the cfg file I am using for that
> >> experiment.
> >>>
> >>> Thank you
> >>> Romain
> >>>
> >>>
> >>>
> >>>
> >>> Romain Jolivet
> >>> Postdoctoral Scholar
> >>> Geological and Planetary Sciences
> >>> California Institute of Technology, Pasadena, CA
> >>> rjolivet at caltech.edu
> >>> +1 (626) 560 6356
> >>>
> >>>
> >>>
> >>> _______________________________________________
> >>> CIG-SHORT mailing list
> >>> CIG-SHORT at geodynamics.org
> >>> http://geodynamics.org/cgi-bin/mailman/listinfo/cig-short
> >>>
> >>
> >> _______________________________________________
> >> CIG-SHORT mailing list
> >> CIG-SHORT at geodynamics.org
> >> http://geodynamics.org/cgi-bin/mailman/listinfo/cig-short
> >>
> >
> >
> >
> > --
> > What most experimenters take for granted before they begin their
> > experiments is infinitely more interesting than any results to which their
> > experiments lead.
> > -- Norbert Wiener
> >
> >
> >
> > --
> > What most experimenters take for granted before they begin their
> > experiments is infinitely more interesting than any results to which their
> > experiments lead.
> > -- Norbert Wiener
> > -------------- next part --------------
> > An HTML attachment was scrubbed...
> > URL: http://geodynamics.org/pipermail/cig-short/attachments/20120829/9e91899c/attachment.html
> >
> > ------------------------------
> >
> > _______________________________________________
> > CIG-SHORT mailing list
> > CIG-SHORT at geodynamics.org
> > http://geodynamics.org/cgi-bin/mailman/listinfo/cig-short
> >
> >
> > End of CIG-SHORT Digest, Vol 72, Issue 21
> > *****************************************
>
> _______________________________________________
> CIG-SHORT mailing list
> CIG-SHORT at geodynamics.org
> http://geodynamics.org/cgi-bin/mailman/listinfo/cig-short
>
>
>
> --
> What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead.
> -- Norbert Wiener
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://geodynamics.org/pipermail/cig-short/attachments/20120830/3120879b/attachment-0001.htm
More information about the CIG-SHORT
mailing list