[CIG-SHORT] Rate and State and Initial Traction
Brad Aagaard
baagaard at usgs.gov
Mon Sep 3 15:55:03 PDT 2012
On 9/3/12 3:25 PM, Romain Jolivet wrote:
> I just saw your email. I thought the state-variable had a default value
> given by the expression you just sent me.
The default initial value for the state variable in rate-state friction
in v1.7.1 is 0.0. As a result it should *always* be set by the user to
something more reasonable.
I will continue to think about how to make the default value more robust
and its default value more obvious to the user.
Brad
> On Aug 30, 2012, at 3:51 PM, cig-short-request at geodynamics.org
> <mailto:cig-short-request at geodynamics.org> wrote:
>
>> Send CIG-SHORT mailing list submissions to
>> cig-short at geodynamics.org <mailto:cig-short at geodynamics.org>
>>
>> To subscribe or unsubscribe via the World Wide Web, visit
>> http://geodynamics.org/cgi-bin/mailman/listinfo/cig-short
>> or, via email, send a message with subject or body 'help' to
>> cig-short-request at geodynamics.org
>>
>> You can reach the person managing the list at
>> cig-short-owner at geodynamics.org
>>
>> When replying, please edit your Subject line so it is more specific
>> than "Re: Contents of CIG-SHORT digest..."
>>
>>
>> Today's Topics:
>>
>> 1. Re: Rate and State and Initial Traction (Brad Aagaard)
>>
>>
>> ----------------------------------------------------------------------
>>
>> Message: 1
>> Date: Thu, 30 Aug 2012 15:51:45 -0700
>> From: Brad Aagaard <baagaard at usgs.gov>
>> Subject: Re: [CIG-SHORT] Rate and State and Initial Traction
>> To: cig-short at geodynamics.org
>> Message-ID: <503FEE81.504 at usgs.gov>
>> Content-Type: text/plain; charset=ISO-8859-1; format=flowed
>>
>> Romain,
>>
>> Your initial state variable for rate-state friction is not set, so the
>> default value of 0.0 is used. As a result the friction coefficient is
>> infinite. I will change the default to something reasonable nonzero
>> value, which will show up in the next PyLith release along with a check
>> for this. In the meantime, set an initial value for the state variable
>> as is done in examples/3d/hex8/step14.cfg.
>>
>> # Set spatial database for the initial value of the state variable.
>> db_initial_state = spatialdata.spatialdb.UniformDB
>> db_initial_state.label = Rate State Ageing State
>> db_initial_state.values = [state-variable]
>> # theta_ss = characteristic_slip_dist / reference_slip_rate
>> # ROMAIN: UPDATE THIS VALUE USING THE ABOVE EXPRESSION FOR THETA_SS
>> db_initial_state.data = [2.5e+9*s]
>>
>> Brad
>>
>>
>>
>> On 08/30/2012 02:29 PM, Romain Jolivet wrote:
>>> My relative tolerance is now 1e-20, and ksp_atol = 1e-12.
>>> zero_tolerance = 1e-10 , by default.
>>> PETSc says ksp converges due to atol (no funny stuff due to machine
>>> epsilon, then), and the ksp residual norm is 3 orders of magnitude
>>> smaller than zero_tolerance, but still the same crash.
>>>
>>> What worries me is that the exact same problem with SlipWeakening
>>> instead of RateStateAgeing runs fine, and converges quite fast.
>>>
>>> 227 KSP Residual norm 6.958904427869e-12
>>> 228 KSP Residual norm 6.691327759355e-12
>>> 229 KSP Residual norm 6.133784618340e-12
>>> 230 KSP Residual norm 5.231708553237e-12
>>> 231 KSP Residual norm 3.601477632037e-12
>>> 232 KSP Residual norm 2.240651195747e-12
>>> 233 KSP Residual norm 1.382256145050e-12
>>> 234 KSP Residual norm 1.051621131548e-12
>>> 235 KSP Residual norm 9.795463232997e-13
>>> Linear solve converged due to CONVERGED_ATOL iterations 235
>>> [0]PETSC ERROR: --------------------- Error Message
>>> ------------------------------------
>>> [0]PETSC ERROR: Floating point exception!
>>> [0]PETSC ERROR: Infinite or not-a-number generated in norm!
>>> [0]PETSC ERROR:
>>> ------------------------------------------------------------------------
>>> [0]PETSC ERROR: Petsc Development HG revision:
>>> 3bede28f1195e406d4257c9fbeebefe827793801 HG Date: Mon Jun 11
>>> 22:14:12 2012 -0400
>>> [0]PETSC ERROR: See docs/changes/index.html for recent updates.
>>> [0]PETSC ERROR: See docs/faq.html for hints about trouble shooting.
>>> [0]PETSC ERROR: See docs/index.html for manual pages.
>>> [0]PETSC ERROR:
>>> ------------------------------------------------------------------------
>>> [0]PETSC ERROR:
>>> /Users/jolivetinsar/MYBIN/pylith/pylith-1.7.1-darwin-10.6.8/bin/mpinemesis
>>> on a pylith-da named dhcp-74-229.caltech.edu by jolivetinsar Thu Aug
>>> 30 14:22:49 2012
>>> [0]PETSC ERROR: Libraries linked from
>>> /Users/buildbot/install/pylith_darwin_10.6_binbot/lib
>>> [0]PETSC ERROR: Configure run at Tue Jun 12 06:09:00 2012
>>> [0]PETSC ERROR: Configure options
>>> --prefix=/Users/buildbot/install/pylith_darwin_10.6_binbot
>>> --download-f-blas-lapack=1 --with-sieve=1 --with-c2html=0
>>> --CXXFLAGS=-DMPICH_IGNORE_CXX_SEEK --with-clanguage=C++
>>> --with-mpicompilers=1 --with-debugging=0 --download-chaco=1
>>> --download-boost=1 --download-ml=1 --with-hdf5=1
>>> --with-hdf5-dir=/Users/buildbot/install/pylith_darwin_10.6_binbot
>>> --with-x=0 --with-shared-libraries=1
>>> [0]Fatal error. Calling MPI_Abort() to abort PyLith application.
>>> PETSC ERROR:
>>> ------------------------------------------------------------------------
>>> [0]PETSC ERROR: VecNorm() line 169 in src/vec/vec/interface/rvector.c
>>> [0]PETSC ERROR: VecNormalize() line 263 in
>>> src/vec/vec/interface/rvector.c
>>> [0]PETSC ERROR: KSPGMRESCycle() line 127 in
>>> src/ksp/ksp/impls/gmres/gmres.c
>>> [0]PETSC ERROR: KSPSolve_GMRES() line 231 in
>>> src/ksp/ksp/impls/gmres/gmres.c
>>> [0]PETSC ERROR: KSPSolve() line 446 in src/ksp/ksp/interface/itfunc.c
>>> [0]PETSC ERROR: _sensitivitySolve() line 2172 in
>>> "unknowndirectory/"faults/FaultCohesiveDyn.cc
>>> Traceback (most recent call last):
>>> File
>>> "/Users/jolivetinsar/MYBIN/pylith/pylith-1.7.1-darwin-10.6.8/lib/python2.7/site-packages/pylith/apps/PetscApplication.py",
>>> line 65, in onComputeNodes
>>> self.main(*args, **kwds)
>>> File
>>> "/Users/jolivetinsar/MYBIN/pylith/pylith-1/Users/jolivetinsar/MYBIN/pylith/pylith-1.7.1-darwin-10.6.8/bin/nemesis:
>>> mpirun: exit 255
>>> /Users/jolivetinsar/MYBIN/pylith/pylith-1.7.1-darwin-10.6.8/bin/pylith:
>>> /Users/jolivetinsar/MYBIN/pylith/pylith-1.7.1-darwin-10.6.8/bin/nemesis:
>>> exit 1
>>>
>>>
>>> Romain Jolivet
>>> Postdoctoral Scholar
>>> Geological and Planetary Sciences
>>> California Institute of Technology, Pasadena, CA
>>> rjolivet at caltech.edu
>>> +1 (626) 560 6356
>>>
>>> On Aug 30, 2012, at 2:19 PM, cig-short-request at geodynamics.org wrote:
>>>
>>>> Send CIG-SHORT mailing list submissions to
>>>> cig-short at geodynamics.org
>>>>
>>>> To subscribe or unsubscribe via the World Wide Web, visit
>>>> http://geodynamics.org/cgi-bin/mailman/listinfo/cig-short
>>>> or, via email, send a message with subject or body 'help' to
>>>> cig-short-request at geodynamics.org
>>>>
>>>> You can reach the person managing the list at
>>>> cig-short-owner at geodynamics.org
>>>>
>>>> When replying, please edit your Subject line so it is more specific
>>>> than "Re: Contents of CIG-SHORT digest..."
>>>>
>>>>
>>>> Today's Topics:
>>>>
>>>> 1. Re: Rate and State and Initial Traction (Brad Aagaard)
>>>> 2. Re: Rate and State and Initial Traction (Matthew Knepley)
>>>>
>>>>
>>>> ----------------------------------------------------------------------
>>>>
>>>> Message: 1
>>>> Date: Thu, 30 Aug 2012 14:14:31 -0700
>>>> From: Brad Aagaard <baagaard at usgs.gov>
>>>> Subject: Re: [CIG-SHORT] Rate and State and Initial Traction
>>>> To: Matthew Knepley <knepley at mcs.anl.gov>
>>>> Cc: cig-short at geodynamics.org
>>>> Message-ID: <503FD7B7.3050102 at usgs.gov>
>>>> Content-Type: text/plain; charset=ISO-8859-1; format=flowed
>>>>
>>>> Matt,
>>>>
>>>> I suggested that Romain change ksp_rtol only, not ksp_atol. The relative
>>>> tolerance will be small enough so that the absolute tolerance (1.0e-12)
>>>> will be what stops the KSP solve.
>>>>
>>>> Brad
>>>>
>>>>
>>>> On 08/30/2012 02:08 PM, Matthew Knepley wrote:
>>>>> On Thu, Aug 30, 2012 at 4:04 PM, Brad Aagaard <baagaard at usgs.gov>
>>>>> wrote:
>>>>>
>>>>>> Romain,
>>>>>>
>>>>>> The stacktrace shows which solve is encountering inf or nan. The
>>>>>> routine
>>>>>> is _sensitivitySolve(). This is the solve associated with determining
>>>>>> how much to increment the slip to match the friction criterion.
>>>>>>
>>>>>> However, it looks like the problem is in the KSP solver
>>>>>> tolerances. Note
>>>>>> that the solver stops with a residual of 2e-10 due to ksp_rtol being
>>>>>> met. However, the friction_tolerance is (by default) 1.0e-10. The
>>>>>> error
>>>>>> in the solution is on the order of 2.0e-10 but the tolerance for
>>>>>> detecting whether a point is slipping is 1.0e-10. You want the
>>>>>> error in
>>>>>> the linear solution to be smaller than the tolerance for detecting
>>>>>> sliding. This means you should reduce your ksp_rtol to something much
>>>>>> smaller, like 1.0e-20, to force ksp_atol to control convergence.
>>>>>> Usually, the starting residual is a few orders of magnitude
>>>>>> smaller than
>>>>>> what you have so the tolerances I suggested work.
>>>>>>
>>>>>
>>>>> Okay, last time I told him not to use anything less than 1e-16
>>>>> since there
>>>>> are funny effects below roundoff. Could you try 1e-15?
>>>>>
>>>>> We do need consistency checks in these nested iterations.
>>>>>
>>>>> Thanks,
>>>>>
>>>>> Matt
>>>>>
>>>>>
>>>>>> Bottom line: Quasi-static friction solves generally require the linear
>>>>>> solves to converge due to the absolute tolerance, which *must* be
>>>>>> smaller than the zero_tolerance for friction.
>>>>>>
>>>>>> We will look into adding more consistency checking into the code
>>>>>> to try
>>>>>> and catch these types of errors.
>>>>>>
>>>>>> Regards,
>>>>>> Brad
>>>>>>
>>>>>>
>>>>>> On 08/30/2012 01:38 PM, Romain Jolivet wrote:
>>>>>>> Thanks for the advice. I changed the preconditioner settings to
>>>>>>> what you
>>>>>> suggested. I don't see that jump in the linear norm anymore.
>>>>>>> However, I still have the floating point exception problem. This
>>>>>>> arise
>>>>>> after the first round of ksp.
>>>>>>>
>>>>>>> In the log file (attached), I see to messages saying the ksp
>>>>>>> solver for
>>>>>> friction does converge (twice), then the SNES residual norm is
>>>>>> printed,
>>>>>> then KSP starts, and when it has to compute the new SNES residual
>>>>>> norm, it
>>>>>> says that VecNorm() encoutered a Inf or NAN value:
>>>>>>>
>>>>>>> 141 KSP Residual norm 1.185626397065e-09
>>>>>>> 142 KSP Residual norm 8.558244459183e-10
>>>>>>> 143 KSP Residual norm 4.537120666370e-10
>>>>>>> 144 KSP Residual norm 2.695590025796e-10
>>>>>>> 145 KSP Residual norm 2.263153111449e-10
>>>>>>> 146 KSP Residual norm 2.156010427246e-10
>>>>>>> Linear solve converged due to CONVERGED_RTOL iterations 146
>>>>>>> [0]PETSC ERROR: --------------------- Error Message
>>>>>> ------------------------------------
>>>>>>> [0]PETSC ERROR: Floating point exception!
>>>>>>> [0]PETSC ERROR: Infinite or not-a-number generated in norm!
>>>>>>> [0]PETSC ERROR:
>>>>>> ------------------------------------------------------------------------
>>>>>>> [0]PETSC ERROR: Petsc Development HG revision:
>>>>>> 3bede28f1195e406d4257c9fbeebefe827793801 HG Date: Mon Jun 11
>>>>>> 22:14:12 2012
>>>>>> -0400
>>>>>>> [0]PETSC ERROR: See docs/changes/index.html for recent updates.
>>>>>>> [0]PETSC ERROR: See docs/faq.html for hints about trouble shooting.
>>>>>>> [0]PETSC ERROR: See docs/index.html for manual pages.
>>>>>>> [0]PETSC ERROR:
>>>>>> ------------------------------------------------------------------------
>>>>>>> [0]PETSC ERROR:
>>>>>> /Users/jolivetinsar/MYBIN/pylith/pylith-1.7.1-darwin-10.6.8/bin/mpinemesis
>>>>>> on a pylith-da named dhcp-74-229.caltech.edu by jolivetinsar Thu
>>>>>> Aug 30
>>>>>> 13:34:55 2012
>>>>>>> [0]PETSC ERROR: Libraries linked from
>>>>>> /Users/buildbot/install/pylith_darwin_10.6_binbot/lib
>>>>>>> [0]PETSC ERROR: Configure run at Tue Jun 12 06:09:00 2012
>>>>>>> [0]PETSC ERROR: Configure options
>>>>>> --prefix=/Users/buildbot/install/pylith_darwin_10.6_binbot
>>>>>> --download-f-blas-lapack=1 --with-sieve=1 --with-c2html=0
>>>>>> --CXXFLAGS=-DMPICH_IGNORE_CXX_SEEK --with-clanguage=C++
>>>>>> --with-mpicompilers=1 --with-debugging=0 --download-chaco=1
>>>>>> --download-boost=1 --download-ml=1 --with-hdf5=1
>>>>>> --with-hdf5-dir=/Users/buildbot/install/pylith_darwin_10.6_binbot
>>>>>> --with-x=0 --with-shared-libraries=1
>>>>>>> [0]PETSC ERROR:
>>>>>> ------------------------------------------------------------------------
>>>>>>> [0]PETSC ERROR: VecNorm() line 169 in src/vec/vec/interface/rvector.c
>>>>>>> [0]PETSC ERROR: VecNormalize() line 263 in
>>>>>> src/vec/vec/interface/rvector.c
>>>>>>> [0]PETSC ERROR: KSPGMRESCycle() line 127 in
>>>>>> src/ksp/ksp/impls/gmres/gmres.c
>>>>>>> [0]PETSC ERROR: KSPSolve_GMRES() line 231 in
>>>>>> src/ksp/ksp/impls/gmres/gmres.c
>>>>>>> [0]PETSC ERROR: KSPSolve() line 446 in src/ksp/ksp/interface/itfunc.c
>>>>>>> [0]PETSC ERROR: _sensitivitySolve() line 2172 in
>>>>>> "unknowndirectory/"faults/FaultCohesiveDyn.cc
>>>>>>> Traceback (most recent call last):
>>>>>>> File
>>>>>> "/Users/jolivetinsar/MYBIN/pylith/pylith-1.7.1-darwin-10.6.8/lib/python2.7/site-packages/pylith/apps/PetscApplication.py",
>>>>>> line 65, in onComputeNodes
>>>>>>> Fatal error. Calling MPI_Abort() to abort PyLith application.
>>>>>>> self.main(*args, **kwds)
>>>>>>> File
>>>>>> "/Users/jolivetinsar/MYBIN/pylith/pylith-1.7.1-darwin-10.6.8/lib/python2.7/site-packages/pylith/apps/PyLithApp.py",
>>>>>> line 119, in main
>>>>>>> self.problem.run(self)
>>>>>>> File
>>>>>> "/Users/jolivetinsar/MYBIN/pylith/pylith-1.7.1-darwin-10.6.8/lib/python2.7/site-packages/pylith/problems/TimeDependent.py",
>>>>>> line 154, in run
>>>>>>> self.formulation.step(t, dt)
>>>>>>> File
>>>>>> "/Users/jolivetinsar/MYBIN/pylith/pylith-1.7.1-darwin-10.6.8/lib/python2.7/site-packages/pylith/problems/Implicit.py",
>>>>>> line 219, in step
>>>>>>> self.solver.solve(dispIncr, self.jacobian, residual)
>>>>>>> File
>>>>>> "/Users/jolivetinsar/MYBIN/pylith/pylith-1.7.1-darwin-10.6.8/lib/python2.7/site-packages/pylith/problems/problems.py",
>>>>>> line 180, in solve
>>>>>>> def solve(self, *args): return
>>>>>> _problems.SolverNonlinear_solve(self, *args)
>>>>>>> RuntimeError: std::exception
>>>>>>> application called MPI_Abort(MPI_COMM_WORLD, -1) - process 0
>>>>>>> /Users/jolivetinsar/MYBIN/pylith/pylith-1.7.1-darwin-10.6.8/bin/nemesis:
>>>>>> mpirun: exit 255
>>>>>>> /Users/jolivetinsar/MYBIN/pylith/pylith-1.7.1-darwin-10.6.8/bin/pylith:
>>>>>> /Users/jolivetinsar/MYBIN/pylith/pylith-1.7.1-darwin-10.6.8/bin/nemesis:
>>>>>> exit 1
>>>>>>>
>>>>>>> Any idea?
>>>>>>> Romain
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> Romain Jolivet
>>>>>>> Postdoctoral Scholar
>>>>>>> Geological and Planetary Sciences
>>>>>>> California Institute of Technology, Pasadena, CA
>>>>>>> rjolivet at caltech.edu
>>>>>>> +1 (626) 560 6356
>>>>>>>
>>>>>>> On Aug 30, 2012, at 12:00 PM, cig-short-request at geodynamics.org
>>>>>>> wrote:
>>>>>>>
>>>>>>>> Send CIG-SHORT mailing list submissions to
>>>>>>>> cig-short at geodynamics.org
>>>>>>>>
>>>>>>>> To subscribe or unsubscribe via the World Wide Web, visit
>>>>>>>> http://geodynamics.org/cgi-bin/mailman/listinfo/cig-short
>>>>>>>> or, via email, send a message with subject or body 'help' to
>>>>>>>> cig-short-request at geodynamics.org
>>>>>>>>
>>>>>>>> You can reach the person managing the list at
>>>>>>>> cig-short-owner at geodynamics.org
>>>>>>>>
>>>>>>>> When replying, please edit your Subject line so it is more specific
>>>>>>>> than "Re: Contents of CIG-SHORT digest..."
>>>>>>>>
>>>>>>>>
>>>>>>>> Today's Topics:
>>>>>>>>
>>>>>>>> 1. Re: Rate and State and Initial Traction (Brad Aagaard)
>>>>>>>> 2. Fwd: Rate and State and Initial Traction (Matthew Knepley)
>>>>>>>>
>>>>>>>>
>>>>>>>> ----------------------------------------------------------------------
>>>>>>>>
>>>>>>>> Message: 1
>>>>>>>> Date: Wed, 29 Aug 2012 12:36:08 -0700
>>>>>>>> From: Brad Aagaard <baagaard at usgs.gov>
>>>>>>>> Subject: Re: [CIG-SHORT] Rate and State and Initial Traction
>>>>>>>> To: cig-short at geodynamics.org
>>>>>>>> Message-ID: <503E6F28.6050001 at usgs.gov>
>>>>>>>> Content-Type: text/plain; charset=ISO-8859-1; format=flowed
>>>>>>>>
>>>>>>>> Romain,
>>>>>>>>
>>>>>>>> It looks like you are using Schur complement preconditioner
>>>>>>>> settings.
>>>>>>>> The jump in the KSP residual at iteration 10 is odd and suggests
>>>>>>>> something is not quite right in the linear solve (probably
>>>>>>>> related to
>>>>>>>> the preconditioner). Try using the AMG preconditioner. It may not be
>>>>>>>> quite as fast but it is more robust. Matt can probably help you
>>>>>>>> tune the
>>>>>>>> Schur complement solver conditions some more if necessary.
>>>>>>>>
>>>>>>>> Your SNES tolerances are quite large. I suggest using tolerances
>>>>>>>> similar
>>>>>>>> to those in examples/3d/hex8/step14.cfg.
>>>>>>>>
>>>>>>>> SOLVER TOLERANCES
>>>>>>>>
>>>>>>>> ksp_rtol = 1.0e-16
>>>>>>>> ksp_atol = 1.0e-12
>>>>>>>>
>>>>>>>> snes_rtol = 1.0e-14
>>>>>>>> snes_atol = 1.0e-10
>>>>>>>>
>>>>>>>>
>>>>>>>> AMG PRECONDITIONER SETTINGS
>>>>>>>>
>>>>>>>> [pylithapp.timedependent.formulation]
>>>>>>>> split_fields = True
>>>>>>>> matrix_type = aij
>>>>>>>>
>>>>>>>> [pylithapp.petsc]
>>>>>>>> ksp_gmres_restart = 100
>>>>>>>> fs_pc_type = fieldsplit
>>>>>>>> fs_pc_fieldsplit_real_diagonal = true
>>>>>>>> fs_pc_fieldsplit_type = multiplicative
>>>>>>>> fs_fieldsplit_0_pc_type = ml
>>>>>>>> fs_fieldsplit_1_pc_type = jacobi
>>>>>>>> fs_fieldsplit_0_ksp_type = preonly
>>>>>>>> fs_fieldsplit_1_ksp_type = preonly
>>>>>>>>
>>>>>>>> Brad
>>>>>>>>
>>>>>>>>
>>>>>>>> On 08/29/2012 11:42 AM, Romain Jolivet wrote:
>>>>>>>>> Hello Brad, Matt, Charles and the pylith team,
>>>>>>>>>
>>>>>>>>> Following your advice from the other day, I am experimenting
>>>>>>>>> how to
>>>>>> create earthquakes on a frictional interface by applying a
>>>>>> traction change
>>>>>> history. It works fine with the SlipWeakening law in 2D. I'm
>>>>>> trying to move
>>>>>> on to the RateStateAgeing behavior and I have problems applying
>>>>>> the initial
>>>>>> traction conditions:
>>>>>>>>>
>>>>>>>>> I use a SimpleDB to apply the frictional properties on the
>>>>>>>>> fault: one
>>>>>> domain is rate-strengthening, the other is rate-weakening with a
>>>>>> linear
>>>>>> transition between the two domains. If I do not try to turn on any
>>>>>> traction_perturbation option, the code runs great, and I have a
>>>>>> slow slip
>>>>>> all the along the fault at any time.
>>>>>>>>>
>>>>>>>>> As I want the rate-weakening domain to be stuck sometimes, I
>>>>>>>>> want to
>>>>>> turn on traction_perturbation.db_initial. I use a SimpleDB to put a
>>>>>> negative traction all along the fault, and a shear traction
>>>>>> compatible with
>>>>>> the friction coefficients I impose. However, when I run this, I
>>>>>> have the
>>>>>> following error message from PETSc (I attach the log):
>>>>>>>>>
>>>>>>>>>>>
>>>>>> /Users/jolivetinsar/MYBIN/pylith/pylith-1.7.1-darwin-10.6.8/lib/python2.7/site-packages/pylith/problems/Implicit.py:215:step
>>>>>>>>> -- implicit(info)
>>>>>>>>> -- Solving equations.
>>>>>>>>> 0 SNES Function norm 2.679589636063e+01
>>>>>>>>> 0 KSP Residual norm 9.637565795900e-01
>>>>>>>>> 1 KSP Residual norm 3.564937135471e-08
>>>>>>>>> 2 KSP Residual norm 4.313963401911e-16
>>>>>>>>> 3 KSP Residual norm 3.050427683646e-16
>>>>>>>>> 4 KSP Residual norm 2.490663081181e-16
>>>>>>>>> 5 KSP Residual norm 2.156976950523e-16
>>>>>>>>> 6 KSP Residual norm 1.929258577738e-16
>>>>>>>>> 7 KSP Residual norm 1.761163907914e-16
>>>>>>>>> 8 KSP Residual norm 1.630520838230e-16
>>>>>>>>> 9 KSP Residual norm 1.525212509706e-16
>>>>>>>>> 10 KSP Residual norm 1.833325155789e-11
>>>>>>>>> 11 KSP Residual norm 7.039784754868e-23
>>>>>>>>> Linear solve converged due to CONVERGED_ATOL iterations 11
>>>>>>>>> [0]PETSC ERROR: --------------------- Error Message
>>>>>> ------------------------------------
>>>>>>>>> [0]PETSC ERROR: Floating point exception!
>>>>>>>>> [0]PETSC ERROR: Infinite or not-a-number generated in norm!
>>>>>>>>>
>>>>>>>>> If anyone has an idea about that error message....
>>>>>>>>>
>>>>>>>>> I think it has to do with some slip vector. If I set
>>>>>> friction.min_slip_rate=1e-1, this problem does not happen (the
>>>>>> nonlinear
>>>>>> solver does not converge, but at least, no crash). If I set the normal
>>>>>> traction to be positive, it does not crash as well (and it does not
>>>>>> converge as well). I join to this message the cfg file I am using
>>>>>> for that
>>>>>> experiment.
>>>>>>>>>
>>>>>>>>> Thank you
>>>>>>>>> Romain
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> Romain Jolivet
>>>>>>>>> Postdoctoral Scholar
>>>>>>>>> Geological and Planetary Sciences
>>>>>>>>> California Institute of Technology, Pasadena, CA
>>>>>>>>> rjolivet at caltech.edu
>>>>>>>>> +1 (626) 560 6356
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> _______________________________________________
>>>>>>>>> CIG-SHORT mailing list
>>>>>>>>> CIG-SHORT at geodynamics.org
>>>>>>>>> http://geodynamics.org/cgi-bin/mailman/listinfo/cig-short
>>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> ------------------------------
>>>>>>>>
>>>>>>>> Message: 2
>>>>>>>> Date: Wed, 29 Aug 2012 22:09:25 -0500
>>>>>>>> From: Matthew Knepley <knepley at mcs.anl.gov>
>>>>>>>> Subject: [CIG-SHORT] Fwd: Rate and State and Initial Traction
>>>>>>>> To: cig-short at geodynamics.org
>>>>>>>> Message-ID:
>>>>>>>> <CAMYG4GkMcUqRjhMaYZ8Ooj-x=
>>>>>> 06QKFrc7fiwW1dStmDSDO1qWA at mail.gmail.com>
>>>>>>>> Content-Type: text/plain; charset="iso-8859-1"
>>>>>>>>
>>>>>>>> On Wed, Aug 29, 2012 at 2:36 PM, Brad Aagaard <baagaard at usgs.gov>
>>>>>> wrote:
>>>>>>>>
>>>>>>>>> Romain,
>>>>>>>>>
>>>>>>>>> It looks like you are using Schur complement preconditioner
>>>>>>>>> settings.
>>>>>>>>> The jump in the KSP residual at iteration 10 is odd and suggests
>>>>>>>>> something is not quite right in the linear solve (probably
>>>>>>>>> related to
>>>>>>>>>
>>>>>>>>
>>>>>>>> Brad is right that the tolerances here will produce some
>>>>>>>> garbage. Going
>>>>>>>> below
>>>>>>>> machine epsilon (about 10^-16) means you get randomness and
>>>>>>>> ultimately
>>>>>>>> underflow which I think you got in a norm calculation since
>>>>>>>> things get
>>>>>>>> squared.
>>>>>>>> The solver should have converged on iterate 2, and will with the
>>>>>> tolerances
>>>>>>>> that
>>>>>>>> Brad suggested.
>>>>>>>>
>>>>>>>> Thanks,
>>>>>>>>
>>>>>>>> Matt
>>>>>>>>
>>>>>>>>
>>>>>>>>> the preconditioner). Try using the AMG preconditioner. It may
>>>>>>>>> not be
>>>>>>>>> quite as fast but it is more robust. Matt can probably help you
>>>>>>>>> tune
>>>>>> the
>>>>>>>>> Schur complement solver conditions some more if necessary.
>>>>>>>>>
>>>>>>>>> Your SNES tolerances are quite large. I suggest using tolerances
>>>>>> similar
>>>>>>>>> to those in examples/3d/hex8/step14.cfg.
>>>>>>>>>
>>>>>>>>> SOLVER TOLERANCES
>>>>>>>>>
>>>>>>>>> ksp_rtol = 1.0e-16
>>>>>>>>> ksp_atol = 1.0e-12
>>>>>>>>>
>>>>>>>>> snes_rtol = 1.0e-14
>>>>>>>>> snes_atol = 1.0e-10
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> AMG PRECONDITIONER SETTINGS
>>>>>>>>>
>>>>>>>>> [pylithapp.timedependent.formulation]
>>>>>>>>> split_fields = True
>>>>>>>>> matrix_type = aij
>>>>>>>>>
>>>>>>>>> [pylithapp.petsc]
>>>>>>>>> ksp_gmres_restart = 100
>>>>>>>>> fs_pc_type = fieldsplit
>>>>>>>>> fs_pc_fieldsplit_real_diagonal = true
>>>>>>>>> fs_pc_fieldsplit_type = multiplicative
>>>>>>>>> fs_fieldsplit_0_pc_type = ml
>>>>>>>>> fs_fieldsplit_1_pc_type = jacobi
>>>>>>>>> fs_fieldsplit_0_ksp_type = preonly
>>>>>>>>> fs_fieldsplit_1_ksp_type = preonly
>>>>>>>>>
>>>>>>>>> Brad
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> On 08/29/2012 11:42 AM, Romain Jolivet wrote:
>>>>>>>>>> Hello Brad, Matt, Charles and the pylith team,
>>>>>>>>>>
>>>>>>>>>> Following your advice from the other day, I am experimenting
>>>>>>>>>> how to
>>>>>>>>> create earthquakes on a frictional interface by applying a traction
>>>>>> change
>>>>>>>>> history. It works fine with the SlipWeakening law in 2D. I'm
>>>>>>>>> trying to
>>>>>> move
>>>>>>>>> on to the RateStateAgeing behavior and I have problems applying the
>>>>>> initial
>>>>>>>>> traction conditions:
>>>>>>>>>>
>>>>>>>>>> I use a SimpleDB to apply the frictional properties on the
>>>>>>>>>> fault: one
>>>>>>>>> domain is rate-strengthening, the other is rate-weakening with
>>>>>>>>> a linear
>>>>>>>>> transition between the two domains. If I do not try to turn on any
>>>>>>>>> traction_perturbation option, the code runs great, and I have a
>>>>>>>>> slow
>>>>>> slip
>>>>>>>>> all the along the fault at any time.
>>>>>>>>>>
>>>>>>>>>> As I want the rate-weakening domain to be stuck sometimes, I
>>>>>>>>>> want to
>>>>>>>>> turn on traction_perturbation.db_initial. I use a SimpleDB to put a
>>>>>>>>> negative traction all along the fault, and a shear traction
>>>>>>>>> compatible
>>>>>> with
>>>>>>>>> the friction coefficients I impose. However, when I run this, I
>>>>>>>>> have
>>>>>> the
>>>>>>>>> following error message from PETSc (I attach the log):
>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>
>>>>>> /Users/jolivetinsar/MYBIN/pylith/pylith-1.7.1-darwin-10.6.8/lib/python2.7/site-packages/pylith/problems/Implicit.py:215:step
>>>>>>>>>> -- implicit(info)
>>>>>>>>>> -- Solving equations.
>>>>>>>>>> 0 SNES Function norm 2.679589636063e+01
>>>>>>>>>> 0 KSP Residual norm 9.637565795900e-01
>>>>>>>>>> 1 KSP Residual norm 3.564937135471e-08
>>>>>>>>>> 2 KSP Residual norm 4.313963401911e-16
>>>>>>>>>> 3 KSP Residual norm 3.050427683646e-16
>>>>>>>>>> 4 KSP Residual norm 2.490663081181e-16
>>>>>>>>>> 5 KSP Residual norm 2.156976950523e-16
>>>>>>>>>> 6 KSP Residual norm 1.929258577738e-16
>>>>>>>>>> 7 KSP Residual norm 1.761163907914e-16
>>>>>>>>>> 8 KSP Residual norm 1.630520838230e-16
>>>>>>>>>> 9 KSP Residual norm 1.525212509706e-16
>>>>>>>>>> 10 KSP Residual norm 1.833325155789e-11
>>>>>>>>>> 11 KSP Residual norm 7.039784754868e-23
>>>>>>>>>> Linear solve converged due to CONVERGED_ATOL iterations 11
>>>>>>>>>> [0]PETSC ERROR: --------------------- Error Message
>>>>>>>>> ------------------------------------
>>>>>>>>>> [0]PETSC ERROR: Floating point exception!
>>>>>>>>>> [0]PETSC ERROR: Infinite or not-a-number generated in norm!
>>>>>>>>>>
>>>>>>>>>> If anyone has an idea about that error message....
>>>>>>>>>>
>>>>>>>>>> I think it has to do with some slip vector. If I set
>>>>>>>>> friction.min_slip_rate=1e-1, this problem does not happen (the
>>>>>> nonlinear
>>>>>>>>> solver does not converge, but at least, no crash). If I set the
>>>>>>>>> normal
>>>>>>>>> traction to be positive, it does not crash as well (and it does not
>>>>>>>>> converge as well). I join to this message the cfg file I am
>>>>>>>>> using for
>>>>>> that
>>>>>>>>> experiment.
>>>>>>>>>>
>>>>>>>>>> Thank you
>>>>>>>>>> Romain
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> Romain Jolivet
>>>>>>>>>> Postdoctoral Scholar
>>>>>>>>>> Geological and Planetary Sciences
>>>>>>>>>> California Institute of Technology, Pasadena, CA
>>>>>>>>>> rjolivet at caltech.edu
>>>>>>>>>> +1 (626) 560 6356
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> _______________________________________________
>>>>>>>>>> CIG-SHORT mailing list
>>>>>>>>>> CIG-SHORT at geodynamics.org
>>>>>>>>>> http://geodynamics.org/cgi-bin/mailman/listinfo/cig-short
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>> _______________________________________________
>>>>>>>>> CIG-SHORT mailing list
>>>>>>>>> CIG-SHORT at geodynamics.org
>>>>>>>>> http://geodynamics.org/cgi-bin/mailman/listinfo/cig-short
>>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> --
>>>>>>>> What most experimenters take for granted before they begin their
>>>>>>>> experiments is infinitely more interesting than any results to which
>>>>>> their
>>>>>>>> experiments lead.
>>>>>>>> -- Norbert Wiener
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> --
>>>>>>>> What most experimenters take for granted before they begin their
>>>>>>>> experiments is infinitely more interesting than any results to which
>>>>>> their
>>>>>>>> experiments lead.
>>>>>>>> -- Norbert Wiener
>>>>>>>> -------------- next part --------------
>>>>>>>> An HTML attachment was scrubbed...
>>>>>>>> URL:
>>>>>> http://geodynamics.org/pipermail/cig-short/attachments/20120829/9e91899c/attachment.html
>>>>>>>>
>>>>>>>> ------------------------------
>>>>>>>>
>>>>>>>> _______________________________________________
>>>>>>>> CIG-SHORT mailing list
>>>>>>>> CIG-SHORT at geodynamics.org
>>>>>>>> http://geodynamics.org/cgi-bin/mailman/listinfo/cig-short
>>>>>>>>
>>>>>>>>
>>>>>>>> End of CIG-SHORT Digest, Vol 72, Issue 21
>>>>>>>> *****************************************
>>>>>>>
>>>>>>> _______________________________________________
>>>>>>> CIG-SHORT mailing list
>>>>>>> CIG-SHORT at geodynamics.org
>>>>>>> http://geodynamics.org/cgi-bin/mailman/listinfo/cig-short
>>>>>>>
>>>>>>
>>>>>> _______________________________________________
>>>>>> CIG-SHORT mailing list
>>>>>> CIG-SHORT at geodynamics.org
>>>>>> http://geodynamics.org/cgi-bin/mailman/listinfo/cig-short
>>>>>>
>>>>>
>>>>>
>>>>>
>>>>
>>>>
>>>>
>>>> ------------------------------
>>>>
>>>> Message: 2
>>>> Date: Thu, 30 Aug 2012 16:19:45 -0500
>>>> From: Matthew Knepley <knepley at mcs.anl.gov>
>>>> Subject: Re: [CIG-SHORT] Rate and State and Initial Traction
>>>> To: Romain Jolivet <rjolivet at caltech.edu>
>>>> Cc: cig-short at geodynamics.org
>>>> Message-ID:
>>>> <CAMYG4GkacxPh4TMvcXeWu84bFdzqvVuaQcd7mvXQJGdtddzkMw at mail.gmail.com>
>>>> Content-Type: text/plain; charset="iso-8859-1"
>>>>
>>>> On Thu, Aug 30, 2012 at 4:11 PM, Romain Jolivet
>>>> <rjolivet at caltech.edu>wrote:
>>>>
>>>>> I did just run:
>>>>>
>>>>> pylith step01.cfg --petsc.fp_trap=0
>>>>>
>>>>> Is that what you meant? If it is, I still have the same problem :-)
>>>>> and it
>>>>> does not change what I see on the screen. Should it change something?\
>>>>>
>>>>
>>>> Then it is not underflow.
>>>>
>>>>
>>>>> However, the norm computation problem does not appear when I set the
>>>>> zero_tolerance to 1.0 (it does not converge though, but I don't
>>>>> have this
>>>>> Floating point exception...)
>>>>>
>>>>
>>>> As Brad says, change the relative tolerance, but leave the absolute
>>>> tolerance intact.
>>>>
>>>> Matt
>>>>
>>>>
>>>>> Romain
>>>>>
>>>>>
>>>>> Romain Jolivet
>>>>> Postdoctoral Scholar
>>>>> Geological and Planetary Sciences
>>>>> California Institute of Technology, Pasadena, CA
>>>>> rjolivet at caltech.edu
>>>>> +1 (626) 560 6356
>>>>>
>>>>> On Aug 30, 2012, at 2:00 PM, Matthew Knepley wrote:
>>>>>
>>>>> On Thu, Aug 30, 2012 at 3:38 PM, Romain Jolivet
>>>>> <rjolivet at caltech.edu>wrote:
>>>>>
>>>>>> Thanks for the advice. I changed the preconditioner settings to
>>>>>> what you
>>>>>> suggested. I don't see that jump in the linear norm anymore.
>>>>>> However, I still have the floating point exception problem. This arise
>>>>>> after the first round of ksp.
>>>>>>
>>>>>> In the log file (attached), I see to messages saying the ksp
>>>>>> solver for
>>>>>> friction does converge (twice), then the SNES residual norm is
>>>>>> printed,
>>>>>> then KSP starts, and when it has to compute the new SNES residual
>>>>>> norm, it
>>>>>> says that VecNorm() encoutered a Inf or NAN value:
>>>>>>
>>>>>
>>>>> It could still be underflow. Try running with --petsc.fp_trap=0
>>>>>
>>>>> Thanks,
>>>>>
>>>>> Matt
>>>>>
>>>>>
>>>>>> 141 KSP Residual norm 1.185626397065e-09
>>>>>> 142 KSP Residual norm 8.558244459183e-10
>>>>>> 143 KSP Residual norm 4.537120666370e-10
>>>>>> 144 KSP Residual norm 2.695590025796e-10
>>>>>> 145 KSP Residual norm 2.263153111449e-10
>>>>>> 146 KSP Residual norm 2.156010427246e-10
>>>>>> Linear solve converged due to CONVERGED_RTOL iterations 146
>>>>>> [0]PETSC ERROR: --------------------- Error Message
>>>>>> ------------------------------------
>>>>>> [0]PETSC ERROR: Floating point exception!
>>>>>> [0]PETSC ERROR: Infinite or not-a-number generated in norm!
>>>>>> [0]PETSC ERROR:
>>>>>> ------------------------------------------------------------------------
>>>>>> [0]PETSC ERROR: Petsc Development HG revision:
>>>>>> 3bede28f1195e406d4257c9fbeebefe827793801 HG Date: Mon Jun 11
>>>>>> 22:14:12 2012
>>>>>> -0400
>>>>>> [0]PETSC ERROR: See docs/changes/index.html for recent updates.
>>>>>> [0]PETSC ERROR: See docs/faq.html for hints about trouble shooting.
>>>>>> [0]PETSC ERROR: See docs/index.html for manual pages.
>>>>>> [0]PETSC ERROR:
>>>>>> ------------------------------------------------------------------------
>>>>>> [0]PETSC ERROR:
>>>>>> /Users/jolivetinsar/MYBIN/pylith/pylith-1.7.1-darwin-10.6.8/bin/mpinemesis
>>>>>> on a pylith-da named dhcp-74-229.caltech.edu by jolivetinsar Thu
>>>>>> Aug 30
>>>>>> 13:34:55 2012
>>>>>> [0]PETSC ERROR: Libraries linked from
>>>>>> /Users/buildbot/install/pylith_darwin_10.6_binbot/lib
>>>>>> [0]PETSC ERROR: Configure run at Tue Jun 12 06:09:00 2012
>>>>>> [0]PETSC ERROR: Configure options
>>>>>> --prefix=/Users/buildbot/install/pylith_darwin_10.6_binbot
>>>>>> --download-f-blas-lapack=1 --with-sieve=1 --with-c2html=0
>>>>>> --CXXFLAGS=-DMPICH_IGNORE_CXX_SEEK --with-clanguage=C++
>>>>>> --with-mpicompilers=1 --with-debugging=0 --download-chaco=1
>>>>>> --download-boost=1 --download-ml=1 --with-hdf5=1
>>>>>> --with-hdf5-dir=/Users/buildbot/install/pylith_darwin_10.6_binbot
>>>>>> --with-x=0 --with-shared-libraries=1
>>>>>> [0]PETSC ERROR:
>>>>>> ------------------------------------------------------------------------
>>>>>> [0]PETSC ERROR: VecNorm() line 169 in src/vec/vec/interface/rvector.c
>>>>>> [0]PETSC ERROR: VecNormalize() line 263 in
>>>>>> src/vec/vec/interface/rvector.c
>>>>>> [0]PETSC ERROR: KSPGMRESCycle() line 127 in
>>>>>> src/ksp/ksp/impls/gmres/gmres.c
>>>>>> [0]PETSC ERROR: KSPSolve_GMRES() line 231 in
>>>>>> src/ksp/ksp/impls/gmres/gmres.c
>>>>>> [0]PETSC ERROR: KSPSolve() line 446 in src/ksp/ksp/interface/itfunc.c
>>>>>> [0]PETSC ERROR: _sensitivitySolve() line 2172 in
>>>>>> "unknowndirectory/"faults/FaultCohesiveDyn.cc
>>>>>> Traceback (most recent call last):
>>>>>> File
>>>>>> "/Users/jolivetinsar/MYBIN/pylith/pylith-1.7.1-darwin-10.6.8/lib/python2.7/site-packages/pylith/apps/PetscApplication.py",
>>>>>> line 65, in onComputeNodes
>>>>>> Fatal error. Calling MPI_Abort() to abort PyLith application.
>>>>>> self.main(*args, **kwds)
>>>>>> File
>>>>>> "/Users/jolivetinsar/MYBIN/pylith/pylith-1.7.1-darwin-10.6.8/lib/python2.7/site-packages/pylith/apps/PyLithApp.py",
>>>>>> line 119, in main
>>>>>> self.problem.run(self)
>>>>>> File
>>>>>> "/Users/jolivetinsar/MYBIN/pylith/pylith-1.7.1-darwin-10.6.8/lib/python2.7/site-packages/pylith/problems/TimeDependent.py",
>>>>>> line 154, in run
>>>>>> self.formulation.step(t, dt)
>>>>>> File
>>>>>> "/Users/jolivetinsar/MYBIN/pylith/pylith-1.7.1-darwin-10.6.8/lib/python2.7/site-packages/pylith/problems/Implicit.py",
>>>>>> line 219, in step
>>>>>> self.solver.solve(dispIncr, self.jacobian, residual)
>>>>>> File
>>>>>> "/Users/jolivetinsar/MYBIN/pylith/pylith-1.7.1-darwin-10.6.8/lib/python2.7/site-packages/pylith/problems/problems.py",
>>>>>> line 180, in solve
>>>>>> def solve(self, *args): return
>>>>>> _problems.SolverNonlinear_solve(self,
>>>>>> *args)
>>>>>> RuntimeError: std::exception
>>>>>> application called MPI_Abort(MPI_COMM_WORLD, -1) - process 0
>>>>>> /Users/jolivetinsar/MYBIN/pylith/pylith-1.7.1-darwin-10.6.8/bin/nemesis:
>>>>>> mpirun: exit 255
>>>>>> /Users/jolivetinsar/MYBIN/pylith/pylith-1.7.1-darwin-10.6.8/bin/pylith:
>>>>>> /Users/jolivetinsar/MYBIN/pylith/pylith-1.7.1-darwin-10.6.8/bin/nemesis:
>>>>>> exit 1
>>>>>>
>>>>>> Any idea?
>>>>>> Romain
>>>>>>
>>>>>>
>>>>>>
>>>>>> Romain Jolivet
>>>>>> Postdoctoral Scholar
>>>>>> Geological and Planetary Sciences
>>>>>> California Institute of Technology, Pasadena, CA
>>>>>> rjolivet at caltech.edu
>>>>>> +1 (626) 560 6356
>>>>>>
>>>>>> On Aug 30, 2012, at 12:00 PM, cig-short-request at geodynamics.org wrote:
>>>>>>
>>>>>>> Send CIG-SHORT mailing list submissions to
>>>>>>> cig-short at geodynamics.org
>>>>>>>
>>>>>>> To subscribe or unsubscribe via the World Wide Web, visit
>>>>>>> http://geodynamics.org/cgi-bin/mailman/listinfo/cig-short
>>>>>>> or, via email, send a message with subject or body 'help' to
>>>>>>> cig-short-request at geodynamics.org
>>>>>>>
>>>>>>> You can reach the person managing the list at
>>>>>>> cig-short-owner at geodynamics.org
>>>>>>>
>>>>>>> When replying, please edit your Subject line so it is more specific
>>>>>>> than "Re: Contents of CIG-SHORT digest..."
>>>>>>>
>>>>>>>
>>>>>>> Today's Topics:
>>>>>>>
>>>>>>> 1. Re: Rate and State and Initial Traction (Brad Aagaard)
>>>>>>> 2. Fwd: Rate and State and Initial Traction (Matthew Knepley)
>>>>>>>
>>>>>>>
>>>>>>> ----------------------------------------------------------------------
>>>>>>>
>>>>>>> Message: 1
>>>>>>> Date: Wed, 29 Aug 2012 12:36:08 -0700
>>>>>>> From: Brad Aagaard <baagaard at usgs.gov>
>>>>>>> Subject: Re: [CIG-SHORT] Rate and State and Initial Traction
>>>>>>> To: cig-short at geodynamics.org
>>>>>>> Message-ID: <503E6F28.6050001 at usgs.gov>
>>>>>>> Content-Type: text/plain; charset=ISO-8859-1; format=flowed
>>>>>>>
>>>>>>> Romain,
>>>>>>>
>>>>>>> It looks like you are using Schur complement preconditioner settings.
>>>>>>> The jump in the KSP residual at iteration 10 is odd and suggests
>>>>>>> something is not quite right in the linear solve (probably related to
>>>>>>> the preconditioner). Try using the AMG preconditioner. It may not be
>>>>>>> quite as fast but it is more robust. Matt can probably help you
>>>>>>> tune the
>>>>>>> Schur complement solver conditions some more if necessary.
>>>>>>>
>>>>>>> Your SNES tolerances are quite large. I suggest using tolerances
>>>>>>> similar
>>>>>>> to those in examples/3d/hex8/step14.cfg.
>>>>>>>
>>>>>>> SOLVER TOLERANCES
>>>>>>>
>>>>>>> ksp_rtol = 1.0e-16
>>>>>>> ksp_atol = 1.0e-12
>>>>>>>
>>>>>>> snes_rtol = 1.0e-14
>>>>>>> snes_atol = 1.0e-10
>>>>>>>
>>>>>>>
>>>>>>> AMG PRECONDITIONER SETTINGS
>>>>>>>
>>>>>>> [pylithapp.timedependent.formulation]
>>>>>>> split_fields = True
>>>>>>> matrix_type = aij
>>>>>>>
>>>>>>> [pylithapp.petsc]
>>>>>>> ksp_gmres_restart = 100
>>>>>>> fs_pc_type = fieldsplit
>>>>>>> fs_pc_fieldsplit_real_diagonal = true
>>>>>>> fs_pc_fieldsplit_type = multiplicative
>>>>>>> fs_fieldsplit_0_pc_type = ml
>>>>>>> fs_fieldsplit_1_pc_type = jacobi
>>>>>>> fs_fieldsplit_0_ksp_type = preonly
>>>>>>> fs_fieldsplit_1_ksp_type = preonly
>>>>>>>
>>>>>>> Brad
>>>>>>>
>>>>>>>
>>>>>>> On 08/29/2012 11:42 AM, Romain Jolivet wrote:
>>>>>>>> Hello Brad, Matt, Charles and the pylith team,
>>>>>>>>
>>>>>>>> Following your advice from the other day, I am experimenting how to
>>>>>> create earthquakes on a frictional interface by applying a
>>>>>> traction change
>>>>>> history. It works fine with the SlipWeakening law in 2D. I'm
>>>>>> trying to move
>>>>>> on to the RateStateAgeing behavior and I have problems applying
>>>>>> the initial
>>>>>> traction conditions:
>>>>>>>>
>>>>>>>> I use a SimpleDB to apply the frictional properties on the
>>>>>>>> fault: one
>>>>>> domain is rate-strengthening, the other is rate-weakening with a
>>>>>> linear
>>>>>> transition between the two domains. If I do not try to turn on any
>>>>>> traction_perturbation option, the code runs great, and I have a
>>>>>> slow slip
>>>>>> all the along the fault at any time.
>>>>>>>>
>>>>>>>> As I want the rate-weakening domain to be stuck sometimes, I want to
>>>>>> turn on traction_perturbation.db_initial. I use a SimpleDB to put a
>>>>>> negative traction all along the fault, and a shear traction
>>>>>> compatible with
>>>>>> the friction coefficients I impose. However, when I run this, I
>>>>>> have the
>>>>>> following error message from PETSc (I attach the log):
>>>>>>>>
>>>>>>>>>>
>>>>>> /Users/jolivetinsar/MYBIN/pylith/pylith-1.7.1-darwin-10.6.8/lib/python2.7/site-packages/pylith/problems/Implicit.py:215:step
>>>>>>>> -- implicit(info)
>>>>>>>> -- Solving equations.
>>>>>>>> 0 SNES Function norm 2.679589636063e+01
>>>>>>>> 0 KSP Residual norm 9.637565795900e-01
>>>>>>>> 1 KSP Residual norm 3.564937135471e-08
>>>>>>>> 2 KSP Residual norm 4.313963401911e-16
>>>>>>>> 3 KSP Residual norm 3.050427683646e-16
>>>>>>>> 4 KSP Residual norm 2.490663081181e-16
>>>>>>>> 5 KSP Residual norm 2.156976950523e-16
>>>>>>>> 6 KSP Residual norm 1.929258577738e-16
>>>>>>>> 7 KSP Residual norm 1.761163907914e-16
>>>>>>>> 8 KSP Residual norm 1.630520838230e-16
>>>>>>>> 9 KSP Residual norm 1.525212509706e-16
>>>>>>>> 10 KSP Residual norm 1.833325155789e-11
>>>>>>>> 11 KSP Residual norm 7.039784754868e-23
>>>>>>>> Linear solve converged due to CONVERGED_ATOL iterations 11
>>>>>>>> [0]PETSC ERROR: --------------------- Error Message
>>>>>> ------------------------------------
>>>>>>>> [0]PETSC ERROR: Floating point exception!
>>>>>>>> [0]PETSC ERROR: Infinite or not-a-number generated in norm!
>>>>>>>>
>>>>>>>> If anyone has an idea about that error message....
>>>>>>>>
>>>>>>>> I think it has to do with some slip vector. If I set
>>>>>> friction.min_slip_rate=1e-1, this problem does not happen (the
>>>>>> nonlinear
>>>>>> solver does not converge, but at least, no crash). If I set the normal
>>>>>> traction to be positive, it does not crash as well (and it does not
>>>>>> converge as well). I join to this message the cfg file I am using
>>>>>> for that
>>>>>> experiment.
>>>>>>>>
>>>>>>>> Thank you
>>>>>>>> Romain
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> Romain Jolivet
>>>>>>>> Postdoctoral Scholar
>>>>>>>> Geological and Planetary Sciences
>>>>>>>> California Institute of Technology, Pasadena, CA
>>>>>>>> rjolivet at caltech.edu
>>>>>>>> +1 (626) 560 6356
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> _______________________________________________
>>>>>>>> CIG-SHORT mailing list
>>>>>>>> CIG-SHORT at geodynamics.org
>>>>>>>> http://geodynamics.org/cgi-bin/mailman/listinfo/cig-short
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> ------------------------------
>>>>>>>
>>>>>>> Message: 2
>>>>>>> Date: Wed, 29 Aug 2012 22:09:25 -0500
>>>>>>> From: Matthew Knepley <knepley at mcs.anl.gov>
>>>>>>> Subject: [CIG-SHORT] Fwd: Rate and State and Initial Traction
>>>>>>> To: cig-short at geodynamics.org
>>>>>>> Message-ID:
>>>>>>> <CAMYG4GkMcUqRjhMaYZ8Ooj-x=
>>>>>> 06QKFrc7fiwW1dStmDSDO1qWA at mail.gmail.com>
>>>>>>> Content-Type: text/plain; charset="iso-8859-1"
>>>>>>>
>>>>>>> On Wed, Aug 29, 2012 at 2:36 PM, Brad Aagaard <baagaard at usgs.gov>
>>>>>> wrote:
>>>>>>>
>>>>>>>> Romain,
>>>>>>>>
>>>>>>>> It looks like you are using Schur complement preconditioner
>>>>>>>> settings.
>>>>>>>> The jump in the KSP residual at iteration 10 is odd and suggests
>>>>>>>> something is not quite right in the linear solve (probably
>>>>>>>> related to
>>>>>>>>
>>>>>>>
>>>>>>> Brad is right that the tolerances here will produce some garbage.
>>>>>>> Going
>>>>>>> below
>>>>>>> machine epsilon (about 10^-16) means you get randomness and
>>>>>>> ultimately
>>>>>>> underflow which I think you got in a norm calculation since
>>>>>>> things get
>>>>>>> squared.
>>>>>>> The solver should have converged on iterate 2, and will with the
>>>>>> tolerances
>>>>>>> that
>>>>>>> Brad suggested.
>>>>>>>
>>>>>>> Thanks,
>>>>>>>
>>>>>>> Matt
>>>>>>>
>>>>>>>
>>>>>>>> the preconditioner). Try using the AMG preconditioner. It may not be
>>>>>>>> quite as fast but it is more robust. Matt can probably help you tune
>>>>>> the
>>>>>>>> Schur complement solver conditions some more if necessary.
>>>>>>>>
>>>>>>>> Your SNES tolerances are quite large. I suggest using tolerances
>>>>>> similar
>>>>>>>> to those in examples/3d/hex8/step14.cfg.
>>>>>>>>
>>>>>>>> SOLVER TOLERANCES
>>>>>>>>
>>>>>>>> ksp_rtol = 1.0e-16
>>>>>>>> ksp_atol = 1.0e-12
>>>>>>>>
>>>>>>>> snes_rtol = 1.0e-14
>>>>>>>> snes_atol = 1.0e-10
>>>>>>>>
>>>>>>>>
>>>>>>>> AMG PRECONDITIONER SETTINGS
>>>>>>>>
>>>>>>>> [pylithapp.timedependent.formulation]
>>>>>>>> split_fields = True
>>>>>>>> matrix_type = aij
>>>>>>>>
>>>>>>>> [pylithapp.petsc]
>>>>>>>> ksp_gmres_restart = 100
>>>>>>>> fs_pc_type = fieldsplit
>>>>>>>> fs_pc_fieldsplit_real_diagonal = true
>>>>>>>> fs_pc_fieldsplit_type = multiplicative
>>>>>>>> fs_fieldsplit_0_pc_type = ml
>>>>>>>> fs_fieldsplit_1_pc_type = jacobi
>>>>>>>> fs_fieldsplit_0_ksp_type = preonly
>>>>>>>> fs_fieldsplit_1_ksp_type = preonly
>>>>>>>>
>>>>>>>> Brad
>>>>>>>>
>>>>>>>>
>>>>>>>> On 08/29/2012 11:42 AM, Romain Jolivet wrote:
>>>>>>>>> Hello Brad, Matt, Charles and the pylith team,
>>>>>>>>>
>>>>>>>>> Following your advice from the other day, I am experimenting
>>>>>>>>> how to
>>>>>>>> create earthquakes on a frictional interface by applying a traction
>>>>>> change
>>>>>>>> history. It works fine with the SlipWeakening law in 2D. I'm
>>>>>>>> trying to
>>>>>> move
>>>>>>>> on to the RateStateAgeing behavior and I have problems applying the
>>>>>> initial
>>>>>>>> traction conditions:
>>>>>>>>>
>>>>>>>>> I use a SimpleDB to apply the frictional properties on the
>>>>>>>>> fault: one
>>>>>>>> domain is rate-strengthening, the other is rate-weakening with a
>>>>>>>> linear
>>>>>>>> transition between the two domains. If I do not try to turn on any
>>>>>>>> traction_perturbation option, the code runs great, and I have a slow
>>>>>> slip
>>>>>>>> all the along the fault at any time.
>>>>>>>>>
>>>>>>>>> As I want the rate-weakening domain to be stuck sometimes, I
>>>>>>>>> want to
>>>>>>>> turn on traction_perturbation.db_initial. I use a SimpleDB to put a
>>>>>>>> negative traction all along the fault, and a shear traction
>>>>>>>> compatible
>>>>>> with
>>>>>>>> the friction coefficients I impose. However, when I run this, I have
>>>>>> the
>>>>>>>> following error message from PETSc (I attach the log):
>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>
>>>>>> /Users/jolivetinsar/MYBIN/pylith/pylith-1.7.1-darwin-10.6.8/lib/python2.7/site-packages/pylith/problems/Implicit.py:215:step
>>>>>>>>> -- implicit(info)
>>>>>>>>> -- Solving equations.
>>>>>>>>> 0 SNES Function norm 2.679589636063e+01
>>>>>>>>> 0 KSP Residual norm 9.637565795900e-01
>>>>>>>>> 1 KSP Residual norm 3.564937135471e-08
>>>>>>>>> 2 KSP Residual norm 4.313963401911e-16
>>>>>>>>> 3 KSP Residual norm 3.050427683646e-16
>>>>>>>>> 4 KSP Residual norm 2.490663081181e-16
>>>>>>>>> 5 KSP Residual norm 2.156976950523e-16
>>>>>>>>> 6 KSP Residual norm 1.929258577738e-16
>>>>>>>>> 7 KSP Residual norm 1.761163907914e-16
>>>>>>>>> 8 KSP Residual norm 1.630520838230e-16
>>>>>>>>> 9 KSP Residual norm 1.525212509706e-16
>>>>>>>>> 10 KSP Residual norm 1.833325155789e-11
>>>>>>>>> 11 KSP Residual norm 7.039784754868e-23
>>>>>>>>> Linear solve converged due to CONVERGED_ATOL iterations 11
>>>>>>>>> [0]PETSC ERROR: --------------------- Error Message
>>>>>>>> ------------------------------------
>>>>>>>>> [0]PETSC ERROR: Floating point exception!
>>>>>>>>> [0]PETSC ERROR: Infinite or not-a-number generated in norm!
>>>>>>>>>
>>>>>>>>> If anyone has an idea about that error message....
>>>>>>>>>
>>>>>>>>> I think it has to do with some slip vector. If I set
>>>>>>>> friction.min_slip_rate=1e-1, this problem does not happen (the
>>>>>> nonlinear
>>>>>>>> solver does not converge, but at least, no crash). If I set the
>>>>>>>> normal
>>>>>>>> traction to be positive, it does not crash as well (and it does not
>>>>>>>> converge as well). I join to this message the cfg file I am
>>>>>>>> using for
>>>>>> that
>>>>>>>> experiment.
>>>>>>>>>
>>>>>>>>> Thank you
>>>>>>>>> Romain
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> Romain Jolivet
>>>>>>>>> Postdoctoral Scholar
>>>>>>>>> Geological and Planetary Sciences
>>>>>>>>> California Institute of Technology, Pasadena, CA
>>>>>>>>> rjolivet at caltech.edu
>>>>>>>>> +1 (626) 560 6356
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> _______________________________________________
>>>>>>>>> CIG-SHORT mailing list
>>>>>>>>> CIG-SHORT at geodynamics.org
>>>>>>>>> http://geodynamics.org/cgi-bin/mailman/listinfo/cig-short
>>>>>>>>>
>>>>>>>>
>>>>>>>> _______________________________________________
>>>>>>>> CIG-SHORT mailing list
>>>>>>>> CIG-SHORT at geodynamics.org
>>>>>>>> http://geodynamics.org/cgi-bin/mailman/listinfo/cig-short
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> --
>>>>>>> What most experimenters take for granted before they begin their
>>>>>>> experiments is infinitely more interesting than any results to which
>>>>>> their
>>>>>>> experiments lead.
>>>>>>> -- Norbert Wiener
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> --
>>>>>>> What most experimenters take for granted before they begin their
>>>>>>> experiments is infinitely more interesting than any results to which
>>>>>> their
>>>>>>> experiments lead.
>>>>>>> -- Norbert Wiener
>>>>>>> -------------- next part --------------
>>>>>>> An HTML attachment was scrubbed...
>>>>>>> URL:
>>>>>> http://geodynamics.org/pipermail/cig-short/attachments/20120829/9e91899c/attachment.html
>>>>>>>
>>>>>>> ------------------------------
>>>>>>>
>>>>>>> _______________________________________________
>>>>>>> CIG-SHORT mailing list
>>>>>>> CIG-SHORT at geodynamics.org
>>>>>>> http://geodynamics.org/cgi-bin/mailman/listinfo/cig-short
>>>>>>>
>>>>>>>
>>>>>>> End of CIG-SHORT Digest, Vol 72, Issue 21
>>>>>>> *****************************************
>>>>>>
>>>>>> _______________________________________________
>>>>>> CIG-SHORT mailing list
>>>>>> CIG-SHORT at geodynamics.org
>>>>>> http://geodynamics.org/cgi-bin/mailman/listinfo/cig-short
>>>>>>
>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> What most experimenters take for granted before they begin their
>>>>> experiments is infinitely more interesting than any results to
>>>>> which their
>>>>> experiments lead.
>>>>> -- Norbert Wiener
>>>>>
>>>>>
>>>>>
>>>>
>>>>
>>>> --
>>>> What most experimenters take for granted before they begin their
>>>> experiments is infinitely more interesting than any results to which
>>>> their
>>>> experiments lead.
>>>> -- Norbert Wiener
>>>> -------------- next part --------------
>>>> An HTML attachment was scrubbed...
>>>> URL:
>>>> http://geodynamics.org/pipermail/cig-short/attachments/20120830/964a474f/attachment.htm
>>>>
>>>> ------------------------------
>>>>
>>>> _______________________________________________
>>>> CIG-SHORT mailing list
>>>> CIG-SHORT at geodynamics.org
>>>> http://geodynamics.org/cgi-bin/mailman/listinfo/cig-short
>>>>
>>>>
>>>> End of CIG-SHORT Digest, Vol 72, Issue 25
>>>> *****************************************
>>>
>>> _______________________________________________
>>> CIG-SHORT mailing list
>>> CIG-SHORT at geodynamics.org
>>> http://geodynamics.org/cgi-bin/mailman/listinfo/cig-short
>>>
>>
>>
>>
>> ------------------------------
>>
>> _______________________________________________
>> CIG-SHORT mailing list
>> CIG-SHORT at geodynamics.org
>> http://geodynamics.org/cgi-bin/mailman/listinfo/cig-short
>>
>>
>> End of CIG-SHORT Digest, Vol 72, Issue 28
>> *****************************************
>
>
>
> _______________________________________________
> CIG-SHORT mailing list
> CIG-SHORT at geodynamics.org
> http://geodynamics.org/cgi-bin/mailman/listinfo/cig-short
>
More information about the CIG-SHORT
mailing list