[CIG-SHORT] fault opening with nonzero traction

Brad Aagaard baagaard at usgs.gov
Wed Dec 6 09:04:54 PST 2017


Here is the important part of the error message:
 > RuntimeError: WARNING! Fault opening with nonzero traction., v_fault:
 > 13506, opening: 1.01291e-10, normal traction: -0.00340407

See this post from Nov 28 on the cig-short email list (note the 
discussion of the zero_tolerance and zero_tolerance_normal parameters):
http://lists.geodynamics.org/pipermail/cig-short/2017-November/003068.html

The PETSC_ERROR stuff is the other processes calling abort as a result 
of one process aborting and notifying the others. The number in square 
brackets before PETSC_ERROR is the process number.

Regards,
Brad


On 12/06/2017 06:02 AM, Yasemin Korkusuz wrote:
> Dear Aagaard,
> 
> First, I am sorry for my late reply due to some problems. Thanks for 
> your quick response.
> 
> I generally run my cases for 200m and 300m resolutions because I could 
> not get results for 100m grid size. Also, I did not get error messages 
> for the stability conditions, as they are satisfied.
> I have copied an error message below this email. This is for a simple 
> around 120km long 2 segments fault. Both segments are vertical and the 
> angle between the segments is 10 degree. I attached a figure to 
> represent the geometry.
> 
> Does it related to the MPI or what are your suggestions?
> 
> Thanks  a lot
> Yasemin
> 
> 
> ----------------------------------------------------------------------
> ...
> -- Advancing solution from t=4.74*s to t=4.75*s.
>   >> 
> /home/yasemin/softwares/pylith/pylith-2.1.4-linux-x86_64/lib/python2.7/site-packages/pylith/problems/TimeDependent.py:208:run
>   -- timedependent(info)
>   -- Finishing advancing solution from t=4.74*s to t=4.75*s.
>   >> 
> /home/yasemin/softwares/pylith/pylith-2.1.4-linux-x86_64/lib/python2.7/site-packages/pylith/feassemble/IntegratorElasticity.py:125:writeData
>   -- elasticityexplicittet4(info)
>   -- Writing material data.
>   >> 
> /home/yasemin/softwares/pylith/pylith-2.1.4-linux-x86_64/lib/python2.7/site-packages/pylith/faults/Fault.py:191:writeData
>   -- faultcohesivedyn(info)
>   -- Writing fault data.
>   >> 
> /home/yasemin/softwares/pylith/pylith-2.1.4-linux-x86_64/lib/python2.7/site-packages/pylith/problems/TimeDependent.py:184:run
>   -- timedependent(info)
>   -- Main time loop, current time is t=4.75*s
>   >> 
> /home/yasemin/softwares/pylith/pylith-2.1.4-linux-x86_64/lib/python2.7/site-packages/pylith/problems/TimeDependent.py:195:run
>   -- timedependent(info)
>   -- Preparing to advance solution from time t=4.75*s to t=4.76*s.
>   >> 
> /home/yasemin/softwares/pylith/pylith-2.1.4-linux-x86_64/lib/python2.7/site-packages/pylith/problems/TimeDependent.py:201:run
>   -- timedependent(info)
>   -- Advancing solution from t=4.75*s to t=4.76*s.
> Fatal error. Calling MPI_Abort() to abort PyLith application.
> Traceback (most recent call last):
>    File 
> "/home/yasemin/softwares/pylith/pylith-2.1.4-linux-x86_64/lib/python2.7/site-packages/pylith/apps/PetscApplication.py", 
> line 74, in onComputeNodes
>      self.main(*args, **kwds)
>    File 
> "/home/yasemin/softwares/pylith/pylith-2.1.4-linux-x86_64/lib/python2.7/site-packages/pylith/apps/PyLithApp.py", 
> line 135, in main
>      self.problem.run(self)
>    File 
> "/home/yasemin/softwares/pylith/pylith-2.1.4-linux-x86_64/lib/python2.7/site-packages/pylith/problems/TimeDependent.py", 
> line 203, in run
>      self.formulation.step(t, dt)
>    File 
> "/home/yasemin/softwares/pylith/pylith-2.1.4-linux-x86_64/lib/python2.7/site-packages/pylith/problems/Explicit.py", 
> line 215, in step
>      self._reformResidual(t, dt)
>    File 
> "/home/yasemin/softwares/pylith/pylith-2.1.4-linux-x86_64/lib/python2.7/site-packages/pylith/problems/Formulation.py", 
> line 598, in _reformResidual
>      self.reformResidual()
>    File 
> "/home/yasemin/softwares/pylith/pylith-2.1.4-linux-x86_64/lib/python2.7/site-packages/pylith/problems/problems.py", 
> line 88, in reformResidual
>      def reformResidual(self, tmpResidualVec=None, 
> tmpSolveSolnVec=None): return _problems.Formulation_reformResidual(self, 
> tmpResidualVec, tmpSolveSolnVec)
> RuntimeError: WARNING! Fault opening with nonzero traction., v_fault: 
> 13506, opening: 1.01291e-10, normal traction: -0.00340407
> 
> application called MPI_Abort(MPI_COMM_WORLD, -1) - process 17
> Traceback (most recent call last):
>    File "<string>", line 1, in <module>
>    File 
> "/home/yasemin/softwares/pylith/pylith-2.1.4-linux-x86_64/lib/python2.7/site-packages/pythia-0.8.1.17-py2.7.egg/mpi/__init__.py", 
> line 42, in mpistart
>      kwds = kwds)
>    File 
> "/home/yasemin/softwares/pylith/pylith-2.1.4-linux-x86_64/lib/python2.7/site-packages/pythia-0.8.1.17-py2.7.egg/pyre/applications/__init__.py", 
> line 41, in start
>      shell.run(**kwds)
>    File 
> "/home/yasemin/softwares/pylith/pylith-2.1.4-linux-x86_64/lib/python2.7/site-packages/pythia-0.8.1.17-py2.7.egg/pyre/applications/Shell.py", 
> line 143, in run
>      method(*args, **kwds)
>    File 
> "/home/yasemin/softwares/pylith/pylith-2.1.4-linux-x86_64/lib/python2.7/site-packages/pythia-0.8.1.17-py2.7.egg/pyre/applications/SuperScript.py", 
> line 49, in execute
>      self.main(*args, **kwds)
>    File 
> "/home/yasemin/softwares/pylith/pylith-2.1.4-linux-x86_64/lib/python2.7/site-packages/pythia-0.8.1.17-py2.7.egg/pyre/applications/SuperScript.py", 
> line 58, in main
>      self.runSubscriTraceback (most recent call last):
>    File "<string>", line 1, in <module>
>    File 
> "/home/yasemin/softwares/pylith/pylith-2.1.4-linux-x86_64/lib/python2.7/site-packages/pythia-0.8.1.17-py2.7.egg/mpi/__init__.py", 
> line 42, in mpistart
>      kwds = kwds)
>    File 
> "/home/yasemin/softwares/pylith/pylith-2.1.4-linux-x86_64/lib/python2.7/site-packages/pythia-0.8.1.17-py2.7.egg/pyre/applications/__init__.py", 
> line 41, in start
>      shell.run(**kwds)
>    File 
> "/home/yasemin/softwares/pylith/pylith-2.1.4-linux-x86_64/lib/python2.7/site-packages/pythia-0.8.1.17-py2.7.egg/pyre/applications/Shell.py", 
> line 143, in run
>      method(*args, **kwds)
>    File 
> "/home/yasemin/softwares/pylith/pylith-2.1.4-linux-x86_64/lib/python2.7/site-packages/pythia-0.8.1.17-py2.7.egg/pyre/applications/SuperScript.py", 
> line 49, in execute
>      self.main(*args, **kwds)
>    File 
> "/home/yasemin/softwares/pylith/pylith-2.1.4-linux-x86_64/lib/python2.7/site-packages/pythia-0.8.1.17-py2.7.egg/pyre/applications/SuperScript.py", 
> line 58, in main
>      self.runSubscriTraceback (most recent call last):
>    File "<string>", line 1, in <module>
>    File 
> "/home/yasemin/softwares/pylith/pylith-2.1.4-linux-x86_64/lib/python2.7/site-packages/pythia-0.8.1.17-py2.7.egg/mpi/__init__.py", 
> line 42, in mpistart
>      kwds = kwds)
>    File 
> "/home/yasemin/softwares/pylith/pylith-2.1.4-linux-x86_64/lib/python2.7/site-packages/pythia-0.8.1.17-py2.7.egg/pyre/applications/__init__.py", 
> line 41, in start
>      shell.run(**kwds)
>    File 
> "/home/yasemin/softwares/pylith/pylith-2.1.4-linux-x86_64/lib/python2.7/site-packages/pythia-0.8.1.17-py2.7.egg/pyre/applications/Shell.py", 
> line 143, in run
>      method(*args, **kwds)
>    File 
> "/home/yasemin/softwares/pylith/pylith-2.1.4-linux-x86_64/lib/python2.7/site-packages/pythia-0.8.1.17-py2.7.egg/pyre/applications/SuperScript.py", 
> line 49, in execute
>      self.main(*args, **kwds)
>    File 
> "/home/yasemin/softwares/pylith/pylith-2.1.4-linux-x86_64/lib/python2.7/site-packages/pythia-0.8.1.17-py2.7.egg/pyre/applications/SuperScript.py", 
> line 58, in main
>      self.runSubscri[3]PETSC ERROR: 
> ------------------------------------------------------------------------
> [3]PETSC ERROR: Caught signal number 3 Quit: Some other process (or the 
> batch system) has told this process to end
> [3]PETSC ERROR: Try option -start_in_debugger or -on_error_attach_debugger
> [3]PETSC ERROR: or see 
> http://www.mcs.anl.gov/petsc/documentation/faq.html#valgrind
> [3]PETSC ERROR: or try http://valgrind.org on GNU/linux and Apple Mac OS 
> X to find memory corruption errors
> [3]PETSC ERROR: configure using --with-debugging=yes, recompile, link, 
> and run
> [3]PETSC ERROR: to get more information on the crash.
> [3]PETSC ERROR: --------------------- Error Message 
> --------------------------------------------------------------
> [3]PETSC ERROR: Signal received
> [3]PETSC ERROR: See http://www.mcs.anl.gov/petsc/documentation/faq.html 
> for trouble shooting.
> [3]PETSC ERROR: Petsc Development GIT revision: v3.7.2-876-g7e45237  GIT 
> Date: 2016-06-16 04:41:50 -0500
> [3]PETSC ERROR: 
> /home/yasemin/softwares/pylith/pylith-2.1.4-linux-x86_64/[4]PETSC ERROR: 
> ------------------------------------------------------------------------
> [4]PETSC ERROR: Caught signal number 3 Quit: Some other process (or the 
> batch system) has told this process to end
> [4]PETSC ERROR: Try option -start_in_debugger or -on_error_attach_debugger
> [4]PETSC ERROR: or see 
> http://www.mcs.anl.gov/petsc/documentation/faq.html#valgrind
> [4]PETSC ERROR: or try http://valgrind.org on GNU/linux and Apple Mac OS 
> X to find memory corruption errors
> [4]PETSC ERROR: configure using --with-debugging=yes, recompile, link, 
> and run
> [4]PETSC ERROR: to get more information on the crash.
> [4]PETSC ERROR: --------------------- Error Message 
> --------------------------------------------------------------
> [4]PETSC ERROR: Signal received
> [4]PETSC ERROR: See http://www.mcs.anl.gov/petsc/documentation/faq.html 
> for trouble shooting.
> [4]PETSC ERROR: Petsc Development GIT revision: v3.7.2-876-g7e45237  GIT 
> Date: 2016-06-16 04:41:50 -0500
> [4]PETSC ERROR: 
> /home/yasemin/softwares/pylith/pylith-2.1.4-linux-x86_64/[5]PETSC ERROR: 
> ------------------------------------------------------------------------
> [5]PETSC ERROR: Caught signal number 3 Quit: Some other process (or the 
> batch system) has told this process to end
> [5]PETSC ERROR: Try option -start_in_debugger or -on_error_attach_debugger
> [5]PETSC ERROR: or see 
> http://www.mcs.anl.gov/petsc/documentation/faq.html#valgrind
> [5]PETSC ERROR: or try http://valgrind.org on GNU/linux and Apple Mac OS 
> X to find memory corruption errors
> [5]PETSC ERROR: configure using --with-debugging=yes, recompile, link, 
> and run
> [5]PETSC ERROR: to get more information on the crash.
> [5]PETSC ERROR: --------------------- Error Message 
> --------------------------------------------------------------
> [5]PETSC ERROR: Signal received
> [5]PETSC ERROR: See http://www.mcs.anl.gov/petsc/documentation/faq.html 
> for trouble shooting.
> [5]PETSC ERROR: Petsc Development GIT revision: v3.7.2-876-g7e45237  GIT 
> Date: 2016-06-16 04:41:50 -0500
> [5]PETSC ERROR: 
> /home/yasemin/softwares/pylith/pylith-2.1.4-linux-x86_64/[6]PETSC ERROR: 
> ------------------------------------------------------------------------
> [6]PETSC ERROR: Caught signal number 3 Quit: Some other process (or the 
> batch system) has told this process to end
> [6]PETSC ERROR: Try option -start_in_debugger or -on_error_attach_debugger
> [6]PETSC ERROR: or see 
> http://www.mcs.anl.gov/petsc/documentation/faq.html#valgrind
> [6]PETSC ERROR: or try http://valgrind.org on GNU/linux and Apple Mac OS 
> X to find memory corruption errors
> [6]PETSC ERROR: configure using --with-debugging=yes, recompile, link, 
> and run
> [6]PETSC ERROR: to get more information on the crash.
> [6]PETSC ERROR: --------------------- Error Message 
> --------------------------------------------------------------
> [6]PETSC ERROR: Signal received
> [6]PETSC ERROR: See http://www.mcs.anl.gov/petsc/documentation/faq.html 
> for trouble shooting.
> [6]PETSC ERROR: Petsc Development GIT revision: v3.7.2-876-g7e45237  GIT 
> Date: 2016-06-16 04:41:50 -0500
> [6]PETSC ERROR: 
> /home/yasemin/softwares/pylith/pylith-2.1.4-linux-x86_64/[7]PETSC ERROR: 
> ------------------------------------------------------------------------
> [7]PETSC ERROR: Caught signal number 3 Quit: Some other process (or the 
> batch system) has told this process to end
> [7]PETSC ERROR: Try option -start_in_debugger or -on_error_attach_debugger
> [7]PETSC ERROR: or see 
> http://www.mcs.anl.gov/petsc/documentation/faq.html#valgrind
> [7]PETSC ERROR: or try http://valgrind.org on GNU/linux and Apple Mac OS 
> X to find memory corruption errors
> [7]PETSC ERROR: configure using --with-debugging=yes, recompile, link, 
> and run
> [7]PETSC ERROR: to get more information on the crash.
> [7]PETSC ERROR: --------------------- Error Message 
> --------------------------------------------------------------
> [7]PETSC ERROR: Signal received
> [7]PETSC ERROR: See http://www.mcs.anl.gov/petsc/documentation/faq.html 
> for trouble shooting.
> [7]PETSC ERROR: Petsc Development GIT revision: v3.7.2-876-g7e45237  GIT 
> Date: 2016-06-16 04:41:50 -0500
> [7]PETSC ERROR: 
> /home/yasemin/softwares/pylith/pylith-2.1.4-linux-x86_64/[8]PETSC ERROR: 
> ------------------------------------------------------------------------
> [8]PETSC ERROR: Caught signal number 3 Quit: Some other process (or the 
> batch system) has told this process to end
> [8]PETSC ERROR: Try option -start_in_debugger or -on_error_attach_debugger
> [8]PETSC ERROR: or see 
> http://www.mcs.anl.gov/petsc/documentation/faq.html#valgrind
> [8]PETSC ERROR: or try http://valgrind.org on GNU/linux and Apple Mac OS 
> X to find memory corruption errors
> [8]PETSC ERROR: configure using --with-debugging=yes, recompile, link, 
> and run
> [8]PETSC ERROR: to get more information on the crash.
> [8]PETSC ERROR: --------------------- Error Message 
> --------------------------------------------------------------
> [8]PETSC ERROR: Signal received
> [8]PETSC ERROR: See http://www.mcs.anl.gov/petsc/documentation/faq.html 
> for trouble shooting.
> [8]PETSC ERROR: Petsc Development GIT revision: v3.7.2-876-g7e45237  GIT 
> Date: 2016-06-16 04:41:50 -0500
> [8]PETSC ERROR: 
> /home/yasemin/softwares/pylith/pylith-2.1.4-linux-x86_64/[9]PETSC ERROR: 
> ------------------------------------------------------------------------
> [9]PETSC ERROR: Caught signal number 3 Quit: Some other process (or the 
> batch system) has told this process to end
> [9]PETSC ERROR: Try option -start_in_debugger or -on_error_attach_debugger
> [9]PETSC ERROR: or see 
> http://www.mcs.anl.gov/petsc/documentation/faq.html#valgrind
> [9]PETSC ERROR: or try http://valgrind.org on GNU/linux and Apple Mac OS 
> X to find memory corruption errors
> [9]PETSC ERROR: configure using --with-debugging=yes, recompile, link, 
> and run
> [9]PETSC ERROR: to get more information on the crash.
> [9]PETSC ERROR: --------------------- Error Message 
> --------------------------------------------------------------
> [9]PETSC ERROR: Signal received
> [9]PETSC ERROR: See http://www.mcs.anl.gov/petsc/documentation/faq.html 
> for trouble shooting.
> [9]PETSC ERROR: Petsc Development GIT revision: v3.7.2-876-g7e45237  GIT 
> Date: 2016-06-16 04:41:50 -0500
> [9]PETSC ERROR: 
> /home/yasemin/softwares/pylith/pylith-2.1.4-linux-x86_64/[10]PETSC 
> ERROR: 
> ------------------------------------------------------------------------
> [10]PETSC ERROR: Caught signal number 3 Quit: Some other process (or the 
> batch system) has told this process to end
> [10]PETSC ERROR: Try option -start_in_debugger or -on_error_attach_debugger
> [10]PETSC ERROR: or see 
> http://www.mcs.anl.gov/petsc/documentation/faq.html#valgrind
> [10]PETSC ERROR: or try http://valgrind.org on GNU/linux and Apple Mac 
> OS X to find memory corruption errors
> [10]PETSC ERROR: configure using --with-debugging=yes, recompile, link, 
> and run
> [10]PETSC ERROR: to get more information on the crash.
> [10]PETSC ERROR: --------------------- Error Message 
> --------------------------------------------------------------
> [10]PETSC ERROR: Signal received
> [10]PETSC ERROR: See http://www.mcs.anl.gov/petsc/documentation/faq.html 
> for trouble shooting.
> [10]PETSC ERROR: Petsc Development GIT revision: v3.7.2-876-g7e45237  
> GIT Date: 2016-06-16 04:41:50 -0500
> [10]PETSC ERROR: /home/yasemin/softwares/pylith/pylith-2.1.4-l[11]PETSC 
> ERROR: 
> ------------------------------------------------------------------------
> [11]PETSC ERROR: Caught signal number 3 Quit: Some other process (or the 
> batch system) has told this process to end
> [11]PETSC ERROR: Try option -start_in_debugger or -on_error_attach_debugger
> [11]PETSC ERROR: or see 
> http://www.mcs.anl.gov/petsc/documentation/faq.html#valgrind
> [11]PETSC ERROR: or try http://valgrind.org on GNU/linux and Apple Mac 
> OS X to find memory corruption errors
> [11]PETSC ERROR: configure using --with-debugging=yes, recompile, link, 
> and run
> [11]PETSC ERROR: to get more information on the crash.
> [11]PETSC ERROR: --------------------- Error Message 
> --------------------------------------------------------------
> [11]PETSC ERROR: Signal received
> [11]PETSC ERROR: See http://www.mcs.anl.gov/petsc/documentation/faq.html 
> for trouble shooting.
> [11]PETSC ERROR: Petsc Development GIT revision: v3.7.2-876-g7e45237  
> GIT Date: 2016-06-16 04:41:50 -0500
> [11]PETSC ERROR: /home/yasemin/softwares/pylith/pylith-2.1.4-l[12]PETSC 
> ERROR: 
> ------------------------------------------------------------------------
> [12]PETSC ERROR: Caught signal number 3 Quit: Some other process (or the 
> batch system) has told this process to end
> [12]PETSC ERROR: Try option -start_in_debugger or -on_error_attach_debugger
> [12]PETSC ERROR: or see 
> http://www.mcs.anl.gov/petsc/documentation/faq.html#valgrind
> [12]PETSC ERROR: or try http://valgrind.org on GNU/linux and Apple Mac 
> OS X to find memory corruption errors
> [12]PETSC ERROR: configure using --with-debugging=yes, recompile, link, 
> and run
> [12]PETSC ERROR: to get more information on the crash.
> [12]PETSC ERROR: --------------------- Error Message 
> --------------------------------------------------------------
> [12]PETSC ERROR: Signal received
> [12]PETSC ERROR: See http://www.mcs.anl.gov/petsc/documentation/faq.html 
> for trouble shooting.
> [12]PETSC ERROR: Petsc Development GIT revision: v3.7.2-876-g7e45237  
> GIT Date: 2016-06-16 04:41:50 -0500
> [12]PETSC ERROR: /home/yasemin/softwares/pylith/pylith-2.1.4-l[13]PETSC 
> ERROR: 
> ------------------------------------------------------------------------
> [13]PETSC ERROR: Caught signal number 3 Quit: Some other process (or the 
> batch system) has told this process to end
> [13]PETSC ERROR: Try option -start_in_debugger or -on_error_attach_debugger
> [13]PETSC ERROR: or see 
> http://www.mcs.anl.gov/petsc/documentation/faq.html#valgrind
> [13]PETSC ERROR: or try http://valgrind.org on GNU/linux and Apple Mac 
> OS X to find memory corruption errors
> [13]PETSC ERROR: configure using --with-debugging=yes, recompile, link, 
> and run
> [13]PETSC ERROR: to get more information on the crash.
> [13]PETSC ERROR: --------------------- Error Message 
> --------------------------------------------------------------
> [13]PETSC ERROR: Signal received
> [13]PETSC ERROR: See http://www.mcs.anl.gov/petsc/documentation/faq.html 
> for trouble shooting.
> [13]PETSC ERROR: Petsc Development GIT revision: v3.7.2-876-g7e45237  
> GIT Date: 2016-06-16 04:41:50 -0500
> [13]PETSC ERROR: /home/yasemin/softwares/pylith/pylith-2.1.4-l[14]PETSC 
> ERROR: 
> ------------------------------------------------------------------------
> [14]PETSC ERROR: Caught signal number 3 Quit: Some other process (or the 
> batch system) has told this process to end
> [14]PETSC ERROR: Try option -start_in_debugger or -on_error_attach_debugger
> [14]PETSC ERROR: or see 
> http://www.mcs.anl.gov/petsc/documentation/faq.html#valgrind
> [14]PETSC ERROR: or try http://valgrind.org on GNU/linux and Apple Mac 
> OS X to find memory corruption errors
> [14]PETSC ERROR: configure using --with-debugging=yes, recompile, link, 
> and run
> [14]PETSC ERROR: to get more information on the crash.
> [14]PETSC ERROR: --------------------- Error Message 
> --------------------------------------------------------------
> [14]PETSC ERROR: Signal received
> [14]PETSC ERROR: See http://www.mcs.anl.gov/petsc/documentation/faq.html 
> for trouble shooting.
> [14]PETSC ERROR: Petsc Development GIT revision: v3.7.2-876-g7e45237  
> GIT Date: 2016-06-16 04:41:50 -0500
> [14]PETSC ERROR: /home/yasemin/softwares/pylith/pylith-2.1.4-l[15]PETSC 
> ERROR: 
> ------------------------------------------------------------------------
> [15]PETSC ERROR: Caught signal number 3 Quit: Some other process (or the 
> batch system) has told this process to end
> [15]PETSC ERROR: Try option -start_in_debugger or -on_error_attach_debugger
> [15]PETSC ERROR: or see 
> http://www.mcs.anl.gov/petsc/documentation/faq.html#valgrind
> [15]PETSC ERROR: or try http://valgrind.org on GNU/linux and Apple Mac 
> OS X to find memory corruption errors
> [15]PETSC ERROR: configure using --with-debugging=yes, recompile, link, 
> and run
> [15]PETSC ERROR: to get more information on the crash.
> [15]PETSC ERROR: --------------------- Error Message 
> --------------------------------------------------------------
> [15]PETSC ERROR: Signal received
> [15]PETSC ERROR: See http://www.mcs.anl.gov/petsc/documentation/faq.html 
> for trouble shooting.
> [15]PETSC ERROR: Petsc Development GIT revision: v3.7.2-876-g7e45237  
> GIT Date: 2016-06-16 04:41:50 -0500
> [15]PETSC ERROR: /home/yasemin/softwares/pylith/pylith-2.1.4-l[16]PETSC 
> ERROR: 
> ------------------------------------------------------------------------
> [16]PETSC ERROR: Caught signal number 3 Quit: Some other process (or the 
> batch system) has told this process to end
> [16]PETSC ERROR: Try option -start_in_debugger or -on_error_attach_debugger
> [16]PETSC ERROR: or see 
> http://www.mcs.anl.gov/petsc/documentation/faq.html#valgrind
> [16]PETSC ERROR: or try http://valgrind.org on GNU/linux and Apple Mac 
> OS X to find memory corruption errors
> [16]PETSC ERROR: configure using --with-debugging=yes, recompile, link, 
> and run
> [16]PETSC ERROR: to get more information on the crash.
> [16]PETSC ERROR: --------------------- Error Message 
> --------------------------------------------------------------
> [16]PETSC ERROR: Signal received
> [16]PETSC ERROR: See http://www.mcs.anl.gov/petsc/documentation/faq.html 
> for trouble shooting.
> [16]PETSC ERROR: Petsc Development GIT revision: v3.7.2-876-g7e45237  
> GIT Date: 2016-06-16 04:41:50 -0500
> [16]PETSC ERROR: /home/yasemin/softwares/pylith/pylith-2.1.4-lpt(*args, 
> **kwds)
>    File 
> "/home/yasemin/softwares/pylith/pylith-2.1.4-linux-x86_64/lib/python2.7/site-packages/pythia-0.8.1.17-py2.7.egg/pyre/applications/SuperScript.py", 
> line 62, in runSubscript
>      self.subscript.run(*args, **kwds)
>    File 
> "/home/yasemin/softwares/pylith/pylith-2.1.4-linux-x86_64/lib/python2.7/site-packages/pythia-0.8.1.17-py2.7.egg/pyre/applications/Application.py", 
> line 42, in run
>      shell.run(*args, **kwds)
>    File 
> "/home/yasemin/softwares/pylith/pylith-2.1.4-linux-x86_64/lib/python2.7/site-packages/pythia-0.8.1.17-py2.7.egg/pyre/applications/Shell.py", 
> line 143, in run
>      method(*args, **kwds)
>    File 
> "/home/yasemin/softwares/pylith/pylith-2.1.4-linux-x86_64/lib/python2.7/site-packages/pythia-0.8.1.17-py2.7.egg/mpi/Application.py", 
> line 118, in _onComputeNodes
>      self.onComputeNodes(*args, **kwds)
>    File 
> "/home/yasemin/softwares/pylith/pylith-2.1.4-linux-x86_64/lib/python2.7/site-packages/pylith/apps/PetscApplication.py", 
> line 74, in onComputeNodes
>      self.main(*args, **kwds)
>    File "pt(*args, **kwds)
>    File 
> "/home/yasemin/softwares/pylith/pylith-2.1.4-linux-x86_64/lib/python2.7/site-packages/pythia-0.8.1.17-py2.7.egg/pyre/applications/SuperScript.py", 
> line 62, in runSubscript
>      self.subscript.run(*args, **kwds)
>    File 
> "/home/yasemin/softwares/pylith/pylith-2.1.4-linux-x86_64/lib/python2.7/site-packages/pythia-0.8.1.17-py2.7.egg/pyre/applications/Application.py", 
> line 42, in run
>      shell.run(*args, **kwds)
>    File 
> "/home/yasemin/softwares/pylith/pylith-2.1.4-linux-x86_64/lib/python2.7/site-packages/pythia-0.8.1.17-py2.7.egg/pyre/applications/Shell.py", 
> line 143, in run
>      method(*args, **kwds)
>    File 
> "/home/yasemin/softwares/pylith/pylith-2.1.4-linux-x86_64/lib/python2.7/site-packages/pythia-0.8.1.17-py2.7.egg/mpi/Application.py", 
> line 118, in _onComputeNodes
>      self.onComputeNodes(*args, **kwds)
>    File 
> "/home/yasemin/softwares/pylith/pylith-2.1.4-linux-x86_64/lib/python2.7/site-packages/pylith/apps/PetscApplication.py", 
> line 74, in onComputeNodes
>      self.main(*args, **kwds)
>    File "pt(*args, **kwds)
>    File 
> "/home/yasemin/softwares/pylith/pylith-2.1.4-linux-x86_64/lib/python2.7/site-packages/pythia-0.8.1.17-py2.7.egg/pyre/applications/SuperScript.py", 
> line 62, in runSubscript
>      self.subscript.run(*args, **kwds)
>    File 
> "/home/yasemin/softwares/pylith/pylith-2.1.4-linux-x86_64/lib/python2.7/site-packages/pythia-0.8.1.17-py2.7.egg/pyre/applications/Application.py", 
> line 42, in run
>      shell.run(*args, **kwds)
>    File 
> "/home/yasemin/softwares/pylith/pylith-2.1.4-linux-x86_64/lib/python2.7/site-packages/pythia-0.8.1.17-py2.7.egg/pyre/applications/Shell.py", 
> line 143, in run
>      method(*args, **kwds)
>    File 
> "/home/yasemin/softwares/pylith/pylith-2.1.4-linux-x86_64/lib/python2.7/site-packages/pythia-0.8.1.17-py2.7.egg/mpi/Application.py", 
> line 118, in _onComputeNodes
>      self.onComputeNodes(*args, **kwds)
>    File 
> "/home/yasemin/softwares/pylith/pylith-2.1.4-linux-x86_64/lib/python2.7/site-packages/pylith/apps/PetscApplication.py", 
> line 74, in onComputeNodes
>      self.main(*args, **kwds)
>    File "bin/mpinemesis on a arch-pylith named agri by yasemin Wed Dec  
> 6 16:34:15 2017
> [3]PETSC ERROR: Configure options --prefix=/home/brad/pylith-binary/dist 
> --with-c2html=0 --with-x=0 --with-clanguage=C --with-mpicompilers=1 
> --with-debugging=0 --with-shared-libraries=1 --with-64-bit-points=1 
> --with-large-file-io=1 --download-chaco=1 --download-ml=1 
> --download-f2cblaslapack=1 --with-hwloc=0 --with-ssl=0 --with-x=0 
> --with-c2html=0 --with-lgrind=0 --with-hdf5=1 
> --with-hdf5-dir=/home/brad/pylith-binary/dist --LIBS=-lz --with-fc=0 
> CPPFLAGS="-I/home/brad/pylith-binary/dist/include " 
> LDFLAGS="-L/home/brad/pylith-binary/dist/lib " CFLAGS="-g -O2" 
> CXXFLAGS="-g -O2 -DMPICH_IGNORE_CXX_SEEK" FCFLAGS= 
> PETSC_DIR=/home/brad/pylith-binary/build/petsc-pylith PETSC_ARCH=arch-pylith
> [3]PETSC ERROR: #1 User provided function() line 0 in  unknown file
> application called MPI_Abort(MPI_COMM_WORLD, 59) - process 3
> bin/mpinemesis on a arch-pylith named agri by yasemin Wed Dec  6 
> 16:34:15 2017
> [4]PETSC ERROR: Configure options --prefix=/home/brad/pylith-binary/dist 
> --with-c2html=0 --with-x=0 --with-clanguage=C --with-mpicompilers=1 
> --with-debugging=0 --with-shared-libraries=1 --with-64-bit-points=1 
> --with-large-file-io=1 --download-chaco=1 --download-ml=1 
> --download-f2cblaslapack=1 --with-hwloc=0 --with-ssl=0 --with-x=0 
> --with-c2html=0 --with-lgrind=0 --with-hdf5=1 
> --with-hdf5-dir=/home/brad/pylith-binary/dist --LIBS=-lz --with-fc=0 
> CPPFLAGS="-I/home/brad/pylith-binary/dist/include " 
> LDFLAGS="-L/home/brad/pylith-binary/dist/lib " CFLAGS="-g -O2" 
> CXXFLAGS="-g -O2 -DMPICH_IGNORE_CXX_SEEK" FCFLAGS= 
> PETSC_DIR=/home/brad/pylith-binary/build/petsc-pylith PETSC_ARCH=arch-pylith
> [4]PETSC ERROR: #1 User provided function() line 0 in  unknown file
> application called MPI_Abort(MPI_COMM_WORLD, 59) - process 4
> bin/mpinemesis on a arch-pylith named agri by yasemin Wed Dec  6 
> 16:34:15 2017
> [5]PETSC ERROR: Configure options --prefix=/home/brad/pylith-binary/dist 
> --with-c2html=0 --with-x=0 --with-clanguage=C --with-mpicompilers=1 
> --with-debugging=0 --with-shared-libraries=1 --with-64-bit-points=1 
> --with-large-file-io=1 --download-chaco=1 --download-ml=1 
> --download-f2cblaslapack=1 --with-hwloc=0 --with-ssl=0 --with-x=0 
> --with-c2html=0 --with-lgrind=0 --with-hdf5=1 
> --with-hdf5-dir=/home/brad/pylith-binary/dist --LIBS=-lz --with-fc=0 
> CPPFLAGS="-I/home/brad/pylith-binary/dist/include " 
> LDFLAGS="-L/home/brad/pylith-binary/dist/lib " CFLAGS="-g -O2" 
> CXXFLAGS="-g -O2 -DMPICH_IGNORE_CXX_SEEK" FCFLAGS= 
> PETSC_DIR=/home/brad/pylith-binary/build/petsc-pylith PETSC_ARCH=arch-pylith
> [5]PETSC ERROR: #1 User provided function() line 0 in  unknown file
> application called MPI_Abort(MPI_COMM_WORLD, 59) - process 5
> bin/mpinemesis on a arch-pylith named agri by yasemin Wed Dec  6 
> 16:34:15 2017
> [6]PETSC ERROR: Configure options --prefix=/home/brad/pylith-binary/dist 
> --with-c2html=0 --with-x=0 --with-clanguage=C --with-mpicompilers=1 
> --with-debugging=0 --with-shared-libraries=1 --with-64-bit-points=1 
> --with-large-file-io=1 --download-chaco=1 --download-ml=1 
> --download-f2cblaslapack=1 --with-hwloc=0 --with-ssl=0 --with-x=0 
> --with-c2html=0 --with-lgrind=0 --with-hdf5=1 
> --with-hdf5-dir=/home/brad/pylith-binary/dist --LIBS=-lz --with-fc=0 
> CPPFLAGS="-I/home/brad/pylith-binary/dist/include " 
> LDFLAGS="-L/home/brad/pylith-binary/dist/lib " CFLAGS="-g -O2" 
> CXXFLAGS="-g -O2 -DMPICH_IGNORE_CXX_SEEK" FCFLAGS= 
> PETSC_DIR=/home/brad/pylith-binary/build/petsc-pylith PETSC_ARCH=arch-pylith
> [6]PETSC ERROR: #1 User provided function() line 0 in  unknown file
> application called MPI_Abort(MPI_COMM_WORLD, 59) - process 6
> bin/mpinemesis on a arch-pylith named agri by yasemin Wed Dec  6 
> 16:34:15 2017
> [7]PETSC ERROR: Configure options --prefix=/home/brad/pylith-binary/dist 
> --with-c2html=0 --with-x=0 --with-clanguage=C --with-mpicompilers=1 
> --with-debugging=0 --with-shared-libraries=1 --with-64-bit-points=1 
> --with-large-file-io=1 --download-chaco=1 --download-ml=1 
> --download-f2cblaslapack=1 --with-hwloc=0 --with-ssl=0 --with-x=0 
> --with-c2html=0 --with-lgrind=0 --with-hdf5=1 
> --with-hdf5-dir=/home/brad/pylith-binary/dist --LIBS=-lz --with-fc=0 
> CPPFLAGS="-I/home/brad/pylith-binary/dist/include " 
> LDFLAGS="-L/home/brad/pylith-binary/dist/lib " CFLAGS="-g -O2" 
> CXXFLAGS="-g -O2 -DMPICH_IGNORE_CXX_SEEK" FCFLAGS= 
> PETSC_DIR=/home/brad/pylith-binary/build/petsc-pylith PETSC_ARCH=arch-pylith
> [7]PETSC ERROR: #1 User provided function() line 0 in  unknown file
> application called MPI_Abort(MPI_COMM_WORLD, 59) - process 7
> bin/mpinemesis on a arch-pylith named agri by yasemin Wed Dec  6 
> 16:34:15 2017
> [8]PETSC ERROR: Configure options --prefix=/home/brad/pylith-binary/dist 
> --with-c2html=0 --with-x=0 --with-clanguage=C --with-mpicompilers=1 
> --with-debugging=0 --with-shared-libraries=1 --with-64-bit-points=1 
> --with-large-file-io=1 --download-chaco=1 --download-ml=1 
> --download-f2cblaslapack=1 --with-hwloc=0 --with-ssl=0 --with-x=0 
> --with-c2html=0 --with-lgrind=0 --with-hdf5=1 
> --with-hdf5-dir=/home/brad/pylith-binary/dist --LIBS=-lz --with-fc=0 
> CPPFLAGS="-I/home/brad/pylith-binary/dist/include " 
> LDFLAGS="-L/home/brad/pylith-binary/dist/lib " CFLAGS="-g -O2" 
> CXXFLAGS="-g -O2 -DMPICH_IGNORE_CXX_SEEK" FCFLAGS= 
> PETSC_DIR=/home/brad/pylith-binary/build/petsc-pylith PETSC_ARCH=arch-pylith
> [8]PETSC ERROR: #1 User provided function() line 0 in  unknown file
> application called MPI_Abort(MPI_COMM_WORLD, 59) - process 8
> bin/mpinemesis on a arch-pylith named agri by yasemin Wed Dec  6 
> 16:34:15 2017
> [9]PETSC ERROR: Configure options --prefix=/home/brad/pylith-binary/dist 
> --with-c2html=0 --with-x=0 --with-clanguage=C --with-mpicompilers=1 
> --with-debugging=0 --with-shared-libraries=1 --with-64-bit-points=1 
> --with-large-file-io=1 --download-chaco=1 --download-ml=1 
> --download-f2cblaslapack=1 --with-hwloc=0 --with-ssl=0 --with-x=0 
> --with-c2html=0 --with-lgrind=0 --with-hdf5=1 
> --with-hdf5-dir=/home/brad/pylith-binary/dist --LIBS=-lz --with-fc=0 
> CPPFLAGS="-I/home/brad/pylith-binary/dist/include " 
> LDFLAGS="-L/home/brad/pylith-binary/dist/lib " CFLAGS="-g -O2" 
> CXXFLAGS="-g -O2 -DMPICH_IGNORE_CXX_SEEK" FCFLAGS= 
> PETSC_DIR=/home/brad/pylith-binary/build/petsc-pylith PETSC_ARCH=arch-pylith
> [9]PETSC ERROR: #1 User provided function() line 0 in  unknown file
> application called MPI_Abort(MPI_COMM_WORLD, 59) - process 9
> inux-x86_64/bin/mpinemesis on a arch-pylith named agri by yasemin Wed 
> Dec  6 16:34:15 2017
> [10]PETSC ERROR: Configure options 
> --prefix=/home/brad/pylith-binary/dist --with-c2html=0 --with-x=0 
> --with-clanguage=C --with-mpicompilers=1 --with-debugging=0 
> --with-shared-libraries=1 --with-64-bit-points=1 --with-large-file-io=1 
> --download-chaco=1 --download-ml=1 --download-f2cblaslapack=1 
> --with-hwloc=0 --with-ssl=0 --with-x=0 --with-c2html=0 --with-lgrind=0 
> --with-hdf5=1 --with-hdf5-dir=/home/brad/pylith-binary/dist --LIBS=-lz 
> --with-fc=0 CPPFLAGS="-I/home/brad/pylith-binary/dist/include " 
> LDFLAGS="-L/home/brad/pylith-binary/dist/lib " CFLAGS="-g -O2" 
> CXXFLAGS="-g -O2 -DMPICH_IGNORE_CXX_SEEK" FCFLAGS= 
> PETSC_DIR=/home/brad/pylith-binary/build/petsc-pylith PETSC_ARCH=arch-pylith
> [10]PETSC ERROR: #1 User provided function() line 0 in  unknown file
> application called MPI_Abort(MPI_COMM_WORLD, 59) - process 10
> inux-x86_64/bin/mpinemesis on a arch-pylith named agri by yasemin Wed 
> Dec  6 16:34:15 2017
> [11]PETSC ERROR: Configure options 
> --prefix=/home/brad/pylith-binary/dist --with-c2html=0 --with-x=0 
> --with-clanguage=C --with-mpicompilers=1 --with-debugging=0 
> --with-shared-libraries=1 --with-64-bit-points=1 --with-large-file-io=1 
> --download-chaco=1 --download-ml=1 --download-f2cblaslapack=1 
> --with-hwloc=0 --with-ssl=0 --with-x=0 --with-c2html=0 --with-lgrind=0 
> --with-hdf5=1 --with-hdf5-dir=/home/brad/pylith-binary/dist --LIBS=-lz 
> --with-fc=0 CPPFLAGS="-I/home/brad/pylith-binary/dist/include " 
> LDFLAGS="-L/home/brad/pylith-binary/dist/lib " CFLAGS="-g -O2" 
> CXXFLAGS="-g -O2 -DMPICH_IGNORE_CXX_SEEK" FCFLAGS= 
> PETSC_DIR=/home/brad/pylith-binary/build/petsc-pylith PETSC_ARCH=arch-pylith
> [11]PETSC ERROR: #1 User provided function() line 0 in  unknown file
> application called MPI_Abort(MPI_COMM_WORLD, 59) - process 11
> inux-x86_64/bin/mpinemesis on a arch-pylith named agri by yasemin Wed 
> Dec  6 16:34:15 2017
> [12]PETSC ERROR: Configure options 
> --prefix=/home/brad/pylith-binary/dist --with-c2html=0 --with-x=0 
> --with-clanguage=C --with-mpicompilers=1 --with-debugging=0 
> --with-shared-libraries=1 --with-64-bit-points=1 --with-large-file-io=1 
> --download-chaco=1 --download-ml=1 --download-f2cblaslapack=1 
> --with-hwloc=0 --with-ssl=0 --with-x=0 --with-c2html=0 --with-lgrind=0 
> --with-hdf5=1 --with-hdf5-dir=/home/brad/pylith-binary/dist --LIBS=-lz 
> --with-fc=0 CPPFLAGS="-I/home/brad/pylith-binary/dist/include " 
> LDFLAGS="-L/home/brad/pylith-binary/dist/lib " CFLAGS="-g -O2" 
> CXXFLAGS="-g -O2 -DMPICH_IGNORE_CXX_SEEK" FCFLAGS= 
> PETSC_DIR=/home/brad/pylith-binary/build/petsc-pylith PETSC_ARCH=arch-pylith
> [12]PETSC ERROR: #1 User provided function() line 0 in  unknown file
> application called MPI_Abort(MPI_COMM_WORLD, 59) - process 12
> inux-x86_64/bin/mpinemesis on a arch-pylith named agri by yasemin Wed 
> Dec  6 16:34:15 2017
> [13]PETSC ERROR: Configure options 
> --prefix=/home/brad/pylith-binary/dist --with-c2html=0 --with-x=0 
> --with-clanguage=C --with-mpicompilers=1 --with-debugging=0 
> --with-shared-libraries=1 --with-64-bit-points=1 --with-large-file-io=1 
> --download-chaco=1 --download-ml=1 --download-f2cblaslapack=1 
> --with-hwloc=0 --with-ssl=0 --with-x=0 --with-c2html=0 --with-lgrind=0 
> --with-hdf5=1 --with-hdf5-dir=/home/brad/pylith-binary/dist --LIBS=-lz 
> --with-fc=0 CPPFLAGS="-I/home/brad/pylith-binary/dist/include " 
> LDFLAGS="-L/home/brad/pylith-binary/dist/lib " CFLAGS="-g -O2" 
> CXXFLAGS="-g -O2 -DMPICH_IGNORE_CXX_SEEK" FCFLAGS= 
> PETSC_DIR=/home/brad/pylith-binary/build/petsc-pylith PETSC_ARCH=arch-pylith
> [13]PETSC ERROR: #1 User provided function() line 0 in  unknown file
> application called MPI_Abort(MPI_COMM_WORLD, 59) - process 13
> inux-x86_64/bin/mpinemesis on a arch-pylith named agri by yasemin Wed 
> Dec  6 16:34:15 2017
> [14]PETSC ERROR: Configure options 
> --prefix=/home/brad/pylith-binary/dist --with-c2html=0 --with-x=0 
> --with-clanguage=C --with-mpicompilers=1 --with-debugging=0 
> --with-shared-libraries=1 --with-64-bit-points=1 --with-large-file-io=1 
> --download-chaco=1 --download-ml=1 --download-f2cblaslapack=1 
> --with-hwloc=0 --with-ssl=0 --with-x=0 --with-c2html=0 --with-lgrind=0 
> --with-hdf5=1 --with-hdf5-dir=/home/brad/pylith-binary/dist --LIBS=-lz 
> --with-fc=0 CPPFLAGS="-I/home/brad/pylith-binary/dist/include " 
> LDFLAGS="-L/home/brad/pylith-binary/dist/lib " CFLAGS="-g -O2" 
> CXXFLAGS="-g -O2 -DMPICH_IGNORE_CXX_SEEK" FCFLAGS= 
> PETSC_DIR=/home/brad/pylith-binary/build/petsc-pylith PETSC_ARCH=arch-pylith
> [14]PETSC ERROR: #1 User provided function() line 0 in  unknown file
> application called MPI_Abort(MPI_COMM_WORLD, 59) - process 14
> inux-x86_64/bin/mpinemesis on a arch-pylith named agri by yasemin Wed 
> Dec  6 16:34:15 2017
> [15]PETSC ERROR: Configure options 
> --prefix=/home/brad/pylith-binary/dist --with-c2html=0 --with-x=0 
> --with-clanguage=C --with-mpicompilers=1 --with-debugging=0 
> --with-shared-libraries=1 --with-64-bit-points=1 --with-large-file-io=1 
> --download-chaco=1 --download-ml=1 --download-f2cblaslapack=1 
> --with-hwloc=0 --with-ssl=0 --with-x=0 --with-c2html=0 --with-lgrind=0 
> --with-hdf5=1 --with-hdf5-dir=/home/brad/pylith-binary/dist --LIBS=-lz 
> --with-fc=0 CPPFLAGS="-I/home/brad/pylith-binary/dist/include " 
> LDFLAGS="-L/home/brad/pylith-binary/dist/lib " CFLAGS="-g -O2" 
> CXXFLAGS="-g -O2 -DMPICH_IGNORE_CXX_SEEK" FCFLAGS= 
> PETSC_DIR=/home/brad/pylith-binary/build/petsc-pylith PETSC_ARCH=arch-pylith
> [15]PETSC ERROR: #1 User provided function() line 0 in  unknown file
> application called MPI_Abort(MPI_COMM_WORLD, 59) - process 15
> inux-x86_64/bin/mpinemesis on a arch-pylith named agri by yasemin Wed 
> Dec  6 16:34:15 2017
> [16]PETSC ERROR: Configure options 
> --prefix=/home/brad/pylith-binary/dist --with-c2html=0 --with-x=0 
> --with-clanguage=C --with-mpicompilers=1 --with-debugging=0 
> --with-shared-libraries=1 --with-64-bit-points=1 --with-large-file-io=1 
> --download-chaco=1 --download-ml=1 --download-f2cblaslapack=1 
> --with-hwloc=0 --with-ssl=0 --with-x=0 --with-c2html=0 --with-lgrind=0 
> --with-hdf5=1 --with-hdf5-dir=/home/brad/pylith-binary/dist --LIBS=-lz 
> --with-fc=0 CPPFLAGS="-I/home/brad/pylith-binary/dist/include " 
> LDFLAGS="-L/home/brad/pylith-binary/dist/lib " CFLAGS="-g -O2" 
> CXXFLAGS="-g -O2 -DMPICH_IGNORE_CXX_SEEK" FCFLAGS= 
> PETSC_DIR=/home/brad/pylith-binary/build/petsc-pylith PETSC_ARCH=arch-pylith
> [16]PETSC ERROR: #1 User provided function() line 0 in  unknown file
> application called MPI_Abort(MPI_COMM_WORLD, 59) - process 16
> /home/yasemin/softwares/pylith/pylith-2.1.4-linux-x86_64/lib/python2.7/site-packages/pylith/apps/PyLithApp.py", 
> line 135, in main
>      self.problem.run(self)
>    File 
> "/home/yasemin/softwares/pylith/pylith-2.1.4-linux-x86_64/lib/python2.7/site-packages/pylith/problems/TimeDependent.py", 
> line 203, in run
>      self.formulation.step(t, dt)
>    File 
> "/home/yasemin/softwares/pylith/pylith-2.1.4-linux-x86_64/lib/python2.7/site-packages/pylith/problems/Explicit.py", 
> line 215, in step
>      self._reformResidual(t, dt)
>    File 
> "/home/yasemin/softwares/pylith/pylith-2.1.4-linux-x86_64/lib/python2.7/site-packages/pylith/problems/Formulation.py", 
> line 598, in _reformResidual
>      self.reformResidual()
>    File 
> "/home/yasemin/softwares/pylith/pylith-2.1.4-linux-x86_64/lib/python2.7/site-packages/pylith/problems/problems.py", 
> line 88, in reformResidual
>      def reformResidual(self, tmpResidualVec=None, 
> tmpSolveSolnVec=None): return _problems.Formulation_reformResidual(self, 
> tmpResidualVec, tmpSolveSolnVec)
> KeyboardInterrupt
> [0]PETSC 
> ER/home/yasemin/softwares/pylith/pylith-2.1.4-linux-x86_64/lib/python2.7/site-packages/pylith/apps/PyLithApp.py", 
> line 135, in main
>      self.problem.run(self)
>    File 
> "/home/yasemin/softwares/pylith/pylith-2.1.4-linux-x86_64/lib/python2.7/site-packages/pylith/problems/TimeDependent.py", 
> line 203, in run
>      self.formulation.step(t, dt)
>    File 
> "/home/yasemin/softwares/pylith/pylith-2.1.4-linux-x86_64/lib/python2.7/site-packages/pylith/problems/Explicit.py", 
> line 215, in step
>      self._reformResidual(t, dt)
>    File 
> "/home/yasemin/softwares/pylith/pylith-2.1.4-linux-x86_64/lib/python2.7/site-packages/pylith/problems/Formulation.py", 
> line 598, in _reformResidual
>      self.reformResidual()
>    File 
> "/home/yasemin/softwares/pylith/pylith-2.1.4-linux-x86_64/lib/python2.7/site-packages/pylith/problems/problems.py", 
> line 88, in reformResidual
>      def reformResidual(self, tmpResidualVec=None, 
> tmpSolveSolnVec=None): return _problems.Formulation_reformResidual(self, 
> tmpResidualVec, tmpSolveSolnVec)
> KeyboardInterrupt
> [1]PETSC 
> ER/home/yasemin/softwares/pylith/pylith-2.1.4-linux-x86_64/lib/python2.7/site-packages/pylith/apps/PyLithApp.py", 
> line 135, in main
>      self.problem.run(self)
>    File 
> "/home/yasemin/softwares/pylith/pylith-2.1.4-linux-x86_64/lib/python2.7/site-packages/pylith/problems/TimeDependent.py", 
> line 203, in run
>      self.formulation.step(t, dt)
>    File 
> "/home/yasemin/softwares/pylith/pylith-2.1.4-linux-x86_64/lib/python2.7/site-packages/pylith/problems/Explicit.py", 
> line 215, in step
>      self._reformResidual(t, dt)
>    File 
> "/home/yasemin/softwares/pylith/pylith-2.1.4-linux-x86_64/lib/python2.7/site-packages/pylith/problems/Formulation.py", 
> line 598, in _reformResidual
>      self.reformResidual()
>    File 
> "/home/yasemin/softwares/pylith/pylith-2.1.4-linux-x86_64/lib/python2.7/site-packages/pylith/problems/problems.py", 
> line 88, in reformResidual
>      def reformResidual(self, tmpResidualVec=None, 
> tmpSolveSolnVec=None): return _problems.Formulation_reformResidual(self, 
> tmpResidualVec, tmpSolveSolnVec)
> KeyboardInterrupt
> [2]PETSC ERROR: 
> ------------------------------------------------------------------------
> [0]PETSC ERROR: Caught signal number 3 Quit: Some other process (or the 
> batch system) has told this process to end
> [0]PETSC ERROR: Try option -start_in_debugger or -on_error_attach_debugger
> [0]PETSC ERROR: or see 
> http://www.mcs.anl.gov/petsc/documentation/faq.html#valgrind
> [0]PETSC ERROR: or try http://valgrind.org on GNU/linux and Apple Mac OS 
> X to find memory corruption errors
> [0]PETSC ERROR: configure using --with-debugging=yes, recompile, link, 
> and run
> [0]PETSC ERROR: to get more information on the crash.
> [0]PETSC ERROR: --------------------- Error Message 
> --------------------------------------------------------------
> [0]PETSC ERROR: Signal received
> [0]PETSC ERROR: See http://www.mcs.anl.gov/petsc/documentation/faq.html 
> for trouble shooting.
> [0]PETSC ERROR: Petsc Development GIT revision: v3.7.2-876-g7e45237  GIT 
> Date: 2016-06-16 04:41:50 -0500
> [0]PETSC ERROR: 
> /home/yasemin/softwares/pylith/pylith-2.1.4-linux-x86_64/bin/mpinemeROR: 
> ------------------------------------------------------------------------
> [1]PETSC ERROR: Caught signal number 3 Quit: Some other process (or the 
> batch system) has told this process to end
> [1]PETSC ERROR: Try option -start_in_debugger or -on_error_attach_debugger
> [1]PETSC ERROR: or see 
> http://www.mcs.anl.gov/petsc/documentation/faq.html#valgrind
> [1]PETSC ERROR: or try http://valgrind.org on GNU/linux and Apple Mac OS 
> X to find memory corruption errors
> [1]PETSC ERROR: configure using --with-debugging=yes, recompile, link, 
> and run
> [1]PETSC ERROR: to get more information on the crash.
> [1]PETSC ERROR: --------------------- Error Message 
> --------------------------------------------------------------
> [1]PETSC ERROR: Signal received
> [1]PETSC ERROR: See http://www.mcs.anl.gov/petsc/documentation/faq.html 
> for trouble shooting.
> [1]PETSC ERROR: Petsc Development GIT revision: v3.7.2-876-g7e45237  GIT 
> Date: 2016-06-16 04:41:50 -0500
> [1]PETSC ERROR: 
> /home/yasemin/softwares/pylith/pylith-2.1.4-linux-x86_64/bin/mpinemeROR: 
> ------------------------------------------------------------------------
> [2]PETSC ERROR: Caught signal number 3 Quit: Some other process (or the 
> batch system) has told this process to end
> [2]PETSC ERROR: Try option -start_in_debugger or -on_error_attach_debugger
> [2]PETSC ERROR: or see 
> http://www.mcs.anl.gov/petsc/documentation/faq.html#valgrind
> [2]PETSC ERROR: or try http://valgrind.org on GNU/linux and Apple Mac OS 
> X to find memory corruption errors
> [2]PETSC ERROR: configure using --with-debugging=yes, recompile, link, 
> and run
> [2]PETSC ERROR: to get more information on the crash.
> [2]PETSC ERROR: --------------------- Error Message 
> --------------------------------------------------------------
> [2]PETSC ERROR: Signal received
> [2]PETSC ERROR: See http://www.mcs.anl.gov/petsc/documentation/faq.html 
> for trouble shooting.
> [2]PETSC ERROR: Petsc Development GIT revision: v3.7.2-876-g7e45237  GIT 
> Date: 2016-06-16 04:41:50 -0500
> [2]PETSC ERROR: 
> /home/yasemin/softwares/pylith/pylith-2.1.4-linux-x86_64/bin/mpinemesis 
> on a arch-pylith named agri by yasemin Wed Dec  6 16:34:15 2017
> [0]PETSC ERROR: Configure options --prefix=/home/brad/pylith-binary/dist 
> --with-c2html=0 --with-x=0 --with-clanguage=C --with-mpicompilers=1 
> --with-debugging=0 --with-shared-libraries=1 --with-64-bit-points=1 
> --with-large-file-io=1 --download-chaco=1 --download-ml=1 
> --download-f2cblaslapack=1 --with-hwloc=0 --with-ssl=0 --with-x=0 
> --with-c2html=0 --with-lgrind=0 --with-hdf5=1 
> --with-hdf5-dir=/home/brad/pylith-binary/dist --LIBS=-lz --with-fc=0 
> CPPFLAGS="-I/home/brad/pylith-binary/dist/include " 
> LDFLAGS="-L/home/brad/pylith-binary/dist/lib " CFLAGS="-g -O2" 
> CXXFLAGS="-g -O2 -DMPICH_IGNORE_CXX_SEEK" FCFLAGS= 
> PETSC_DIR=/home/brad/pylith-binary/build/petsc-pylith PETSC_ARCH=arch-pylith
> [0]PETSC ERROR: #1 User provided function() line 0 in  unknown file
> application called MPI_Abort(MPI_COMM_WORLD, 59) - process 0
> sis on a arch-pylith named agri by yasemin Wed Dec  6 16:34:15 2017
> [1]PETSC ERROR: Configure options --prefix=/home/brad/pylith-binary/dist 
> --with-c2html=0 --with-x=0 --with-clanguage=C --with-mpicompilers=1 
> --with-debugging=0 --with-shared-libraries=1 --with-64-bit-points=1 
> --with-large-file-io=1 --download-chaco=1 --download-ml=1 
> --download-f2cblaslapack=1 --with-hwloc=0 --with-ssl=0 --with-x=0 
> --with-c2html=0 --with-lgrind=0 --with-hdf5=1 
> --with-hdf5-dir=/home/brad/pylith-binary/dist --LIBS=-lz --with-fc=0 
> CPPFLAGS="-I/home/brad/pylith-binary/dist/include " 
> LDFLAGS="-L/home/brad/pylith-binary/dist/lib " CFLAGS="-g -O2" 
> CXXFLAGS="-g -O2 -DMPICH_IGNORE_CXX_SEEK" FCFLAGS= 
> PETSC_DIR=/home/brad/pylith-binary/build/petsc-pylith PETSC_ARCH=arch-pylith
> [1]PETSC ERROR: #1 User provided function() line 0 in  unknown file
> application called MPI_Abort(MPI_COMM_WORLD, 59) - process 1
> sis on a arch-pylith named agri by yasemin Wed Dec  6 16:34:15 2017
> [2]PETSC ERROR: Configure options --prefix=/home/brad/pylith-binary/dist 
> --with-c2html=0 --with-x=0 --with-clanguage=C --with-mpicompilers=1 
> --with-debugging=0 --with-shared-libraries=1 --with-64-bit-points=1 
> --with-large-file-io=1 --download-chaco=1 --download-ml=1 
> --download-f2cblaslapack=1 --with-hwloc=0 --with-ssl=0 --with-x=0 
> --with-c2html=0 --with-lgrind=0 --with-hdf5=1 
> --with-hdf5-dir=/home/brad/pylith-binary/dist --LIBS=-lz --with-fc=0 
> CPPFLAGS="-I/home/brad/pylith-binary/dist/include " 
> LDFLAGS="-L/home/brad/pylith-binary/dist/lib " CFLAGS="-g -O2" 
> CXXFLAGS="-g -O2 -DMPICH_IGNORE_CXX_SEEK" FCFLAGS= 
> PETSC_DIR=/home/brad/pylith-binary/build/petsc-pylith PETSC_ARCH=arch-pylith
> [2]PETSC ERROR: #1 User provided function() line 0 in  unknown file
> application called MPI_Abort(MPI_COMM_WORLD, 59) - process 2
> /home/yasemin/softwares/pylith/pylith-2.1.4-linux-x86_64/bin/nemesis: 
> mpirun: exit 255
> /home/yasemin/softwares/pylith/pylith-2.1.4-linux-x86_64/bin/pylith: 
> /home/yasemin/softwares/pylith/pylith-2.1.4-linux-x86_64/bin/nemesis: exit 1
> fearless#
> 
> 




More information about the CIG-SHORT mailing list