[CIG-SHORT] pylith multiprocessor
Matthew Knepley
knepley at gmail.com
Tue Oct 23 11:50:10 PDT 2007
On 10/23/07, Christopher DiCaprio <dicaprio at gps.caltech.edu> wrote:
> Brad,
>
> That doesn't seem to fix it:
> ------------------------------------------------------------------------
> ------------------------------------------------------------------------
> ---------------------------
> > pylith --nodes=2 dislocation.cfg mymachines.cfg
> invalid mpiexec argument -machinefile
> Usage: mpiexec -usize <universesize> -maxtime <seconds> -exitinfo -l\
> -n <numprocs> -soft <softness> -host <hostname> \
> -wdir <working directory> -path <search path> \
> -file <filename> -configfile <filename> \
> -genvnone -genvlist <name1,name2,...> -genv name value\
> -envnone -envlist <name1,name2,...> -env name value\
> execname <args>\
> [ : -n <numprocs> ... execname <args>]
>
> --pyre-start: mpiexec: exit 255
> /home/dicaprio/local/pylith-1.0.1-linux-i686/bin/pylith: /home/
> dicaprio/local/pylith-1.0.1-linux-i686/bin/nemesis: exit 1
> ------------------------------------------------------------------------
> ------------------------------------------------------------------------
> ---------------------------
>
>
> This actually stems from another problem. I was trying to run a
> large mesh (1110132 elements) and I got this error:
> ------------------------------------------------------------------------
> ------------------------------------------------------------------------
> ---------------------------
> n> pylith dislocation.cfg
> >> /home/buildbot/install/x86_linux_single_nosched_binbot/lib/
> python2.5/site-packages/pylith/utils/PetscManager.py:47:initialize
> -- petsc(info)
> -- Initializing PETSc.
> >> /home/buildbot/install/x86_linux_single_nosched_binbot/lib/
> python2.5/site-packages/pylith/meshio/MeshIO.py:45:read
> -- meshiolagrit(info)
> -- Reading finite-element mesh
> >> /home/buildbot/install/x86_linux_single_nosched_binbot/lib/
> python2.5/site-packages/pylith/problems/TimeDependent.py:92:initialize
> -- timedependent(info)
> -- Initializing problem.
> >> /home/buildbot/install/x86_linux_single_nosched_binbot/lib/
> python2.5/site-packages/pylith/problems/Formulation.py:89:initialize
> -- implicit(info)
> -- Initializing materials.
> >> /home/buildbot/install/x86_linux_single_nosched_binbot/lib/
> python2.5/site-packages/pylith/problems/Formulation.py:106:initialize
> -- implicit(info)
> -- Added elasticity integrator for material 'Elastic material -x'.
> >> /home/buildbot/install/x86_linux_single_nosched_binbot/lib/
> python2.5/site-packages/pylith/problems/Formulation.py:106:initialize
> -- implicit(info)
> -- Added elasticity integrator for material 'Elastic material +x'.
> >> /home/buildbot/install/x86_linux_single_nosched_binbot/lib/
> python2.5/site-packages/pylith/problems/Formulation.py:106:initialize
> -- implicit(info)
> -- Added elasticity integrator for material 'Viscoelastic material -x'.
> >> /home/buildbot/install/x86_linux_single_nosched_binbot/lib/
> python2.5/site-packages/pylith/problems/Formulation.py:106:initialize
> -- implicit(info)
> -- Added elasticity integrator for material 'Viscoelastic material +x'.
> >> /home/buildbot/install/x86_linux_single_nosched_binbot/lib/
> python2.5/site-packages/pylith/problems/Formulation.py:106:initialize
> -- implicit(info)
> -- Added elasticity integrator for material 'Viscoelastic material -x'.
> >> /home/buildbot/install/x86_linux_single_nosched_binbot/lib/
> python2.5/site-packages/pylith/problems/Formulation.py:108:initialize
> -- implicit(info)
> -- Initializing boundary conditions.
> >> /home/buildbot/install/x86_linux_single_nosched_binbot/lib/
> python2.5/site-packages/pylith/problems/Formulation.py:121:initialize
> -- implicit(info)
> -- Added boundary condition 'right' as a constraint.
> >> /home/buildbot/install/x86_linux_single_nosched_binbot/lib/
> python2.5/site-packages/pylith/problems/Formulation.py:121:initialize
> -- implicit(info)
> -- Added boundary condition 'left' as a constraint.
> >> /home/buildbot/install/x86_linux_single_nosched_binbot/lib/
> python2.5/site-packages/pylith/problems/Formulation.py:121:initialize
> -- implicit(info)
> -- Added boundary condition '' as a constraint.
> >> /home/buildbot/install/x86_linux_single_nosched_binbot/lib/
> python2.5/site-packages/pylith/problems/Formulation.py:121:initialize
> -- implicit(info)
> -- Added boundary condition '' as a constraint.
> >> /home/buildbot/install/x86_linux_single_nosched_binbot/lib/
> python2.5/site-packages/pylith/problems/Formulation.py:121:initialize
> -- implicit(info)
> -- Added boundary condition '' as a constraint.
> >> /home/buildbot/install/x86_linux_single_nosched_binbot/lib/
> python2.5/site-packages/pylith/problems/Formulation.py:121:initialize
> -- implicit(info)
> -- Added boundary condition '' as a constraint.
> >> /home/buildbot/install/x86_linux_single_nosched_binbot/lib/
> python2.5/site-packages/pylith/problems/Formulation.py:127:initialize
> -- implicit(info)
> -- Initializing interior interfaces.
> >> /home/buildbot/install/x86_linux_single_nosched_binbot/lib/
> python2.5/site-packages/pylith/faults/FaultCohesiveKin.py:70:initialize
> -- faultcohesivekin(info)
> -- Initializing fault 'pfb1b2'.
> Traceback (most recent call last):
> File "<string>", line 1, in <module>
> File "/home/buildbot/install/x86_linux_single_nosched_binbot/lib/
> python2.5/site-packages/pythia-0.8.1.4-py2.5.egg/mpi/__init__.py",
> line 42, in mpistart
> File "/home/buildbot/install/x86_linux_single_nosched_binbot/lib/
> python2.5/site-packages/pythia-0.8.1.4-py2.5.egg/pyre/applications/
> __init__.py", line 41, in start
> File "/home/buildbot/install/x86_linux_single_nosched_binbot/lib/
> python2.5/site-packages/pythia-0.8.1.4-py2.5.egg/pyre/applications/
> Shell.py", line 139, in run
> File "/home/buildbot/install/x86_linux_single_nosched_binbot/lib/
> python2.5/site-packages/pythia-0.8.1.4-py2.5.egg/pyre/applications/
> SuperScript.py", line 49, in execute
> File "/home/buildbot/install/x86_linux_single_nosched_binbot/lib/
> python2.5/site-packages/pythia-0.8.1.4-py2.5.egg/pyre/applications/
> SuperScript.py", line 58, in main
> File "/home/buildbot/install/x86_linux_single_nosched_binbot/lib/
> python2.5/site-packages/pythia-0.8.1.4-py2.5.egg/pyre/applications/
> SuperScript.py", line 62, in runSubscript
> File "/home/buildbot/install/x86_linux_single_nosched_binbot/lib/
> python2.5/site-packages/pythia-0.8.1.4-py2.5.egg/pyre/applications/
> Application.py", line 42, in run
> File "/home/buildbot/install/x86_linux_single_nosched_binbot/lib/
> python2.5/site-packages/pythia-0.8.1.4-py2.5.egg/pyre/applications/
> Shell.py", line 139, in run
> File "/home/buildbot/install/x86_linux_single_nosched_binbot/lib/
> python2.5/site-packages/pythia-0.8.1.4-py2.5.egg/mpi/Application.py",
> line 118, in _onComputeNodes
> File "/home/buildbot/install/x86_linux_single_nosched_binbot/lib/
> python2.5/site-packages/pythia-0.8.1.4-py2.5.egg/mpi/Application.py",
> line 124, in onComputeNodes
> File "/home/buildbot/install/x86_linux_single_nosched_binbot/lib/
> python2.5/site-packages/pylith/PyLithApp.py", line 85, in main
> File "/home/buildbot/install/x86_linux_single_nosched_binbot/lib/
> python2.5/site-packages/pylith/problems/TimeDependent.py", line 101,
> in initialize
> File "/home/buildbot/install/x86_linux_single_nosched_binbot/lib/
> python2.5/site-packages/pylith/problems/Implicit.py", line 90, in
> initialize
> File "/home/buildbot/install/x86_linux_single_nosched_binbot/lib/
> python2.5/site-packages/pylith/problems/Formulation.py", line 129, in
> initialize
> File "/home/buildbot/install/x86_linux_single_nosched_binbot/lib/
> python2.5/site-packages/pylith/faults/FaultCohesiveKin.py", line 77,
> in initialize
> File "/home/buildbot/install/x86_linux_single_nosched_binbot/lib/
> python2.5/site-packages/pylith/faults/Fault.py", line 141, in initialize
> File "/home/buildbot/install/x86_linux_single_nosched_binbot/lib/
> python2.5/site-packages/pylith/feassemble/quadrature/Quadrature.py",
> line 93, in initialize
> File "feassemble.pyx", line 334, in feassemble.Quadrature.initialize
> File "simplearray.pyx", line 405, in simplearray.objAsSimpleArray
> File "simplearray.pyx", line 346, in
> simplearray.SimplePyArray.__init__
> OverflowError: long int too large to convert to int
> --pyre-start: mpiexec: exit 1
> /home/dicaprio/local/pylith-1.0.1-linux-i686/bin/pylith: /home/
> dicaprio/local/pylith-1.0.1-linux-i686/bin/nemesis: exit 1
> ------------------------------------------------------------------------
> ------------------------------------------------------------------------
> ---------------------------
This should have nothing to do with the mesh size. It seems more likely that
the FEM arguments are screwed up. The Quadrature information is local
to an element and we only support linear elements anyway. My guess is
somehow a ridiculously high quadrature order was somehow specified.
Matt
> I figured there may be a limit to the size of the mesh on a single
> processor so I tried to run on multiple processors. I'm happy to run
> this on one processor if partitioning isn't working, but is there a
> way around this problem?
>
> Thanks,
> Chris
>
>
>
>
> On Oct 19, 2007, at 7:27 PM, Brad Aagaard wrote:
>
> > Chris-
> >
> > To run on a machine with multiple processors or cores, use the --
> > nodes=N command line argument (or put nodes=N in a .cfg file),
> > where N is the number of processors/cores.
> >
> > Note: In doing some testing in the last couple of weeks we
> > discovered that the partitioning is not working properly. A mesh is
> > partitioned, but it is very, very far from optimal. We are fixing
> > this now. In the meantime, you are generally much better off
> > running on a single processor/core.
> >
> > Brad
> >
> > Christopher DiCaprio wrote:
> >> I'm trying to run a model on multiple processors, but I run into the
> >
>
> _______________________________________________
> CIG-SHORT mailing list
> CIG-SHORT at geodynamics.org
> http://geodynamics.org/cgi-bin/mailman/listinfo/cig-short
>
--
What most experimenters take for granted before they begin their
experiments is infinitely more interesting than any results to which
their experiments lead.
-- Norbert Wiener
More information about the CIG-SHORT
mailing list