[CIG-SHORT] pylith multiprocessor

Christopher DiCaprio dicaprio at gps.caltech.edu
Tue Oct 23 11:41:11 PDT 2007 

Brad,

That doesn't seem to fix it:
------------------------------------------------------------------------ 
------------------------------------------------------------------------ 
---------------------------
 > pylith --nodes=2 dislocation.cfg mymachines.cfg
invalid mpiexec argument -machinefile
Usage: mpiexec -usize <universesize> -maxtime <seconds> -exitinfo -l\
                -n <numprocs> -soft <softness> -host <hostname> \
                -wdir <working directory> -path <search path> \
                -file <filename> -configfile <filename> \
                -genvnone -genvlist <name1,name2,...> -genv name value\
                -envnone -envlist <name1,name2,...> -env name value\
                execname <args>\
                [ : -n <numprocs> ... execname <args>]

--pyre-start: mpiexec: exit 255
/home/dicaprio/local/pylith-1.0.1-linux-i686/bin/pylith: /home/ 
dicaprio/local/pylith-1.0.1-linux-i686/bin/nemesis: exit 1
------------------------------------------------------------------------ 
------------------------------------------------------------------------ 
---------------------------


This actually stems from another problem.  I was trying to run a  
large mesh (1110132 elements) and I got this error:
------------------------------------------------------------------------ 
------------------------------------------------------------------------ 
---------------------------
n> pylith dislocation.cfg
 >> /home/buildbot/install/x86_linux_single_nosched_binbot/lib/ 
python2.5/site-packages/pylith/utils/PetscManager.py:47:initialize
-- petsc(info)
-- Initializing PETSc.
 >> /home/buildbot/install/x86_linux_single_nosched_binbot/lib/ 
python2.5/site-packages/pylith/meshio/MeshIO.py:45:read
-- meshiolagrit(info)
-- Reading finite-element mesh
 >> /home/buildbot/install/x86_linux_single_nosched_binbot/lib/ 
python2.5/site-packages/pylith/problems/TimeDependent.py:92:initialize
-- timedependent(info)
-- Initializing problem.
 >> /home/buildbot/install/x86_linux_single_nosched_binbot/lib/ 
python2.5/site-packages/pylith/problems/Formulation.py:89:initialize
-- implicit(info)
-- Initializing materials.
 >> /home/buildbot/install/x86_linux_single_nosched_binbot/lib/ 
python2.5/site-packages/pylith/problems/Formulation.py:106:initialize
-- implicit(info)
-- Added elasticity integrator for material 'Elastic material -x'.
 >> /home/buildbot/install/x86_linux_single_nosched_binbot/lib/ 
python2.5/site-packages/pylith/problems/Formulation.py:106:initialize
-- implicit(info)
-- Added elasticity integrator for material 'Elastic material +x'.
 >> /home/buildbot/install/x86_linux_single_nosched_binbot/lib/ 
python2.5/site-packages/pylith/problems/Formulation.py:106:initialize
-- implicit(info)
-- Added elasticity integrator for material 'Viscoelastic material -x'.
 >> /home/buildbot/install/x86_linux_single_nosched_binbot/lib/ 
python2.5/site-packages/pylith/problems/Formulation.py:106:initialize
-- implicit(info)
-- Added elasticity integrator for material 'Viscoelastic material +x'.
 >> /home/buildbot/install/x86_linux_single_nosched_binbot/lib/ 
python2.5/site-packages/pylith/problems/Formulation.py:106:initialize
-- implicit(info)
-- Added elasticity integrator for material 'Viscoelastic material -x'.
 >> /home/buildbot/install/x86_linux_single_nosched_binbot/lib/ 
python2.5/site-packages/pylith/problems/Formulation.py:108:initialize
-- implicit(info)
-- Initializing boundary conditions.
 >> /home/buildbot/install/x86_linux_single_nosched_binbot/lib/ 
python2.5/site-packages/pylith/problems/Formulation.py:121:initialize
-- implicit(info)
-- Added boundary condition 'right' as a constraint.
 >> /home/buildbot/install/x86_linux_single_nosched_binbot/lib/ 
python2.5/site-packages/pylith/problems/Formulation.py:121:initialize
-- implicit(info)
-- Added boundary condition 'left' as a constraint.
 >> /home/buildbot/install/x86_linux_single_nosched_binbot/lib/ 
python2.5/site-packages/pylith/problems/Formulation.py:121:initialize
-- implicit(info)
-- Added boundary condition '' as a constraint.
 >> /home/buildbot/install/x86_linux_single_nosched_binbot/lib/ 
python2.5/site-packages/pylith/problems/Formulation.py:121:initialize
-- implicit(info)
-- Added boundary condition '' as a constraint.
 >> /home/buildbot/install/x86_linux_single_nosched_binbot/lib/ 
python2.5/site-packages/pylith/problems/Formulation.py:121:initialize
-- implicit(info)
-- Added boundary condition '' as a constraint.
 >> /home/buildbot/install/x86_linux_single_nosched_binbot/lib/ 
python2.5/site-packages/pylith/problems/Formulation.py:121:initialize
-- implicit(info)
-- Added boundary condition '' as a constraint.
 >> /home/buildbot/install/x86_linux_single_nosched_binbot/lib/ 
python2.5/site-packages/pylith/problems/Formulation.py:127:initialize
-- implicit(info)
-- Initializing interior interfaces.
 >> /home/buildbot/install/x86_linux_single_nosched_binbot/lib/ 
python2.5/site-packages/pylith/faults/FaultCohesiveKin.py:70:initialize
-- faultcohesivekin(info)
-- Initializing fault 'pfb1b2'.
Traceback (most recent call last):
   File "<string>", line 1, in <module>
   File "/home/buildbot/install/x86_linux_single_nosched_binbot/lib/ 
python2.5/site-packages/pythia-0.8.1.4-py2.5.egg/mpi/__init__.py",  
line 42, in mpistart
   File "/home/buildbot/install/x86_linux_single_nosched_binbot/lib/ 
python2.5/site-packages/pythia-0.8.1.4-py2.5.egg/pyre/applications/ 
__init__.py", line 41, in start
   File "/home/buildbot/install/x86_linux_single_nosched_binbot/lib/ 
python2.5/site-packages/pythia-0.8.1.4-py2.5.egg/pyre/applications/ 
Shell.py", line 139, in run
   File "/home/buildbot/install/x86_linux_single_nosched_binbot/lib/ 
python2.5/site-packages/pythia-0.8.1.4-py2.5.egg/pyre/applications/ 
SuperScript.py", line 49, in execute
   File "/home/buildbot/install/x86_linux_single_nosched_binbot/lib/ 
python2.5/site-packages/pythia-0.8.1.4-py2.5.egg/pyre/applications/ 
SuperScript.py", line 58, in main
   File "/home/buildbot/install/x86_linux_single_nosched_binbot/lib/ 
python2.5/site-packages/pythia-0.8.1.4-py2.5.egg/pyre/applications/ 
SuperScript.py", line 62, in runSubscript
   File "/home/buildbot/install/x86_linux_single_nosched_binbot/lib/ 
python2.5/site-packages/pythia-0.8.1.4-py2.5.egg/pyre/applications/ 
Application.py", line 42, in run
   File "/home/buildbot/install/x86_linux_single_nosched_binbot/lib/ 
python2.5/site-packages/pythia-0.8.1.4-py2.5.egg/pyre/applications/ 
Shell.py", line 139, in run
   File "/home/buildbot/install/x86_linux_single_nosched_binbot/lib/ 
python2.5/site-packages/pythia-0.8.1.4-py2.5.egg/mpi/Application.py",  
line 118, in _onComputeNodes
   File "/home/buildbot/install/x86_linux_single_nosched_binbot/lib/ 
python2.5/site-packages/pythia-0.8.1.4-py2.5.egg/mpi/Application.py",  
line 124, in onComputeNodes
   File "/home/buildbot/install/x86_linux_single_nosched_binbot/lib/ 
python2.5/site-packages/pylith/PyLithApp.py", line 85, in main
   File "/home/buildbot/install/x86_linux_single_nosched_binbot/lib/ 
python2.5/site-packages/pylith/problems/TimeDependent.py", line 101,  
in initialize
   File "/home/buildbot/install/x86_linux_single_nosched_binbot/lib/ 
python2.5/site-packages/pylith/problems/Implicit.py", line 90, in  
initialize
   File "/home/buildbot/install/x86_linux_single_nosched_binbot/lib/ 
python2.5/site-packages/pylith/problems/Formulation.py", line 129, in  
initialize
   File "/home/buildbot/install/x86_linux_single_nosched_binbot/lib/ 
python2.5/site-packages/pylith/faults/FaultCohesiveKin.py", line 77,  
in initialize
   File "/home/buildbot/install/x86_linux_single_nosched_binbot/lib/ 
python2.5/site-packages/pylith/faults/Fault.py", line 141, in initialize
   File "/home/buildbot/install/x86_linux_single_nosched_binbot/lib/ 
python2.5/site-packages/pylith/feassemble/quadrature/Quadrature.py",  
line 93, in initialize
   File "feassemble.pyx", line 334, in feassemble.Quadrature.initialize
   File "simplearray.pyx", line 405, in simplearray.objAsSimpleArray
   File "simplearray.pyx", line 346, in  
simplearray.SimplePyArray.__init__
OverflowError: long int too large to convert to int
--pyre-start: mpiexec: exit 1
/home/dicaprio/local/pylith-1.0.1-linux-i686/bin/pylith: /home/ 
dicaprio/local/pylith-1.0.1-linux-i686/bin/nemesis: exit 1
------------------------------------------------------------------------ 
------------------------------------------------------------------------ 
---------------------------

I figured there may be a limit to the size of the mesh on a single  
processor so I tried to run on multiple processors.  I'm happy to run  
this on one processor if partitioning isn't working, but is there a  
way around this problem?

Thanks,
Chris




On Oct 19, 2007, at 7:27 PM, Brad Aagaard wrote:

> Chris-
>
> To run on a machine with multiple processors or cores, use the -- 
> nodes=N command line argument (or put nodes=N in a .cfg file),  
> where N is the number of processors/cores.
>
> Note: In doing some testing in the last couple of weeks we  
> discovered that the partitioning is not working properly. A mesh is  
> partitioned, but it is very, very far from optimal. We are fixing  
> this now. In the meantime, you are generally much better off  
> running on a single processor/core.
>
> Brad
>
> Christopher DiCaprio wrote:
>> I'm trying to run a model on multiple processors, but I run into the
>

More information about the CIG-SHORT mailing list