[CIG-SHORT] Running on cluster

[Brad Aagaard]

Xiao,

Are you trying to use the PyLith binary on a cluster? It will work on a 
single compute node provided you setup the environment correctly (load 
the setup.sh file on the compute node), but it won't run on multiple 
compute nodes.

To run on multiple compute nodes, you must build from source (we 
recommend the PyLith Installer utility for this purpose). See the 
INSTALL file in the PyLith Installer distribution.

Regards,
Brad


On 7/28/14, 2:53 PM, Matthew Knepley wrote:
> On Mon, Jul 28, 2014 at 4:44 PM, Ma, Xiao <xiaoma5 at illinois.edu
> <mailto:xiaoma5 at illinois.edu>> wrote:
>
>     Hi,
>     I am running pylith on a cluster.
>     Here is the input part :
>     ***************************************
>     [pylithapp]
>     scheduler = pbs
>     [pylithapp.pbs]
>     shell = /bin/bash
>     # Export all environment variables to the batch job
>     # Send email to johndoe at mydomain.org <mailto:johndoe at mydomain.org>
>     when the job begins, ends, or aborts
>     qsub-options = -V -m bea -M xiaoma5 at illinois.edu
>     <mailto:xiaoma5 at illinois.edu>
>     [pylithapp.launcher]
>     command = mpirun -np ${nodes} -machinefile ${PBS_NODEFILE}
>
>     *********************************************
>     Error message
>     **********************************************
>     Traceback (most recent call last):
>        File "<string>", line 1, in <module>
>        File
>     "/home/xiaoma5/project-cse/pylith-1.9.0-linux-x86_64/lib/python2.7/site-packages/pythia-0.8.1.15-py2.7.egg/pyre/schedulers/__init__.py",
>     line 53, in jobstart
>          kwds = kwds)
>        File
>     "/home/xiaoma5/project-cse/pylith-1.9.0-linux-x86_64/lib/python2.7/site-packages/pythia-0.8.1.15-py2.7.egg/pyre/applications/__init__.py",
>     line 41, in start
>          shell.run(**kwds)
>        File
>     "/home/xiaoma5/project-cse/pylith-1.9.0-linux-x86_64/lib/python2.7/site-packages/pythia-0.8.1.15-py2.7.egg/pyre/applications/Shell.py",
>     line 143, in run
>          method(*args, **kwds)
>        File
>     "/home/xiaoma5/project-cse/pylith-1.9.0-linux-x86_64/lib/python2.7/site-packages/pythia-0.8.1.15-py2.7.egg/pyre/applications/SuperScript.py",
>     line 47, in execute
>          self.subscript = self.createSubscript(subscriptName)
>        File
>     "/home/xiaoma5/project-cse/pylith-1.9.0-linux-x86_64/lib/python2.7/site-packages/pythia-0.8.1.15-py2.7.egg/pyre/applications/AppRunner.py",
>     line 25, in createSubscript
>          cls = loadObject(name)
>        File
>     "/home/xiaoma5/project-cse/pylith-1.9.0-linux-x86_64/lib/python2.7/site-packages/pythia-0.8.1.15-py2.7.egg/pyre/util/__init__.py",
>     line 54, in loadObject
>          obj = __import__(module, globals(), globals(), ['__name__'])
>        File
>     "/home/xiaoma5/project-cse/pylith-1.9.0-linux-x86_64/lib/python2.7/site-packages/pylith/apps/PyLithApp.py",
>     line 23, in <module>
>          from PetscApplication import PetscApplication
>        File
>     "/home/xiaoma5/project-cse/pylith-1.9.0-linux-x86_64/lib/python2.7/site-packages/pylith/apps/PetscApplication.py",
>     line 27, in <module>
>          class PetscApplication(Application):
>        File
>     "/home/xiaoma5/project-cse/pylith-1.9.0-linux-x86_64/lib/python2.7/site-packages/pylith/apps/PetscApplication.py",
>     line 41, in PetscApplication
>          from pylith.utils.PetscManager import PetscManager
>        File
>     "/home/xiaoma5/project-cse/pylith-1.9.0-linux-x86_64/lib/python2.7/site-packages/pylith/utils/PetscManager.py",
>     line 29, in <module>
>          import pylith.utils.petsc as petsc
>        File
>     "/home/xiaoma5/project-cse/pylith-1.9.0-linux-x86_64/lib/python2.7/site-packages/pylith/utils/petsc.py",
>     line 25, in <module>
>          _petsc = swig_import_helper()
>        File
>     "/home/xiaoma5/project-cse/pylith-1.9.0-linux-x86_64/lib/python2.7/site-packages/pylith/utils/petsc.py",
>     line 21, in swig_import_helper
>          _mod = imp.load_module('_petsc', fp, pathname, description)
>     ImportError: /usr/lib64/libgfortran.so.3: version `GFORTRAN_1.4' not
>     found (required by
>     /home/xiaoma5/project-cse/pylith-1.9.0-linux-x86_64/lib/libpetsc.so)
>
>
> The problem here is that the environment where you built PyLith,
> specifically the gfortran compiler, is different
> from the nodes on which you are trying to run. Possible solutions:
>
>    1) Build on the cluster itself
>
>    2) Build PETSc without Fortran using --with-fc=0 in configure
>
>     Thanks,
>
>       Matt
>
>     ----------------------------------------
>     *****************************************************************************
>     The program is running when I am not using the cluster, so there
>     must be something not right in this cluster command lines.
>     *****************************************************************************
>     %%Command Lines:
>
>     pylith test1.cfg --job.queue=test \
>     --job.name <http://job.name>=test1 --job.stdout=example1.log
>     --job.stderr=example1.err \
>     --job.walltime=5*minute \
>     --nodes=4
>
>     I will really appriciate your help
>     Best,
>     Xiao
>
>     _______________________________________________
>     CIG-SHORT mailing list
>     CIG-SHORT at geodynamics.org <mailto:CIG-SHORT at geodynamics.org>
>     http://lists.geodynamics.org/cgi-bin/mailman/listinfo/cig-short
>
>
>
>
> --
> What most experimenters take for granted before they begin their
> experiments is infinitely more interesting than any results to which
> their experiments lead.
> -- Norbert Wiener
>
>
> _______________________________________________
> CIG-SHORT mailing list
> CIG-SHORT at geodynamics.org
> http://lists.geodynamics.org/cgi-bin/mailman/listinfo/cig-short
>