[CIG-SHORT] ch_p4

[Leif Strand]

Tabrez,

I chose MPICH1's ch_shmem device not for performance reasons, but 
because it works without any additional configuration, which is great 
for prebuilt binaries. I've been told that MPICH2 is more flexible: it 
has something called "gforker" which would work great for the binary 
package. So we'll probably be switching to MPICH2 for future binary 
releases of PyLith.

It isn't really practical to create binaries for clusters. There are 
other factors besides the MPICH device. For example, on our cluster here 
at Caltech, nothing works unless you build with MPICH-GM, which is 
preinstalled on the cluster, and is available only from Myricom... so 
there is no general-purpose prebuilt binary that I could possibly create 
which will work on this cluster. Also, supporting clusters opens the 
door to additional architectures, in addition to Intel and PowerPC. We'd 
need prebuilt binaries for Itanium, x86_64, and who knows what else.

So, running on a cluster means configuring and building something from 
source on the cluster itself. Currently that "something" is all of 
PyLith, including its dependencies (mainly PETSc). I've thought of ways 
to factor-out the MPICH dependency so that the "something" you build 
from source is instead a tiny plug-in MPICH module, but that still 
leaves the problem of supporting all those architectures...

In short, running on a cluster means buliding from source. In light of 
this, we'll continue to improve the build process so that building from 
source is as easy as possible. Let us know about any problems you encounter.

--Leif

Tabrez Ali wrote:

> Leif
>
> That's what I thought too. I'll try to compile pylith from source. Btw 
> is it possible to implement ch_p4 in future releases (of x86 linux 
> binaries) as it can be used on SMP's as well as on clusters 
> (distributed memory) unlike ch_shmem which only works on SMP's/multi 
> core procs?
>
> Btw I dont know if ch_shmem really offers that much of a performance 
> boost over ch_p4 on SMP's.
>
> Tabrez
>
>
> Leif Strand wrote:
>
>> Tabrez,
>>
>> PyLith should work fine on a cluster -- provided that it is built 
>> from source, and is thus configured to use the MPICH installation on 
>> your cluster.
>>
>> The prebuilt binaries, however, only work on a workstation/laptop. 
>> The PyLith binary package includes its own copy of MPICH, which is 
>> configured to use ch_shmem. If these binaries run at all on a 
>> cluster, most likely they will do the wrong thing: i.e., run 
>> exclusively on the login node, as opposed to running in parallel on 
>> the compute nodes.
>>
>> --Leif Strand
>>
>> Tabrez Ali wrote:
>>
>>> Hello
>>>
>>> Is it possible to run pylith on clusters (ch_p4 devices) or it only 
>>> runs on shared memory machines (ch_shmem)? I was trying to use 
>>> mpirun (configured with ch_p4) and it prompts that program needs 
>>> ch_shmem.
>>>
>>> Btw everything works fine with mpich which comes bundled with pylith 
>>> (on a single/dual proc workstation).
>>>
>>> Regards
>>> Tabrez
>>> _______________________________________________
>>> CIG-SHORT mailing list
>>> CIG-SHORT at geodynamics.org
>>> http://geodynamics.org/cgi-bin/mailman/listinfo/cig-short
>>
>>
>>
>> _______________________________________________
>> CIG-SHORT mailing list
>> CIG-SHORT at geodynamics.org
>> http://geodynamics.org/cgi-bin/mailman/listinfo/cig-short
>
>