[CIG-MC] ssh and autocombine.py

[Eh Tan]

Hi Rob,

Great. It seems there is an additional newline character in your
nodelist. Can you send me your machinefile (off the mailing list) so
that I can see where the problem is?

Eh


Robert Moucha wrote:
> Fixed! I went in and tinkered with batchcombine.py, and after reading
> some python stuff, I added the following line to the for loop in the
> function batchpaste:
>
>    node = node.rstrip()
>
> That did it, though it probably should be done somewhere else in the script.
>
> Cheers
> Rob
>
> On Thu, Nov 5, 2009 at 11:36 AM, Robert Moucha <rmoucha at gmail.com> wrote:
>   
>> Hi Eh,
>>
>> I checked and I do have my path variable set correctly so that is not
>> the problem. However I did figure out what is happening, but I don't
>> know enough python to figure how to fix it:
>>
>> If I print the variable cmd in the function batchpaste in
>> batchcombine.py I get the following:
>>
>> ssh c0-0
>>  pasteCitcomData.py /state/partition1/moucha2/citcoms cookbook6
>> coord,velo,visc 3 10 /home/moucha/convection/citcoms/exa
>>
>> Note the return character after the node name (not an artifact of
>> wrapping)! So, the os.system call only executes the ssh c0-0.  This
>> explains why I am successively logged into the nodes whey I run
>> autocombine.py, but the command pasteCitcomData.py is not executed.
>> And when I type exit, the command pasteCitcomData.py is executed, but
>> now the file does not exist on the master node and so I get a file not
>> found error. I went back into batchcombine.py and added:
>>
>> print node
>> print 'this is a test'
>>
>> Now I get:
>>
>> c0-0
>>
>> this is a test
>>
>> So the return character is coming from the variable node (note the
>> blank line between the print statements). My mpirun.nodes looks like
>> this:
>>
>> c0-0
>> c0-34
>> c0-33
>> c0-46
>>
>> Once again, thanks for your help on this.
>>
>> Rob
>>
>>
>> On Wed, Nov 4, 2009 at 4:39 PM, Eh Tan <tan2 at geodynamics.org> wrote:
>>     
>>> Hi Rob,
>>>
>>> You need to add the visual/ directory to your PATH environment variable.
>>> If you are using bash, you can add this line in your ~/.bashrc:
>>>
>>> export PATH=$PATH:$HOME/convection/citcoms/visual
>>>
>>> Eh
>>>
>>> Robert Moucha wrote:
>>>       
>>>> Hi Eh,
>>>>
>>>> Thanks for getting back to me so quickly. Our cluster is set-up with
>>>> password-less ssh into the nodes. I went into
>>>> batchcombine.py and added the line print cmd, to see what command was
>>>> being executed, here is what I get:
>>>> ------------------------
>>>> $ autocombine.py mpirun.nodes pid23816.cfg 10
>>>>
>>>> ssh c0-0  pasteCitcomData.py /state/partition1/moucha2/citcoms
>>>> cookbook6 coord,velo,visc 0 10
>>>> /home/moucha/convection/citcoms/examples/Cookbook6
>>>>
>>>> Last login: Wed Nov  4 16:04:40 2009 from geograppe.local
>>>> Rocks Compute Node
>>>> Rocks 5.0 (V)
>>>> Profile built 16:38 16-Jul-2008
>>>>
>>>> Kickstarted 16:50 16-Jul-2008
>>>> [moucha at compute-0-0 ~]$
>>>> -------------------------
>>>>
>>>> So I tried from my master node:
>>>>
>>>> $ ssh c0-0  pasteCitcomData.py /state/partition1/moucha2/citcoms
>>>> cookbook6 coord,velo,visc 0 10
>>>> /home/moucha/convection/citcoms/examples/Cookbook6
>>>>
>>>> and I get:
>>>>
>>>> bash: pasteCitcomData.py: command not found
>>>>
>>>> Now with the full path:
>>>>
>>>> $ ssh c0-0  ~/software/cig/bin/pasteCitcomData.py
>>>> /state/partition1/moucha2/citcoms cookbook6 coord,velo,visc 0 10
>>>> /home/moucha/convection/citcoms/examples/Cookbook6
>>>>
>>>> It works!  Funny though, because when I log into the node, I can
>>>> execute the command directly.  Do you know of a way to solve this
>>>> without editing the scripts, or is this just the way ssh works?
>>>>
>>>> Thanks
>>>>
>>>> Rob
>>>>
>>>>         
>>> --
>>> Eh Tan
>>> Staff Scientist
>>> Computational Infrastructure for Geodynamics
>>> California Institute of Technology, 158-79
>>> Pasadena, CA 91125
>>> (626) 395-1693
>>> http://www.geodynamics.org
>>>
>>>
>>>
>>>
>>> --
>>> Eh Tan
>>> Staff Scientist
>>> Computational Infrastructure for Geodynamics
>>> California Institute of Technology, 158-79
>>> Pasadena, CA 91125
>>> (626) 395-1693
>>> http://www.geodynamics.org
>>>
>>> _______________________________________________
>>> CIG-MC mailing list
>>> CIG-MC at geodynamics.org
>>> http://geodynamics.org/cgi-bin/mailman/listinfo/cig-mc
>>>
>>>       
>>
>> --
>> GEOTOP - Département des Sciences de la Terre et de l'Atmosphère
>> Université du Québec à Montréal
>> CP 8888, succursale Centre-Ville
>> Montréall, Québec
>> Canada  H3C 3P8
>> Tel:     (1-514) 987-3000, ext 1554#
>> FAX:     (1-514) 987-3635
>>
>>     
>
>
>
>   

-- 
Eh Tan
Staff Scientist
Computational Infrastructure for Geodynamics
California Institute of Technology, 158-79
Pasadena, CA 91125
(626) 395-1693
http://www.geodynamics.org