[aspect-devel] [CIG-MC] Installing ASPECT on Cray XC30

Rene Gassmoeller rene.gassmoeller at mailbox.org
Wed Jul 12 10:24:46 PDT 2017 

Hi Marine,

I have successfully installed ASPECT on several Cray XC30 and XC40 
systems (I was the person with the Cray install script for candi ;-)), 
but the problem here is that these clusters are usually heavily modified 
by the admin teams so a general instruction does not help as much.

That being said, from your description it really seems like Timo's 
suggestion is the last step missing

cmake -D ASPECT_USE_FP_EXCEPTIONS=OFF .

in the ASPECT build directory during compiling should do the trick.

Alternatively, we merged a pull request a few weeks back 
(https://github.com/geodynamics/aspect/pull/1819), which we hoped would 
fix this issue. Do you use an ASPECT version before or after that PR 
(i.e. latest release or latest development version)?


Best,

Rene




On 07/12/2017 01:48 AM, Marine Lasbleis wrote:
> Hi Philip,
>
> Could you tell me exactly which tests you are performing and are failing?
> Sorry, I’m really not familiar with deal.II, so would prefer to be 
> sure about what I’m doing!
>
> (I’m assuming that, since I ran a couple of simple box convection, it 
> is working correctly, but it may indeed not be the case)
>
> We don’t have admin team on our Cray XC30, so I guess it has not been 
> modified from the original version we got.
>
> Marine Lasbleis
> =====
> ELSI Research Scientist
> Earth Life Science Institute
> Tokyo Institute of Technology
> Tokyo, Japan
> +81 70 1572 5070
> marine.lasbleis at elsi.jp <mailto:marine.lasbleis at elsi.jp>
>
> https://members.elsi.jp/~marine.lasbleis/ 
> <https://members.elsi.jp/%7Emarine.lasbleis/>
> http://elsi.jp/en/
>
>> On 12 Jul 2017, at 05:39, HERON, PHILIP J. 
>> <philip.j.heron at durham.ac.uk <mailto:philip.j.heron at durham.ac.uk>> wrote:
>>
>> Hi Marine,
>>
>> I have attempted to install ASPECT on a Cray XC30 (Archer HPC in the 
>> UK) with no success. The HPC admin team has not had any success 
>> either. Please keep up posted on your progress.
>>
>> I can compile and build dealii and aspect, but most dealii tests fail:
>>
>> 4 - mpi.debug (Failed)
>> 6 - p4est.debug (Failed)
>> 7 - step-trilinos.debug (Failed)
>>
>> I will have another go this week.
>>
>> Phil
>>
>> Philip J. Heron
>> Junior Research Fellow
>> Dept. of Earth Sciences
>> Durham University
>> web: http://philheron.com <http://philheron.com/>
>>
>> ------------------------------------------------------------------------
>> *From:*Aspect-devel <aspect-devel-bounces at geodynamics.org 
>> <mailto:aspect-devel-bounces at geodynamics.org>> on behalf of Timo 
>> Heister <heister at clemson.edu <mailto:heister at clemson.edu>>
>> *Sent:*11 July 2017 14:00:09
>> *To:*aspect-devel
>> *Subject:*Re: [aspect-devel] Fwd: [CIG-MC] Installing ASPECT on Cray 
>> XC30
>> Marine,
>>
>> 1. can you post your detailed.log from your ASPECT directory?
>>
>> 2. can you try to call
>> cmake -D ASPECT_USE_FP_EXCEPTIONS=OFF .
>>
>> inside your aspect build directory, and then "make" again? This should
>> disable the floating point exceptions (we use this as a tool to find
>> errors and bugs), but you will likely run into other problems then.
>> But maybe this is enough to make things work.
>>
>>
>> On Tue, Jul 11, 2017 at 2:09 AM, Marine Lasbleis
>> <marine.lasbleis at elsi.jp <mailto:marine.lasbleis at elsi.jp>> wrote:
>> > Hi all,
>> >
>> > Sorry, it appears I used the wrong mailing list!
>> > Let me know if someone has experience building ASPECT (and required
>> > libraries) on a CRAY XC30.
>> >
>> > Best,
>> > Marine
>> >
>> >
>> > Marine Lasbleis
>> >
>> > =====
>> > ELSI Research Scientist
>> > Earth Life Science Institute
>> > Tokyo Institute of Technology
>> > Tokyo, Japan
>> > +81 70 1572 5070
>> > marine.lasbleis at elsi.jp <mailto:marine.lasbleis at elsi.jp>
>> >
>> > 
>> https://urldefense.proofpoint.com/v2/url?u=https-3A__members.elsi.jp_-7Emarine.lasbleis_&d=DwIFaQ&c=Ngd-ta5yRYsqeUsEDgxhcqsYYY1Xs5ogLxWPA_2Wlc4&r=c08Btfq4m9QEScXN3ZQwLZzzWQE7S8CYq1IYuzKV_Zk&m=zrvFCXon07qCkF8qXzZ6rHu6CjY30e55CUFHopV1Hm8&s=LoLdd_1ebxXqVRRXpWavR7d1dGghq8WrxfMfJW5bv90&e= 
>>
>> > 
>> https://urldefense.proofpoint.com/v2/url?u=http-3A__elsi.jp_en_&d=DwIFaQ&c=Ngd-ta5yRYsqeUsEDgxhcqsYYY1Xs5ogLxWPA_2Wlc4&r=c08Btfq4m9QEScXN3ZQwLZzzWQE7S8CYq1IYuzKV_Zk&m=zrvFCXon07qCkF8qXzZ6rHu6CjY30e55CUFHopV1Hm8&s=7qmW2cZqp2dDQA8f55NM9FLl5v83q1G1A8WVaCxGC14&e= 
>>
>> >
>> >
>> > Begin forwarded message:
>> >
>> > From: Max Rudolph <maxrudolph at ucdavis.edu 
>> <mailto:maxrudolph at ucdavis.edu>>
>> > Subject: Re: [CIG-MC] Installing ASPECT on Cray XC30
>> > Date: 11 Jul 2017 11:52:51 GMT+9
>> > To: <cig-mc at geodynamics.org <mailto:cig-mc at geodynamics.org>>
>> > Reply-To: cig-mc at geodynamics.org <mailto:cig-mc at geodynamics.org>
>> >
>> > Marine,
>> > You may want to post this on the aspect-devel mailing list if you 
>> haven't
>> > done so already.
>> >
>> > Max
>> >
>> > On Mon, Jul 10, 2017 at 5:00 PM, Marine Lasbleis 
>> <marine.lasbleis at gmail.com <mailto:marine.lasbleis at gmail.com>>
>> > wrote:
>> >>
>> >> Hi all,
>> >>
>> >> This is my first message here, I hope it’s OK.
>> >> I’m started to work on ASPECT, and installed it already on a desktop
>> >> computer (debian with 8 cores). But would like to install it on the
>> >> available clusters. (I have access to 3 different clusters. Not 
>> sure which
>> >> one is the best for that… And definitely no real admin for the 
>> clusters.
>> >> They are “self-organised”, which is not always for the best)
>> >>
>> >> I’m trying to install ASPECT on the ELSI cluster, which is a CRAY 
>> CX30,
>> >> and while having problems, I found that you may have done the same 
>> a couple
>> >> of weeks ago (I saw this conversation:
>> >> 
>> https://urldefense.proofpoint.com/v2/url?u=http-3A__dealii.narkive.com_jCU1oGdB_deal-2Dii-2Dget-2Derrors-2Dwhen-2Dinstalling-2Ddealii-2Don-2Dopensuse-2Dleap-2D42-2D1-2Dby-2Dusing-2Dcandi&d=DwIFaQ&c=Ngd-ta5yRYsqeUsEDgxhcqsYYY1Xs5ogLxWPA_2Wlc4&r=c08Btfq4m9QEScXN3ZQwLZzzWQE7S8CYq1IYuzKV_Zk&m=zrvFCXon07qCkF8qXzZ6rHu6CjY30e55CUFHopV1Hm8&s=3hIE02zdEMfEZQ-54THur4-xXpYPz_X9QkkDRViqAOc&e= 
>>
>> >> )
>> >>
>> >> For now, what we’ve done: (before seeing candi installation)
>> >> - switch to PrgEnv-gnu
>> >> - try to install p4est. But it seems that we need to use “ftn” and not
>> >> fortran or others, so he can’t do anything, and stop very soon. I 
>> tried to
>> >> modify by hand the configure file (adding ftn where I could find 
>> the system
>> >> was looking for fortran of mpif77.) But I guess it’s definitely 
>> not a good
>> >> idea, and I am obviously still missing a couple of call because I 
>> still got
>> >> the same error.
>> >>
>> >> So, with the conversation, I guessed that 
>> https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_dealii_candi&d=DwIFaQ&c=Ngd-ta5yRYsqeUsEDgxhcqsYYY1Xs5ogLxWPA_2Wlc4&r=c08Btfq4m9QEScXN3ZQwLZzzWQE7S8CYq1IYuzKV_Zk&m=zrvFCXon07qCkF8qXzZ6rHu6CjY30e55CUFHopV1Hm8&s=PNy4RPfYtSh0YwU5s31LHSfmZj0RgQv3wznnVVHjRXw&e= 
>>
>> >> can actually install everything for me.
>> >> Since I’m using a slightly different cluster (CRAY XC30), I will 
>> try to
>> >> give you updates on my progress.
>> >> I’m not familiar with candi, but I decided to give a try, so 
>> please excuse
>> >> me if I am doing obvious mistakes.
>> >>
>> >> I changed the configuration as requested, and loaded the required 
>> modules
>> >> and defined new variables for the info on the compilers.
>> >> In this particular cluster, we need to be careful with the path 
>> where to
>> >> install (the default one is on a drive that is very slow to 
>> access, and
>> >> compilation takes forever), so I had to use a -p path option. 
>> Also, I think
>> >> I used first too many cores to compile, and got a memory error 
>> (internal
>> >> compiler error raised, which seems to be related to available memory)
>> >>
>> >> So, from my day trying to install:
>> >> - I finished the candi.sh script, apparently everything correctly
>> >> installed.
>> >> - I built ASPECT (with this particular cluster, be careful with 
>> cmake. By
>> >> default, the cmake is not up-to-date, and in particular even after
>> >> installation with candi.sh, the available cmake is not the one 
>> that was
>> >> installed)
>> >> I got a couple of warnings, mostly about PETSc, that I thought 
>> were only
>> >> warnings and not problems.
>> >> Most of them were along the line of this one:
>> >> warning: 
>> 'dealii::PETScWrappers::MPI::Vector::supports_distributed_data'
>> >> is deprecated [-Wdeprecated-declarations] , for either PETSc or 
>> Trilinos.
>> >>
>> >> - I’ve run a couple of examples from the cookbook. None are working.
>> >>
>> >> I got this from running ASPEC using aprun -n4 ../aspect burnman.prm
>> >>
>> >> 
>> -----------------------------------------------------------------------------
>> >> -- This is ASPECT, the Advanced Solver for Problems in Earth's 
>> ConvecTion.
>> >> --     . version 1.5.0
>> >> --     . running in DEBUG mode
>> >> --     . running with 4 MPI processes
>> >> --     . using Trilinos
>> >>
>> >> 
>> -----------------------------------------------------------------------------
>> >>
>> >> [0]PETSC ERROR: [1]PETSC ERROR: [3]PETSC ERROR: [2]PETSC ERROR:
>> >> 
>> ------------------------------------------------------------------------
>> >> [0]PETSC ERROR:
>> >> 
>> ------------------------------------------------------------------------
>> >> 
>> ------------------------------------------------------------------------
>> >> [2]PETSC ERROR:
>> >> 
>> ------------------------------------------------------------------------
>> >> [1]PETSC ERROR: [3]PETSC ERROR: Caught signal number 8 FPE: 
>> Floating Point
>> >> Exception,probably divide by zero
>> >> [1]PETSC ERROR: Try option -start_in_debugger or 
>> -on_error_attach_debugger
>> >> Caught signal number 8 FPE: Floating Point Exception,probably 
>> divide by
>> >> zero
>> >> [1]PETSC ERROR: [3]PETSC ERROR: or see
>> >> 
>> https://urldefense.proofpoint.com/v2/url?u=http-3A__www.mcs.anl.gov_petsc_documentation_faq.html-23valgrind&d=DwIFaQ&c=Ngd-ta5yRYsqeUsEDgxhcqsYYY1Xs5ogLxWPA_2Wlc4&r=c08Btfq4m9QEScXN3ZQwLZzzWQE7S8CYq1IYuzKV_Zk&m=zrvFCXon07qCkF8qXzZ6rHu6CjY30e55CUFHopV1Hm8&s=EwRL-vfnsOaRgZyq9yA5cgFyGt6wyqbxmPnUR8DEnjo&e= 
>>
>> >> Try option -start_in_debugger or -on_error_attach_debugger
>> >> [1]PETSC ERROR: [3]PETSC ERROR: or try 
>> https://urldefense.proofpoint.com/v2/url?u=http-3A__valgrind.org&d=DwIFaQ&c=Ngd-ta5yRYsqeUsEDgxhcqsYYY1Xs5ogLxWPA_2Wlc4&r=c08Btfq4m9QEScXN3ZQwLZzzWQE7S8CYq1IYuzKV_Zk&m=zrvFCXon07qCkF8qXzZ6rHu6CjY30e55CUFHopV1Hm8&s=ozKrZdv65t35wUP3q4s5YnSV27_oA6xmJCjsqYFNx3s&e= 
>> on GNU/linux
>> >> and Apple Mac OS X to find memory corruption errors
>> >> or see 
>> https://urldefense.proofpoint.com/v2/url?u=http-3A__www.mcs.anl.gov_petsc_documentation_faq.html-23valgrind&d=DwIFaQ&c=Ngd-ta5yRYsqeUsEDgxhcqsYYY1Xs5ogLxWPA_2Wlc4&r=c08Btfq4m9QEScXN3ZQwLZzzWQE7S8CYq1IYuzKV_Zk&m=zrvFCXon07qCkF8qXzZ6rHu6CjY30e55CUFHopV1Hm8&s=EwRL-vfnsOaRgZyq9yA5cgFyGt6wyqbxmPnUR8DEnjo&e= 
>>
>> >> [1]PETSC ERROR: [3]PETSC ERROR: configure using --with-debugging=yes,
>> >> recompile, link, and run
>> >> or try 
>> https://urldefense.proofpoint.com/v2/url?u=http-3A__valgrind.org&d=DwIFaQ&c=Ngd-ta5yRYsqeUsEDgxhcqsYYY1Xs5ogLxWPA_2Wlc4&r=c08Btfq4m9QEScXN3ZQwLZzzWQE7S8CYq1IYuzKV_Zk&m=zrvFCXon07qCkF8qXzZ6rHu6CjY30e55CUFHopV1Hm8&s=ozKrZdv65t35wUP3q4s5YnSV27_oA6xmJCjsqYFNx3s&e= 
>> on GNU/linux and Apple Mac OS X to find memory
>> >> corruption errors
>> >> [1]PETSC ERROR: [3]PETSC ERROR: to get more information on the crash.
>> >> configure using --with-debugging=yes, recompile, link, and run
>> >> [3]PETSC ERROR: to get more information on the crash.
>> >> [1]PETSC ERROR: --------------------- Error Message
>> >> --------------------------------------------------------------
>> >> Caught signal number 8 FPE: Floating Point Exception,probably 
>> divide by
>> >> zero
>> >>
>> >>
>> >>
>> >>
>> >>
>> >> Any idea where this could come from?
>> >> (any additional files I should show you?)
>> >>
>> >>
>> >> Thanks! (and many thanks to the person who did the candi.sh script for
>> >> Cray XC40 :-) )
>> >> Marine
>> >>
>> >>
>> >>
>> >>
>> >>
>> >> _______________________________________________
>> >> CIG-MC mailing list
>> >> CIG-MC at geodynamics.org <mailto:CIG-MC at geodynamics.org>
>> >> 
>> https://urldefense.proofpoint.com/v2/url?u=http-3A__lists.geodynamics.org_cgi-2Dbin_mailman_listinfo_cig-2Dmc&d=DwIFaQ&c=Ngd-ta5yRYsqeUsEDgxhcqsYYY1Xs5ogLxWPA_2Wlc4&r=c08Btfq4m9QEScXN3ZQwLZzzWQE7S8CYq1IYuzKV_Zk&m=zrvFCXon07qCkF8qXzZ6rHu6CjY30e55CUFHopV1Hm8&s=7tnjS3DwD11pd0Hw_1BkS8zxpTVmv_DKo8iWlM7fRuE&e= 
>>
>> >
>> >
>> >
>> >
>> > --
>> > ---------------------------------
>> > Max Rudolph
>> > Assistant Professor, Earth and Planetary Sciences, UC Davis
>> > webpage
>> > _______________________________________________
>> > CIG-MC mailing list
>> > CIG-MC at geodynamics.org <mailto:CIG-MC at geodynamics.org>
>> > 
>> https://urldefense.proofpoint.com/v2/url?u=http-3A__lists.geodynamics.org_cgi-2Dbin_mailman_listinfo_cig-2Dmc&d=DwIFaQ&c=Ngd-ta5yRYsqeUsEDgxhcqsYYY1Xs5ogLxWPA_2Wlc4&r=c08Btfq4m9QEScXN3ZQwLZzzWQE7S8CYq1IYuzKV_Zk&m=zrvFCXon07qCkF8qXzZ6rHu6CjY30e55CUFHopV1Hm8&s=7tnjS3DwD11pd0Hw_1BkS8zxpTVmv_DKo8iWlM7fRuE&e= 
>>
>> >
>> >
>> >
>> > _______________________________________________
>> > Aspect-devel mailing list
>> > Aspect-devel at geodynamics.org <mailto:Aspect-devel at geodynamics.org>
>> > 
>> https://urldefense.proofpoint.com/v2/url?u=http-3A__lists.geodynamics.org_cgi-2Dbin_mailman_listinfo_aspect-2Ddevel&d=DwIGaQ&c=Ngd-ta5yRYsqeUsEDgxhcqsYYY1Xs5ogLxWPA_2Wlc4&r=R5lvg9JC99XvuTgScgbY_QFS80R7PEA2q0EPwDy7VQw&m=-Akgl8D9XHMqGKMmF_ZJUPsW7I_-1G6mLeoE2Wpszrc&s=0Gsi6keZz8ri0Vnr9RwBCIGZqCsFqbzuDynW8S3C41U&e=
>>
>> -- 
>> Timo Heister
>> http://www.math.clemson.edu/~heister/ 
>> <http://www.math.clemson.edu/%7Eheister/>
>> _______________________________________________
>> Aspect-devel mailing list
>> Aspect-devel at geodynamics.org <mailto:Aspect-devel at geodynamics.org>
>> http://lists.geodynamics.org/cgi-bin/mailman/listinfo/aspect-devel_______________________________________________
>> Aspect-devel mailing list
>> Aspect-devel at geodynamics.org <mailto:Aspect-devel at geodynamics.org>
>> http://lists.geodynamics.org/cgi-bin/mailman/listinfo/aspect-devel
>
>
>
> _______________________________________________
> Aspect-devel mailing list
> Aspect-devel at geodynamics.org
> http://lists.geodynamics.org/cgi-bin/mailman/listinfo/aspect-devel

-- 
Rene Gassmoeller
http://www.math.colostate.edu/~gassmoel/

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.geodynamics.org/pipermail/aspect-devel/attachments/20170712/27fc31fb/attachment-0001.html>

More information about the Aspect-devel mailing list