[aspect-devel] Aspect hangs after several time steps
Lev Karatun
lev.karatun at gmail.com
Thu Feb 25 19:19:33 PST 2016
Hi Rene,
adding the line(s) did help, thank you so much! And thanks to Ian for the
quick fix!
I beleive it should be
> for (unsigned int c=0; c<in.composition[i].size(); ++c)
> out.reaction_terms[i][c] = 0.0;
(correct me if I'm wrong)
I was trying to create a pull request but ran into problems while pairing
git on my machine with the online service, so if you could make a pull
request for me, it would be great.
Thanks again.
Best regards,
Lev Karatun.
2016-02-23 17:58 GMT-05:00 Rene Gassmoeller <rene.gassmoeller at mailbox.org>:
> Hi Lev,
> sorry for the compilation trouble. The issues should be fixed now
> (courtesy of Ian, https://github.com/geodynamics/aspect/pull/768).
>
> Your model actually discovered something like a bug, I do not exactly
> know how it ran on Timo's machine. Did you activate the compositional
> fields Timo?
> The problem is that the 'Simpler' material model is not using
> compositional fields, and does not consider that the model might still be
> solved with passive compositional fields. Therefore the 'reaction_terms'
> variable, which describes the change of the compositional fields over one
> timestep remains unfilled and contains the uninitialized value 'NaN'.
> There are two solutions to this problem: We can either forbid the use of
> compositional fields with the 'Simpler' material model and crash with a
> better error message, or (my preferred solution) just ignore the
> compositional fields in the material model, but make sure the model runs.
> You can get the second solution by adding the line
>
> out.reaction_terms[i] = 0.0;
>
> to source/material_model/simpler.cc:68, and recompiling.
> Could you check if that works? In case it does: Feel free to pack it into
> a pull request and submit it to github if you know how to do this (we want
> to write a section about this in the manual for a long time), but otherwise
> I can do that for you.
>
> Let me know if it works,
> Rene
>
> On 02/23/2016 01:19 PM, Lev Karatun wrote:
>
> Hi Timo,
>
> make test on the version that I was using (1.4.0-pre) produces the
> following output:
>
>
> Running tests...
> Test project /home/lev/aspect/aspect_debug_old
> Start 1: quick_mpi
> 1/1 Test #1: quick_mpi ........................***Not Run 0.00 sec
> 0% tests passed, 1 tests failed out of 1
> Total Test time (real) = 0.00 sec
> The following tests FAILED:
> 1 - quick_mpi (BAD_COMMAND)
>
>
> Not sure what you mean by other examples.. other cookbooks? I didn't try
> all of them but the couple that I tried worked fine.
>
> I'm trying to install the recent version with the bug fix, but it doesn't
> compile =(
>
>
> /home/lev/aspect/aspect_debug/source/simulator/assembly.cc:1767: error:
> ‘struct aspect::internal::Assembly::Scratch::AdvectionSystem<3>’ has no
> member named ‘face_finite_element_values’
> /home/lev/aspect/aspect_debug/source/simulator/assembly.cc:163: error:
> class ‘aspect::internal::Assembly::Scratch::AdvectionSystem<dim>’ does not
> have any field named ‘face_finite_element_values’
>
> and other errors, I attached the full log. Could you have a look please?
>
>
> Best regards,
> Lev Karatun.
>
> 2016-02-23 8:32 GMT-05:00 Timo Heister <heister at clemson.edu> <heister at clemson.edu>:
>
>
> Hey Lev,
>
> running your test 135.prm I discovered a bug introduced over the last
> few days, see https://github.com/geodynamics/aspect/pull/767
>
> But with that fix, your tests runs fine here. What version of ASPECT
> are you using? What does "make test" report? Do other examples work
> for you?
>
>
> On Mon, Feb 22, 2016 at 5:59 PM, Lev Karatun <lev.karatun at gmail.com> <lev.karatun at gmail.com>
> wrote:
>
> Hi again,
>
> so I simplified the setup (prm file attached) to the point when I have a
>
> box
>
> with a single compositional field, tangential velocity b.c. for all 6
>
> faces,
>
> constant temperature throughout the entire box - so I'm literally
>
> modelling
>
> a constant state of material with nothing happening. And the simulation
> fails with the error " The residual in the last step was nan". However,
>
> if I
>
> delete the compositional fields from the input file, the simulation runs
> fine.
> What can be causing this?..
>
> Best regards,
> Lev Karatun.
>
> 2016-02-22 0:07 GMT-05:00 Lev Karatun <lev.karatun at gmail.com> <lev.karatun at gmail.com>:
>
> Hi Timo,
>
> I've simplified the model to the extreme, and still the solver doesn't
> converge (The residual in the last step was nan). It's not even using
>
> any of
>
> the plugins that I used, so I'm really confused. The cookbooks,
>
> however, run
>
> without problems. (and my model in release mode runs successfully too)
>
> I'm trying to run the models on a lab computer, single core. I attached
> the prm file that I'm using, could you have a look please?
> Thanks in advance!
>
> Best regards,
> Lev Karatun.
>
> 2016-02-12 9:10 GMT-05:00 Timo Heister <heister at clemson.edu> <heister at clemson.edu>:
>
> Lev,
>
> your system setup (compiler, operating system, standard libraries)
> doesn't allow for the debugging trick to detect floating point
> exceptions. You could try a recent version of clang (or gcc if you
> were using clang) and see. Sorry, I don't know a sure way to get this
> enabled (I have no problems with it on ubuntu 14.04 using gcc or
> clang).
>
> That said, convergence failures can have other reasons not detectable
> using this technique. It is hard to tell, but you should probably
> simplify your test problem and if it works, increase the complexity
> one by one. Do the included files like convection-box or shell 2d/3d
> work correctly on your machine (is it a cluster?)?
>
> Best,
> Timo
>
> On Fri, Feb 12, 2016 at 5:21 AM, Lev Karatun <lev.karatun at gmail.com> <lev.karatun at gmail.com>
> wrote:
>
> Hi Wolfgang,
>
> I fixed the issues that Timo pointed out, but I'm still getting the
> "did not
> converge" error. Is there a way for me to debug it the same way Timo
> did?
> What am I missing? A compiler with C++11 support? Something else?
>
> Best regards,
> Lev Karatun.
>
> 2016-02-03 9:20 GMT-05:00 Wolfgang Bangerth <bangerth at tamu.edu> <bangerth at tamu.edu>:
>
>
> Lev,
> this is complicated.
>
> You switched on floating point signals by hand, but your compiler
> support
> library has functions that trip this, so your program gets stopped
>
> in
>
> a
> place that is outside anyone's control (and where it doesn't
>
> actually
>
> matter
> much). This is why floating point exceptions were disabled
> automatically
> when you ran the ASPECT cmake script. In other words, to make
> progress, you
> need to disable floating point exceptions again.
>
> Of course, without being able to run FPE on your system, you won't
>
> be
>
> able
> to reproduce the issues Timo found on his system. But you don't have
> to find
> these issues -- he already did that for you. Just fix these places
>
> and
>
> you
> should be able to make progress.
>
> Best
> W.
>
>
> On 02/01/2016 11:00 PM, Lev Karatun wrote:
>
> Hi Wolfgang,
>
> I've tried this, the last stack frame is:
>
> #13 0x00000000012e50d5 in main (argc=2, argv=0x7fffffffe128) at
> /home/lev/aspect/aspect_debug_new/source/main.cc:513
>
>
> The 513 line in main.cc is:
>
> aspect::Simulator<3>::declare_parameters(prm);
>
>
> Looks like it stops at the attempt of reading the Start time line:
>
> #10 0x00007ffff537a569 in
>
> dealii::ParameterHandler::declare_entry
>
> (this=0x7fffffffdc90, entry="Start time", default_value="0",
> pattern=...,
> documentation=
> "The start time of the simulation. Units: Years if the
>
> 'Use
>
> years in
> output instead of seconds' parameter is set; seconds
>
> otherwise.")
>
> at
>
>
>
>
> /home/lev/distrib/dealii_debug_new/source/base/parameter_handler.cc:1628
>
>
> but it's just "set Start time = 0", I never changed it.
>
> Full backtrace:
>
> Program received signal SIGFPE, Arithmetic exception.
> __mpn_lshift () at ../sysdeps/x86_64/lshift.S:26
> 26 movq -8(%rsi,%rdx,8), %mm7
> (gdb) bt
> #0 __mpn_lshift () at ../sysdeps/x86_64/lshift.S:26
> #1 0x0000003db304a53e in ___printf_fp (fp=0x7fffffff71b0,
> info=0x7fffffff70b0, args=<value optimized out>) at
> printf_fp.c:483
> #2 0x0000003db30458a0 in _IO_vfprintf_internal (s=<value
> optimized
> out>,
> format=<value optimized out>, ap=<value optimized out>) at
> vfprintf.c:1640
> #3 0x0000003db306f752 in _IO_vsnprintf (string=0x7fffffff7420
> "",
> maxlen=<value optimized out>, format=0x7fffffff74f0 "%.*g",
> args=0x7fffffff7310)
> at vsnprintf.c:120
> #4 0x0000003dbd87eb4f in std::__convert_from_v (__cloc=<value
> optimized
> out>, __out=0x7fffffff7420 "", __size=45, __fmt=0x7fffffff74f0
> "%.*g")
> at
>
>
>
>
> /usr/src/debug/gcc-4.4.7-20120601/obj-x86_64-redhat-linux/x86_64-redhat-linux/libstdc++-v3/include/x86_64-redhat-linux/bits/c++locale.h:89
>
> #5 0x0000003dbd880f23 in std::num_put<char,
> std::ostreambuf_iterator<char, std::char_traits<char> >
> >::_M_insert_float<double> (this=0x3dbdaf22e0, __s=..., __io=
> ..., __fill=32 ' ', __mod=<value optimized out>,
> __v=-1.7976931348623157e+308)
> at
>
>
>
>
> /usr/src/debug/gcc-4.4.7-20120601/obj-x86_64-redhat-linux/x86_64-redhat-linux/libstdc++-v3/include/bits/locale_facets.tcc:980
>
> #6 0x0000003dbd881249 in std::num_put<char,
> std::ostreambuf_iterator<char, std::char_traits<char> >
>
> ::do_put
>
> (this=<value optimized out>, __s=...,
> __io=<value optimized out>, __fill=<value optimized out>,
> __v=<value
> optimized out>)
> at
>
>
>
>
> /usr/src/debug/gcc-4.4.7-20120601/obj-x86_64-redhat-linux/x86_64-redhat-linux/libstdc++-v3/include/bits/locale_facets.tcc:1127
>
> #7 0x0000003dbd89487f in put (this=0x7fffffff7630,
> __v=-1.7976931348623157e+308)
> at
>
>
>
>
> /usr/src/debug/gcc-4.4.7-20120601/obj-x86_64-redhat-linux/x86_64-redhat-linux/libstdc++-v3/include/bits/locale_facets.h:2390
>
> #8 std::basic_ostream<char, std::char_traits<char>
>
> ::_M_insert<double>
>
> (this=0x7fffffff7630, __v=-1.7976931348623157e+308)
> at
>
>
>
>
> /usr/src/debug/gcc-4.4.7-20120601/obj-x86_64-redhat-linux/x86_64-redhat-linux/libstdc++-v3/include/bits/ostream.tcc:73
>
> #9 0x00007ffff5373468 in dealii::Patterns::Double::description
> (this=0x1f92620) at
>
>
>
>
> /home/lev/distrib/dealii_debug_new/source/base/parameter_handler.cc:290
>
> #10 0x00007ffff537a569 in
>
> dealii::ParameterHandler::declare_entry
>
> (this=0x7fffffffdc90, entry="Start time", default_value="0",
> pattern=...,
> documentation=
> "The start time of the simulation. Units: Years if the
>
> 'Use
>
> years in
> output instead of seconds' parameter is set; seconds
>
> otherwise.")
>
> at
>
>
>
>
> /home/lev/distrib/dealii_debug_new/source/base/parameter_handler.cc:1628
>
> #11 0x000000000103fef2 in
> aspect::Parameters<3>::declare_parameters
> (prm=...) at
>
> /home/lev/aspect/aspect_debug_new/source/simulator/parameters.cc:88
> #12 0x000000000104e2e8 in
> aspect::Simulator<3>::declare_parameters
> (prm=...) at
>
>
>
> /home/lev/aspect/aspect_debug_new/source/simulator/parameters.cc:1283
>
> #13 0x00000000012e50d5 in main (argc=2, argv=0x7fffffffe128) at
> /home/lev/aspect/aspect_debug_new/source/main.cc:513
>
>
>
> Best regards,
> Lev Karatun.
>
> 2016-02-01 21:34 GMT-05:00 Wolfgang Bangerth <bangerth at tamu.edu<mailto:bangerth at tamu.edu> <bangerth at tamu.edu>>:
>
> On 02/01/2016 06:42 PM, Lev Karatun wrote:
>
>
> Could you please tell me what I'm doing wrong?..
> Thanks in advance.
>
>
> I don't think you *need* any of these packages. You may not get
> to
> see the
> exact location where the error happened in BLAS or some other
> low-level
> library, but ultimately you only want to know the place where
>
> you
>
> were in
> your own code. In other words, if the debugger stops at the
> location
> where
> the error happens, get a backtrace and go to the last stack
>
> frame
>
> in
> your
> code. Your task is then to find out what is wrong in that line.
>
> Best
> W.
>
> --
>
>
>
>
> ------------------------------------------------------------------------
>
> Wolfgang Bangerth email: bangerth at math.tamu.edu
> <mailto:bangerth at math.tamu.edu> <bangerth at math.tamu.edu>
> www:http://www.math.tamu.edu/~bangerth/
>
> _______________________________________________
> Aspect-devel mailing list
> Aspect-devel at geodynamics.org<mailto:Aspect-devel at geodynamics.org> <Aspect-devel at geodynamics.org>
> http://lists.geodynamics.org/cgi-bin/mailman/listinfo/aspect-devel
>
>
>
>
> _______________________________________________
> Aspect-devel mailing listAspect-devel at geodynamics.orghttp://lists.geodynamics.org/cgi-bin/mailman/listinfo/aspect-devel
>
>
> --
>
>
>
> ------------------------------------------------------------------------
>
> Wolfgang Bangerth email:bangerth at math.tamu.edu
> www:http://www.math.tamu.edu/~bangerth/
>
> _______________________________________________
> Aspect-devel mailing listAspect-devel at geodynamics.orghttp://lists.geodynamics.org/cgi-bin/mailman/listinfo/aspect-devel
>
>
>
> _______________________________________________
> Aspect-devel mailing listAspect-devel at geodynamics.orghttp://lists.geodynamics.org/cgi-bin/mailman/listinfo/aspect-devel
>
>
>
> --
> Timo Heisterhttp://www.math.clemson.edu/~heister/
> _______________________________________________
> Aspect-devel mailing listAspect-devel at geodynamics.orghttp://lists.geodynamics.org/cgi-bin/mailman/listinfo/aspect-devel
>
>
> _______________________________________________
> Aspect-devel mailing listAspect-devel at geodynamics.orghttp://lists.geodynamics.org/cgi-bin/mailman/listinfo/aspect-devel
>
>
>
> --
> Timo Heisterhttp://www.math.clemson.edu/~heister/
> _______________________________________________
> Aspect-devel mailing listAspect-devel at geodynamics.orghttp://lists.geodynamics.org/cgi-bin/mailman/listinfo/aspect-devel
>
>
>
> _______________________________________________
> Aspect-devel mailing listAspect-devel at geodynamics.orghttp://lists.geodynamics.org/cgi-bin/mailman/listinfo/aspect-devel
>
>
>
> _______________________________________________
> Aspect-devel mailing list
> Aspect-devel at geodynamics.org
> http://lists.geodynamics.org/cgi-bin/mailman/listinfo/aspect-devel
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.geodynamics.org/pipermail/aspect-devel/attachments/20160225/8c7a705b/attachment-0001.html>
More information about the Aspect-devel
mailing list