[aspect-devel] pressure normalization
heister at math.tamu.edu
Fri Sep 6 08:18:11 PDT 2013
one thing to keep in mind is that if some function takes a long time
(like compress() or MPI_sum) it is because some of the processors
arrive late. You want to put MPI_Barrier() in front of blocks that you
want to analyse.
On Fri, Sep 6, 2013 at 11:06 AM, Thomas Geenen <geenen at gmail.com> wrote:
> it seems to have been a glitch in the system.
> i reran the experiments and the walltime spend in the pressure_normalization
> has reduces a lot
> i now see timings for all cores similar to the lowest timings in the
> previous run.
> for that run i observed large outliers in the walltime for several cores
> up to 30 times higher with a lot of time spend in the global mpi routines
> MPI::sum etc
> next on the list is the call to compress
> ps i was running between 100 and 200K dfd per core for the timing runs
> On Fri, Aug 30, 2013 at 8:24 PM, Timo Heister <heister at clemson.edu> wrote:
>> > Interesting. The function contains one explicit MPI call, one creation
>> > of a completely distributed vector (should not be very expensive) plus
>> > one copy from a completely distributed to a ghosted vector (should also
>> > not be very expensive). Can you break down which part of the function is
>> > expensive?
>> It might be one of 3 things:
>> 1. Thomas found a work imbalance in this function (as he said, some
>> processor might not have anything to do). This could show up in his
>> instrumentation of processors being idle (but does not mean it takes a
>> significant amount of total runtime).
>> 2. It is instead a work imbalance/issue in the computation that
>> happens before the normalization and the timers are not synchronized
>> 3. He has only very few unknowns per processor which skews the timings.
>> > That
>> > said, I always tried to follow the principle of least surprise, which
>> > would mean to make sure that the pressure is normalized or that the
>> > linear systems are indeed solved to sufficient accuracy.
>> I agree.
>> > Instead of globally relaxing tolerances or switch off pressure
>> > normalization, how about having a section in the manual in which we list
>> > ways to make the code faster if you know what you do? I'll be happy to
>> > write this section.
>> Sounds good. Some more ideas:
>> - use optimized mode :-)
>> - lower order of temperature/compositional discretization
>> - disable postprocessing if not needed
>> Timo Heister
>> Aspect-devel mailing list
>> Aspect-devel at geodynamics.org
More information about the Aspect-devel