[aspect-devel] Performance issue with box geometry

[Jonathan Perry-Houts]

To add to that message, here is the output from two otherwise identical
aspect runs on 1 node, 12 cores, with the current development version of
Aspect:

** With only global refinements:
...
Number of active cells: 262,144 (on 7 levels)
Number of degrees of freedom: 8,861,381 (6,440,067+274,625+2,146,689)
...
+---------------------------------------------+-----------+------------+
| Total wallclock time elapsed since start    |      142s |            |
|                                             |           |            |
| Section                         | no. calls | wall time | % of total |
+---------------------------------+-----------+-----------+------------+
| Assemble Stokes system          |         1 |     64.8s |        46% |
| Assemble temperature system     |         1 |       10s |       7.1% |
| Build Stokes preconditioner     |         1 |     40.3s |        28% |
| Solve Stokes system             |         1 |     2.25s |       1.6% |
| Initialization                  |         1 |     0.31s |      0.22% |
| Postprocessing                  |         1 | 0.000417s |         0% |
| Setup dof systems               |         1 |       20s |        14% |
| Setup initial conditions        |         1 |     2.77s |         2% |
+---------------------------------+-----------+-----------+------------+

** With only repetitions:
...
Number of active cells: 262,144 (on 1 levels)
Number of degrees of freedom: 8,861,381 (6,440,067+274,625+2,146,689)
...
+---------------------------------------------+-----------+------------+
| Total wallclock time elapsed since start    |      212s |            |
|                                             |           |            |
| Section                         | no. calls | wall time | % of total |
+---------------------------------+-----------+-----------+------------+
| Assemble Stokes system          |         1 |     65.1s |        31% |
| Assemble temperature system     |         1 |     9.85s |       4.7% |
| Build Stokes preconditioner     |         1 |     33.6s |        16% |
| Solve Stokes system             |         1 |     2.28s |       1.1% |
| Initialization                  |         1 |     73.4s |        35% |
| Postprocessing                  |         1 | 0.000295s |         0% |
| Setup dof systems               |         1 |     24.5s |        12% |
| Setup initial conditions        |         1 |     2.04s |      0.96% |
+---------------------------------+-----------+-----------+------------+

And the diff of those two parameter files:

$ diff refinement.prm repetitions.prm
4c4
< set Output directory                       = refinement
---
> set Output directory                       = repetitions
14a15,17
>     set X repetitions = 64
>     set Y repetitions = 64
>     set Z repetitions = 64
34c37
<   set Initial global refinement          = 6
---
>   set Initial global refinement          = 0

On 06/09/2017 11:30 AM, Bart Niday wrote:
> Hi everyone,
> 
> I’m running simulations in a 3d box, using specified “Repetitions” for a mesh with an unequal number of cells in the X, Y, and Z-directions. When I specify enough repetitions, ASPECT reports spending a lot of time in setup - for a mesh with 1 million nodes, it’s spending 20% of runtime in “Initialization”.
> In comparison, with the same parameters and mesh size but without specifying repetitions “Initialization” takes less than 1% of time. From performance it looks like specifying a large number of repetitions causes memory usage problems.
> 
> I can get around this by specifying as few repetitions as possible for the aspect ratio I want, and using global mesh refinement instead, but is there a better workaround? 
> 
> I attached some minimum parameter files that reproduce this for me. On my cluster, refinement.prm finishes three times as fast as repetitions.prm and repetitions.prm spends two thirds of runtime in “Initialization”.
> 
> Thanks,
> —
> 
> 
> 
> 
> Bart Niday
> Ph.D. Student
> Department of Earth Sciences, University of Oregon
> 
> 
> 
> _______________________________________________
> Aspect-devel mailing list
> Aspect-devel at geodynamics.org
> http://lists.geodynamics.org/cgi-bin/mailman/listinfo/aspect-devel
>