[aspect-devel] Aspect stops running after several time steps
Lev Karatun
lev.karatun at gmail.com
Mon Jun 15 22:22:09 PDT 2015
Hi,
one of the models that I'm running gets stuck after 2 time steps. The last
message I see in the log file is: "*** Timestep 3: t=236212 years".
Aspect's processes are still running and taking up resources, but nothing
happens. Similar model, but with 1.5 lower resolution and 1.5 smaller
domain size in x direction runs just fine. I checked the memory usage -
only 1/3 is being used, so this doesn't seem to be a problem. There is
enough free space on hard drive, too.
Could anyone please give me an advice on how I can find the cause of such
stalling?
Thanks in advance. (I attached the log file just in case)
Best regards,
Lev Karatun.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.geodynamics.org/pipermail/aspect-devel/attachments/20150616/3b40aa73/attachment.html>
-------------- next part --------------
-----------------------------------------------------------------------------
-- This is ASPECT, the Advanced Solver for Problems in Earth's ConvecTion.
-- . version 1.3
-- . running in OPTIMIZED mode
-- . running with 32 MPI processes
-- . using Trilinos
-----------------------------------------------------------------------------
Number of active cells: 49,152 (on 5 levels)
Number of degrees of freedom: 3,366,297 (1,241,955+54,417+413,985+413,985+413,985+413,985+413,985)
Number of free surface degrees of freedom: 163251
*** Timestep 0: t=0 years
Solving mesh velocity system... 0 iterations.
Solving temperature system... 0 iterations.
Solving crust system ... 0 iterations.
Solving weak zone system ... 0 iterations.
Solving lithospheric mantle system ... 0 iterations.
Solving sublithospheric mantle system ... 0 iterations.
Rebuilding Stokes preconditioner...
Solving Stokes system... 0+29 iterations.
Postprocessing:
Writing graphical output: try25/solution-00000
RMS, max velocity: 0.00298 m/year, 0.0101 m/year
Number of active cells: 14,208 (on 4 levels)
Number of degrees of freedom: 1,090,604 (402,045+18,484+134,015+134,015+134,015+134,015+134,015)
Number of free surface degrees of freedom: 55452
+---------------------------------------------+------------+------------+
| Total wallclock time elapsed since start | 529s | |
| | | |
| Section | no. calls | wall time | % of total |
+---------------------------------+-----------+------------+------------+
| Assemble Stokes system | 1 | 38.9s | 7.4% |
| Assemble composition system | 4 | 5.83s | 1.1% |
| Assemble temperature system | 1 | 1.77s | 0.33% |
| Build Stokes preconditioner | 1 | 10.5s | 2% |
| Build composition preconditioner| 4 | 1.24s | 0.23% |
| Build temperature preconditioner| 1 | 0.315s | 0.059% |
| Solve Stokes system | 1 | 448s | 85% |
| Solve composition system | 4 | 0.301s | 0.057% |
| Solve temperature system | 1 | 0.076s | 0.014% |
| FreeSurface | 1 | 0.954s | 0.18% |
| Initialization | 2 | 0.738s | 0.14% |
| Postprocessing | 1 | 3.87s | 0.73% |
| Refine mesh structure, part 1 | 1 | 3.79s | 0.72% |
| Refine mesh structure, part 2 | 1 | 0.367s | 0.069% |
| Setup dof systems | 2 | 12s | 2.3% |
+---------------------------------+-----------+------------+------------+
*** Timestep 1: t=78988.3 years
Solving mesh velocity system... 11 iterations.
Solving temperature system... 15 iterations.
Solving crust system ... 19 iterations.
Solving weak zone system ... 18 iterations.
Solving lithospheric mantle system ... 18 iterations.
Solving sublithospheric mantle system ... 17 iterations.
Rebuilding Stokes preconditioner...
Solving Stokes system... 0+20 iterations.
Postprocessing:
Writing graphical output: try25/solution-00001
RMS, max velocity: 0.00355 m/year, 0.0101 m/year
Number of active cells: 10,631 (on 5 levels)
Number of degrees of freedom: 847,178 (312,261+14,482+104,087+104,087+104,087+104,087+104,087)
Number of free surface degrees of freedom: 43446
+---------------------------------------------+------------+------------+
| Total wallclock time elapsed since start | 595s | |
| | | |
| Section | no. calls | wall time | % of total |
+---------------------------------+-----------+------------+------------+
| Assemble Stokes system | 2 | 51.2s | 8.6% |
| Assemble composition system | 8 | 13.2s | 2.2% |
| Assemble temperature system | 2 | 3.82s | 0.64% |
| Build Stokes preconditioner | 2 | 14s | 2.4% |
| Build composition preconditioner| 8 | 1.57s | 0.26% |
| Build temperature preconditioner| 2 | 0.396s | 0.066% |
| Solve Stokes system | 2 | 480s | 81% |
| Solve composition system | 8 | 1.07s | 0.18% |
| Solve temperature system | 2 | 0.244s | 0.041% |
| FreeSurface | 2 | 1.36s | 0.23% |
| Initialization | 2 | 0.738s | 0.12% |
| Postprocessing | 2 | 5.56s | 0.93% |
| Refine mesh structure, part 1 | 2 | 5.24s | 0.88% |
| Refine mesh structure, part 2 | 2 | 0.693s | 0.12% |
| Setup dof systems | 3 | 15.2s | 2.6% |
+---------------------------------+-----------+------------+------------+
*** Timestep 2: t=158131 years
Solving mesh velocity system... 12 iterations.
Solving temperature system... 14 iterations.
Solving crust system ... 19 iterations.
Solving weak zone system ... 19 iterations.
Solving lithospheric mantle system ... 19 iterations.
Solving sublithospheric mantle system ... 17 iterations.
Rebuilding Stokes preconditioner...
Solving Stokes system... 0+21 iterations.
Postprocessing:
Writing graphical output: try25/solution-00002
RMS, max velocity: 0.00378 m/year, 0.0102 m/year
Number of active cells: 10,197 (on 5 levels)
Number of degrees of freedom: 816,709 (301,032+13,957+100,344+100,344+100,344+100,344+100,344)
Number of free surface degrees of freedom: 41871
+---------------------------------------------+------------+------------+
| Total wallclock time elapsed since start | 659s | |
| | | |
| Section | no. calls | wall time | % of total |
+---------------------------------+-----------+------------+------------+
| Assemble Stokes system | 3 | 60.5s | 9.2% |
| Assemble composition system | 12 | 19.1s | 2.9% |
| Assemble temperature system | 3 | 5.54s | 0.84% |
| Build Stokes preconditioner | 3 | 16.8s | 2.5% |
| Build composition preconditioner| 12 | 1.81s | 0.27% |
| Build temperature preconditioner| 3 | 0.455s | 0.069% |
| Solve Stokes system | 3 | 517s | 78% |
| Solve composition system | 12 | 1.69s | 0.26% |
| Solve temperature system | 3 | 0.369s | 0.056% |
| FreeSurface | 3 | 1.71s | 0.26% |
| Initialization | 2 | 0.738s | 0.11% |
| Postprocessing | 3 | 6.99s | 1.1% |
| Refine mesh structure, part 1 | 3 | 6.45s | 0.98% |
| Refine mesh structure, part 2 | 3 | 0.978s | 0.15% |
| Setup dof systems | 4 | 18.4s | 2.8% |
+---------------------------------+-----------+------------+------------+
*** Timestep 3: t=236212 years
More information about the Aspect-devel
mailing list