[CIG-SHORT] Change in dt means new Jacobian

[Birendra jha]

Hi Matt,
 
(0) Yes, it is an optimized build. Installed as per the instructions, with no extra flags.
 
(1) How should I use ML? Is following ok in the pylithapp.cfg :
 
# PETSc
[pylithapp.timedependent.formulation]
matrix_type = aij
[pylithapp.petsc]
pc_type = ml
sub_pc_factor_shift_type = nonzero
# Convergence parameters.
ksp_rtol = 1.0e-10
ksp_atol = 1.0e-12
ksp_max_it = 1000
ksp_gmres_restart = 50

It is still taking 1000 iterations. I have attached a snapshot of the stdout.
Time per solve is increased to 7.5 min. I am running in serial with constant dt of 1 day.
 
Please advise.
 
Thanks a lot
Birendra
 
 
--- On Mon, 9/3/12, Matthew Knepley <knepley at mcs.anl.gov> wrote:


From: Matthew Knepley <knepley at mcs.anl.gov>
Subject: Re: [CIG-SHORT] Change in dt means new Jacobian
To: "Birendra jha" <bjha7333 at yahoo.com>
Cc: "Brad Aagaard" <baagaard at usgs.gov>, cig-short at geodynamics.org
Date: Monday, September 3, 2012, 12:29 AM


On Sun, Sep 2, 2012 at 12:24 PM, Birendra jha <bjha7333 at yahoo.com> wrote:


Dear developers

I am trying to run my Pylith simulation faster. Right now it takes about 3 min for Linear solve (using default KSP solver, about 960 iterations) and 3 min for integrating residual. I have about 190000 hex cells (mesh is very refined in the center and becomes coarse outward). No faults. Elastic materials. I used non-uniform user specified timesteps.

I thought of following options:



(0) Is this an optimized build?
 
(1) Try different solver. I am thinking of using field split preconditioner with multigrid, as in tet4/step02.cfg.



If there are no faults, FieldSplit makes no sense. Just use ML. It should take < 10 iterations.
 
(2) Switch from vtk to hdf5 output



Definitely, also back off the frequency of output.
 
(3) Run in parallel. I am looking into running it using sge. Or just running it on one multicore machine.



Do bother with this until you figure out how to run in serial.
 
(4) Increase linear solve tolerance from 1E-12 to 1E-11.



I would try and gauge what tolerance you need by doing some manufactured solutions.


     Matt


(5) Fix timestep.

Are these good options to consider?

I have a question on (5). Manual says
Warning: Changing the time step requires recomputing the Jacobian of the system, which can greatly increase the runtime if the time-step size changes frequently.

Where does this happen in the program, i.e a change in dt setting needNewJacobian=true? I ask this because I could not find message on the stdout about reforming Jacobian when I am using non-uniform dt (it is possible that I missed it, the timesteps remains constant for a while then changes then becomes constant again).

Thanks and regards
Birendra
_______________________________________________
CIG-SHORT mailing list
CIG-SHORT at geodynamics.org
http://geodynamics.org/cgi-bin/mailman/listinfo/cig-short




-- 
What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead.
-- Norbert Wiener
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://geodynamics.org/pipermail/cig-short/attachments/20120902/e65ce32a/attachment-0001.htm 
-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
Name: stdout_linearsolve.txt
Url: http://geodynamics.org/pipermail/cig-short/attachments/20120902/e65ce32a/attachment-0001.txt