[CIG-SHORT] Change in dt means new Jacobian

[Birendra jha]

Hi
 
1. Attached is the stdout from my run until now. ksp_view = true and ksp_gmres_restart = 500

2. Should I try direct solver with MUMPS? I will probably have to build Petsc again with --download-mumps=1 because I get error:
Matrix format seqaij does not have a solver package mumps for LU. Perhaps you must ./configure with --download-mumps
 
Thank you Charles for commenting on the role of dt in reforming Jacobian. 
 
Thanks
Birendra

--- On Mon, 9/3/12, Matthew Knepley <knepley at mcs.anl.gov> wrote:


From: Matthew Knepley <knepley at mcs.anl.gov>
Subject: Re: [CIG-SHORT] Change in dt means new Jacobian
To: "Birendra jha" <bjha7333 at yahoo.com>
Cc: "Brad Aagaard" <baagaard at usgs.gov>, cig-short at geodynamics.org
Date: Monday, September 3, 2012, 3:11 AM


On Sun, Sep 2, 2012 at 4:09 PM, Birendra jha <bjha7333 at yahoo.com> wrote:







Hi Matt,
 
(0) Yes, it is an optimized build. Installed as per the instructions, with no extra flags.
 
(1) How should I use ML? Is following ok in the pylithapp.cfg :
 
# PETSc
[pylithapp.timedependent.formulation]
matrix_type = aij
[pylithapp.petsc]
pc_type = ml



This does nothing. Remove it.





sub_pc_factor_shift_type = nonzero



 





# Convergence parameters.
ksp_rtol = 1.0e-10
ksp_atol = 1.0e-12
ksp_max_it = 1000



This is only to save memory. Make this 500. 





ksp_gmres_restart = 50


Put in
ksp_monitor
ksp_view
 






It is still taking 1000 iterations. I have attached a snapshot of the stdout.



Always always always always always look at the output of ksp_view. You are definitely
not running what you think you are running.


Also, if you want help with solver convergence, send the entire output. Otherwise, you
just get guessing.


    Matt
 





Time per solve is increased to 7.5 min. I am running in serial with constant dt of 1 day.
 
Please advise.
 
Thanks a lot
Birendra
 
 
--- On Mon, 9/3/12, Matthew Knepley <knepley at mcs.anl.gov> wrote:


From: Matthew Knepley <knepley at mcs.anl.gov>
Subject: Re: [CIG-SHORT] Change in dt means new Jacobian
To: "Birendra jha" <bjha7333 at yahoo.com>
Cc: "Brad Aagaard" <baagaard at usgs.gov>, cig-short at geodynamics.org
Date: Monday, September 3, 2012, 12:29 AM


On Sun, Sep 2, 2012 at 12:24 PM, Birendra jha <bjha7333 at yahoo.com> wrote:


Dear developers

I am trying to run my Pylith simulation faster. Right now it takes about 3 min for Linear solve (using default KSP solver, about 960 iterations) and 3 min for integrating residual. I have about 190000 hex cells (mesh is very refined in the center and becomes coarse outward). No faults. Elastic materials. I used non-uniform user specified timesteps.

I thought of following options:



(0) Is this an optimized build?
 
(1) Try different solver. I am thinking of using field split preconditioner with multigrid, as in tet4/step02.cfg.



If there are no faults, FieldSplit makes no sense. Just use ML. It should take < 10 iterations.
 
(2) Switch from vtk to hdf5 output



Definitely, also back off the frequency of output.
 
(3) Run in parallel. I am looking into running it using sge. Or just running it on one multicore machine.



Do bother with this until you figure out how to run in serial.
 
(4) Increase linear solve tolerance from 1E-12 to 1E-11.



I would try and gauge what tolerance you need by doing some manufactured solutions.


     Matt


(5) Fix timestep.

Are these good options to consider?

I have a question on (5). Manual says
Warning: Changing the time step requires recomputing the Jacobian of the system, which can greatly increase the runtime if the time-step size changes frequently.

Where does this happen in the program, i.e a change in dt setting needNewJacobian=true? I ask this because I could not find message on the stdout about reforming Jacobian when I am using non-uniform dt (it is possible that I missed it, the timesteps remains constant for a while then changes then becomes constant again).

Thanks and regards
Birendra
_______________________________________________
CIG-SHORT mailing list
CIG-SHORT at geodynamics.org
http://geodynamics.org/cgi-bin/mailman/listinfo/cig-short




-- 
What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead.
-- Norbert Wiener




-- 
What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead.
-- Norbert Wiener
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://geodynamics.org/pipermail/cig-short/attachments/20120902/767ce4c7/attachment-0001.htm 
-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
Name: output.txt
Url: http://geodynamics.org/pipermail/cig-short/attachments/20120902/767ce4c7/attachment-0001.txt