[CIG-SHORT] Change in dt means new Jacobian

[Matthew Knepley]

On Sun, Sep 2, 2012 at 4:09 PM, Birendra jha <bjha7333 at yahoo.com> wrote:

> Hi Matt,
>
> (0) Yes, it is an optimized build. Installed as per the instructions, with
> no extra flags.
>
> (1) How should I use ML? Is following ok in the pylithapp.cfg :
>
> # PETSc
> [pylithapp.timedependent.formulation]
> matrix_type = aij
> [pylithapp.petsc]
> pc_type = ml
>

This does nothing. Remove it.

> sub_pc_factor_shift_type = nonzero
>



> # Convergence parameters.
> ksp_rtol = 1.0e-10
> ksp_atol = 1.0e-12
> ksp_max_it = 1000
>

This is only to save memory. Make this 500.

> ksp_gmres_restart = 50
>

Put in
ksp_monitor
ksp_view


>
> It is still taking 1000 iterations. I have attached a snapshot of the
> stdout.
>

Always always always always always look at the output of ksp_view. You are
definitely
not running what you think you are running.

Also, if you want help with solver convergence, send the entire output.
Otherwise, you
just get guessing.

    Matt


> Time per solve is increased to 7.5 min. I am running in serial with
> constant dt of 1 day.
>
> Please advise.
>
> Thanks a lot
> Birendra
>
>
> --- On *Mon, 9/3/12, Matthew Knepley <knepley at mcs.anl.gov>* wrote:
>
>
> From: Matthew Knepley <knepley at mcs.anl.gov>
> Subject: Re: [CIG-SHORT] Change in dt means new Jacobian
> To: "Birendra jha" <bjha7333 at yahoo.com>
> Cc: "Brad Aagaard" <baagaard at usgs.gov>, cig-short at geodynamics.org
> Date: Monday, September 3, 2012, 12:29 AM
>
> On Sun, Sep 2, 2012 at 12:24 PM, Birendra jha <bjha7333 at yahoo.com<http://us.mc1650.mail.yahoo.com/mc/compose?to=bjha7333@yahoo.com>
> > wrote:
>
> Dear developers
>
> I am trying to run my Pylith simulation faster. Right now it takes about 3
> min for Linear solve (using default KSP solver, about 960 iterations) and 3
> min for integrating residual. I have about 190000 hex cells (mesh is very
> refined in the center and becomes coarse outward). No faults. Elastic
> materials. I used non-uniform user specified timesteps.
>
> I thought of following options:
>
>
> (0) Is this an optimized build?
>
>
> (1) Try different solver. I am thinking of using field split
> preconditioner with multigrid, as in tet4/step02.cfg.
>
>
> If there are no faults, FieldSplit makes no sense. Just use ML. It should
> take < 10 iterations.
>
>
> (2) Switch from vtk to hdf5 output
>
>
> Definitely, also back off the frequency of output.
>
>
> (3) Run in parallel. I am looking into running it using sge. Or just
> running it on one multicore machine.
>
>
> Do bother with this until you figure out how to run in serial.
>
>
> (4) Increase linear solve tolerance from 1E-12 to 1E-11.
>
>
> I would try and gauge what tolerance you need by doing some manufactured
> solutions.
>
>      Matt
>
> (5) Fix timestep.
>
> Are these good options to consider?
>
> I have a question on (5). Manual says
> Warning: Changing the time step requires recomputing the Jacobian of the
> system, which can greatly increase the runtime if the time-step size
> changes frequently.
>
> Where does this happen in the program, i.e a change in dt setting
> needNewJacobian=true? I ask this because I could not find message on the
> stdout about reforming Jacobian when I am using non-uniform dt (it is
> possible that I missed it, the timesteps remains constant for a while then
> changes then becomes constant again).
>
> Thanks and regards
> Birendra
> _______________________________________________
> CIG-SHORT mailing list
> CIG-SHORT at geodynamics.org<http://us.mc1650.mail.yahoo.com/mc/compose?to=CIG-SHORT@geodynamics.org>
> http://geodynamics.org/cgi-bin/mailman/listinfo/cig-short
>
>
>
>
> --
> What most experimenters take for granted before they begin their
> experiments is infinitely more interesting than any results to which their
> experiments lead.
> -- Norbert Wiener
>
>


-- 
What most experimenters take for granted before they begin their
experiments is infinitely more interesting than any results to which their
experiments lead.
-- Norbert Wiener
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://geodynamics.org/pipermail/cig-short/attachments/20120902/3eaa111d/attachment.htm