[CIG-SHORT] Change in dt means new Jacobian

[Matthew Knepley]

On Sun, Sep 2, 2012 at 7:32 PM, Birendra jha <bjha7333 at yahoo.com> wrote:

> Hi
>
> 1. Attached is the stdout from my run until now. ksp_view = true and
> ksp_gmres_restart = 500
> 2. Should I try direct solver with MUMPS? I will probably have to build
> Petsc again with --download-mumps=1 because I get error:
> Matrix format seqaij does not have a solver package mumps for LU. Perhaps
> you must ./configure with --download-mumps
>

You can try MUMPS, but you have a much deeper problem. ML will solve linear
elasticity in < 20 iterates.
Your system is misspecified somehow. It has a huge condition number/near
null space, which are the same
thing. This is most likely why you cannot reduce the residual below 1e-10.
I would question any results you
get until you understand what the problem is with this setup.

If you do not agree, try our benchmark problems with uniform refinement.
You can easily make the problem
grow to 500K unknowns, and ML for the elastic problem will be fine.

    Matt


>
> Thank you Charles for commenting on the role of dt in reforming Jacobian.
>
> Thanks
> Birendra
>
> --- On *Mon, 9/3/12, Matthew Knepley <knepley at mcs.anl.gov>* wrote:
>
>
> From: Matthew Knepley <knepley at mcs.anl.gov>
> Subject: Re: [CIG-SHORT] Change in dt means new Jacobian
> To: "Birendra jha" <bjha7333 at yahoo.com>
> Cc: "Brad Aagaard" <baagaard at usgs.gov>, cig-short at geodynamics.org
> Date: Monday, September 3, 2012, 3:11 AM
>
> On Sun, Sep 2, 2012 at 4:09 PM, Birendra jha <bjha7333 at yahoo.com<http://us.mc1650.mail.yahoo.com/mc/compose?to=bjha7333@yahoo.com>
> > wrote:
>
>   Hi Matt,
>
> (0) Yes, it is an optimized build. Installed as per the instructions, with
> no extra flags.
>
> (1) How should I use ML? Is following ok in the pylithapp.cfg :
>
> # PETSc
> [pylithapp.timedependent.formulation]
> matrix_type = aij
> [pylithapp.petsc]
> pc_type = ml
>
>
> This does nothing. Remove it.
>
>   sub_pc_factor_shift_type = nonzero
>
>
>
>
>   # Convergence parameters.
> ksp_rtol = 1.0e-10
> ksp_atol = 1.0e-12
> ksp_max_it = 1000
>
>
> This is only to save memory. Make this 500.
>
>   ksp_gmres_restart = 50
>
>
> Put in
> ksp_monitor
> ksp_view
>
>
>
> It is still taking 1000 iterations. I have attached a snapshot of the
> stdout.
>
>
> Always always always always always look at the output of ksp_view. You are
> definitely
> not running what you think you are running.
>
> Also, if you want help with solver convergence, send the entire output.
> Otherwise, you
> just get guessing.
>
>     Matt
>
>
>   Time per solve is increased to 7.5 min. I am running in serial with
> constant dt of 1 day.
>
> Please advise.
>
> Thanks a lot
> Birendra
>
>
> --- On *Mon, 9/3/12, Matthew Knepley <knepley at mcs.anl.gov<http://us.mc1650.mail.yahoo.com/mc/compose?to=knepley@mcs.anl.gov>
> >* wrote:
>
>
> From: Matthew Knepley <knepley at mcs.anl.gov<http://us.mc1650.mail.yahoo.com/mc/compose?to=knepley@mcs.anl.gov>
> >
> Subject: Re: [CIG-SHORT] Change in dt means new Jacobian
> To: "Birendra jha" <bjha7333 at yahoo.com<http://us.mc1650.mail.yahoo.com/mc/compose?to=bjha7333@yahoo.com>
> >
> Cc: "Brad Aagaard" <baagaard at usgs.gov<http://us.mc1650.mail.yahoo.com/mc/compose?to=baagaard@usgs.gov>>,
> cig-short at geodynamics.org<http://us.mc1650.mail.yahoo.com/mc/compose?to=cig-short@geodynamics.org>
> Date: Monday, September 3, 2012, 12:29 AM
>
> On Sun, Sep 2, 2012 at 12:24 PM, Birendra jha <bjha7333 at yahoo.com<http://us.mc1650.mail.yahoo.com/mc/compose?to=bjha7333@yahoo.com>
> > wrote:
>
> Dear developers
>
> I am trying to run my Pylith simulation faster. Right now it takes about 3
> min for Linear solve (using default KSP solver, about 960 iterations) and 3
> min for integrating residual. I have about 190000 hex cells (mesh is very
> refined in the center and becomes coarse outward). No faults. Elastic
> materials. I used non-uniform user specified timesteps.
>
> I thought of following options:
>
>
> (0) Is this an optimized build?
>
>
> (1) Try different solver. I am thinking of using field split
> preconditioner with multigrid, as in tet4/step02.cfg.
>
>
> If there are no faults, FieldSplit makes no sense. Just use ML. It should
> take < 10 iterations.
>
>
> (2) Switch from vtk to hdf5 output
>
>
> Definitely, also back off the frequency of output.
>
>
> (3) Run in parallel. I am looking into running it using sge. Or just
> running it on one multicore machine.
>
>
> Do bother with this until you figure out how to run in serial.
>
>
> (4) Increase linear solve tolerance from 1E-12 to 1E-11.
>
>
> I would try and gauge what tolerance you need by doing some manufactured
> solutions.
>
>      Matt
>
> (5) Fix timestep.
>
> Are these good options to consider?
>
> I have a question on (5). Manual says
> Warning: Changing the time step requires recomputing the Jacobian of the
> system, which can greatly increase the runtime if the time-step size
> changes frequently.
>
> Where does this happen in the program, i.e a change in dt setting
> needNewJacobian=true? I ask this because I could not find message on the
> stdout about reforming Jacobian when I am using non-uniform dt (it is
> possible that I missed it, the timesteps remains constant for a while then
> changes then becomes constant again).
>
> Thanks and regards
> Birendra
> _______________________________________________
> CIG-SHORT mailing list
> CIG-SHORT at geodynamics.org<http://us.mc1650.mail.yahoo.com/mc/compose?to=CIG-SHORT@geodynamics.org>
> http://geodynamics.org/cgi-bin/mailman/listinfo/cig-short
>
>
>
>
> --
> What most experimenters take for granted before they begin their
> experiments is infinitely more interesting than any results to which their
> experiments lead.
> -- Norbert Wiener
>
>
>
>
> --
> What most experimenters take for granted before they begin their
> experiments is infinitely more interesting than any results to which their
> experiments lead.
> -- Norbert Wiener
>
>


-- 
What most experimenters take for granted before they begin their
experiments is infinitely more interesting than any results to which their
experiments lead.
-- Norbert Wiener
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://geodynamics.org/pipermail/cig-short/attachments/20120902/595bf98e/attachment.htm