[CIG-SHORT] Change in dt means new Jacobian

[Tabrez Ali]

Birendra

You could try a parallel sparse direct solver like MUMPS. When I used 
PyLith a few years back I used MUMPS.

With my own code (also uses PETSc) I am currently solving problems with 
~300000 linear hex elements (with faults) and using MUMPS. It takes 3 
mins for a total of ~30 time steps (though on 24 cores; the first solve 
takes bulk of the time but rest are very fast).

It can be engaged simply by using the flag(s):

-pc_type lu -pc_factor_mat_solver_package mumps

T

On 09/02/2012 01:59 PM, Matthew Knepley wrote:
> On Sun, Sep 2, 2012 at 12:24 PM, Birendra jha <bjha7333 at yahoo.com 
> <mailto:bjha7333 at yahoo.com>> wrote:
>
>     Dear developers
>
>     I am trying to run my Pylith simulation faster. Right now it takes
>     about 3 min for Linear solve (using default KSP solver, about 960
>     iterations) and 3 min for integrating residual. I have about
>     190000 hex cells (mesh is very refined in the center and becomes
>     coarse outward). No faults. Elastic materials. I used non-uniform
>     user specified timesteps.
>
>     I thought of following options:
>
>
> (0) Is this an optimized build?
>
>     (1) Try different solver. I am thinking of using field split
>     preconditioner with multigrid, as in tet4/step02.cfg.
>
>
> If there are no faults, FieldSplit makes no sense. Just use ML. It 
> should take < 10 iterations.
>
>     (2) Switch from vtk to hdf5 output
>
>
> Definitely, also back off the frequency of output.
>
>     (3) Run in parallel. I am looking into running it using sge. Or
>     just running it on one multicore machine.
>
>
> Do bother with this until you figure out how to run in serial.
>
>     (4) Increase linear solve tolerance from 1E-12 to 1E-11.
>
>
> I would try and gauge what tolerance you need by doing some 
> manufactured solutions.
>
>      Matt
>
>     (5) Fix timestep.
>
>     Are these good options to consider?
>
>     I have a question on (5). Manual says
>     Warning: Changing the time step requires recomputing the Jacobian
>     of the system, which can greatly increase the runtime if the
>     time-step size changes frequently.
>
>     Where does this happen in the program, i.e a change in dt setting
>     needNewJacobian=true? I ask this because I could not find message
>     on the stdout about reforming Jacobian when I am using non-uniform
>     dt (it is possible that I missed it, the timesteps remains
>     constant for a while then changes then becomes constant again).
>
>     Thanks and regards
>     Birendra
>     _______________________________________________
>     CIG-SHORT mailing list
>     CIG-SHORT at geodynamics.org <mailto:CIG-SHORT at geodynamics.org>
>     http://geodynamics.org/cgi-bin/mailman/listinfo/cig-short
>
>
>
>
> -- 
> What most experimenters take for granted before they begin their 
> experiments is infinitely more interesting than any results to which 
> their experiments lead.
> -- Norbert Wiener
>
>
> _______________________________________________
> CIG-SHORT mailing list
> CIG-SHORT at geodynamics.org
> http://geodynamics.org/cgi-bin/mailman/listinfo/cig-short

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://geodynamics.org/pipermail/cig-short/attachments/20120902/35932c2d/attachment.htm