On Tue, Sep 18, 2012 at 7:42 PM, Jeffrey Thompson <span dir="ltr"><<a href="mailto:jeffremt@gmail.com" target="_blank">jeffremt@gmail.com</a>></span> wrote:<br><div class="gmail_quote"><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
Hi Brad and Matt,<br><br>Thank you for your rapid responses and suggestions. I've spent the day looking into things, and here's what I've managed to find out:<br><br>Brad, in terms of the mesh, I've run a few models with a uniform mesh size over a range of element lengths, and it seems like in each case, there is still only linear convergence, albeit with fewer iterations on each time step as the mesh is made coarser. However, even at a coarseness beyond a useful level (100 meters spacing, which is not really enough nodes (5) to capture the nature of the applied pressure distribution), I still need 77 iterations to converge in the nonlinear solver. And this is running with the nondimensional scales set equal to the relaxation time, mesh spacing, and shear modulus. This result leads me to think that the linear nature of the convergence is perhaps not related to the mesh. However, your points about my mesh are good and the suggestions are certainly useful in terms of reducing the time it takes the model to iterate and ultimately reach convergence.<br>
<br>Matt, I was able to import the Jacobian for my problem on the coarsest mesh into matlab to look for singularity. The matrix is full-rank and invertible. The calculated condition number from the COND function is 1.7409e+05. And this problem had linear convergence in the nonlinear solver. <br>
</blockquote><div><br></div><div>Okay, Brad explained to me more about your problem. The nonlinearity is coming from frictional constraints, rather than</div><div>material nonlinearities. I now think that we are destroying quadratic convergence, but it is hard to maintain. We are using</div>
<div>a simple gradient method which approximates the true Jacobian and does not handle the constraints exactly either. This</div><div>can result in linear convergence of the nonlinear system. We are investigating more sophisticated solvers (such as those</div>
<div>used for variational inequalities in contact problems), but those will take some doing to get working for this problem. Thus,</div><div>I think the best thing that can be done is to get as good an initial guess as possible right now.</div>
<div><br></div><div> Matt</div><div> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">Thanks again for looking into this for me and for the suggestions,<br>Jeff<div class="HOEnZb">
<div class="h5"><br><br><div class="gmail_quote">On Tue, Sep 18, 2012 at 1:01 PM, Brad Aagaard <span dir="ltr"><<a href="mailto:baagaard@usgs.gov" target="_blank">baagaard@usgs.gov</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">Jeff,<br>
<br>
I suspect your slow convergence (in addition to any solver settings) is<br>
due to your spatial variation of discretization size. Why do you have<br>
such a fine discretization over the entire middle portion of the domain<br>
and then rapid coarsening right near the top and bottom (+y and -y)<br>
edges? You only need fine resolution where there are large strain gradients.<br>
<br>
Carrying around so many extra DOF unnecessarily increases the problem<br>
size and then coarsening right at the edges introduces distorted cells<br>
which decreases the rate of convergence.<br>
<br>
The length scale and shear modulus used in the nondimensionalization<br>
should be comparable to the discretization size and shear modulus in<br>
your problem. Picking ones that minimize the minimum residual is<br>
irrelevant. You want the nondimensionalized displacements and stresses<br>
to on the order of 1.0 in order to produce a well-conditioned system.<br>
<span><font color="#888888"><br>
Brad<br>
</font></span><div><div><br>
<br>
<br>
<br>
On 09/18/2012 11:30 AM, Jeffrey Thompson wrote:<br>
> Hi Cig-Short,<br>
><br>
> I'm having an issue where I'm only able to achieve a slow rate of linear<br>
> convergence using the nonlinear solver, resulting in prohibitively long run<br>
> times. I've attached a tarball with the relevant configuration, mesh, and<br>
> spatialdb files to run my problem.<br>
><br>
> The gist of the problem I'm trying run is an extension of the dyke-opening<br>
> example in the manual, with a horizontal opening under the action of a<br>
> constant pressure (normal traction) distribution between a viscoelastic and<br>
> elastic layer. This is a two dimensional model. I'm using the following<br>
> preconditioner scheme to insure rapid convergence of the linear solver (1-2<br>
> iterations):<br>
><br>
> [pylithapp.petsc]<br>
> ksp_rtol = 1.0e-15<br>
> pc_type = lu<br>
> sub_pc_factor_shift_positive_definite = 0<br>
> sub_pc_factor_shift_nonzero =<br>
><br>
> fs_pc_type = fieldsplit<br>
> fs_pc_fieldsplit_real_diagonal = true<br>
> fs_pc_fieldsplit_type = schur<br>
> fs_pc_fieldsplit_schur_factorization_type = full<br>
><br>
> fs_fieldsplit_0_pc_type = lu<br>
> fs_fieldsplit_0_ksp_type = preonly<br>
> fs_fieldsplit_1_pc_type = jacobi<br>
> fs_fieldsplit_1_ksp_type = gmres<br>
> fs_fieldsplit_1_ksp_rtol = 1.0e-15<br>
> fs_fieldsplit_2_pc_type = jacobi<br>
> fs_fieldsplit_2_ksp_type = gmres<br>
> fs_fieldsplit_2_ksp_rtol = 1.0e-15<br>
> fs_fieldsplit_3_pc_type = jacobi<br>
> fs_fieldsplit_3_ksp_type = gmres<br>
> fs_fieldsplit_3_ksp_rtol = 1.0e-15<br>
><br>
> log_summary = true<br>
> ksp_max_it = 100<br>
> ksp_gmres_restart = 50<br>
> ksp_converged_reason = true<br>
><br>
> snes_ls_monitor = true<br>
> snes_rtol = 1.0e-8<br>
> snes_atol = 1.0e-7<br>
> snes_max_it = 10000<br>
> snes_monitor = true<br>
> snes_converged_reason = true<br>
><br>
> However, as I mentioned earlier, the SNES residual only drops linearly with<br>
> the default solving scheme past the first SNES iteration. I've tried using<br>
> different PETSc options for SNES, but none of the scheme I've tried had<br>
> better results (and many were worse or never converged). To summarize<br>
> these attempts:<br>
> NGMRES: did not converge (residual remained constant)<br>
> NCG: Tried the 5 different flavours, all either diverged or never converged<br>
> (bounced between a few residuals)<br>
> other LINESEARCH options: none were as good as the default behavior<br>
><br>
> I've also tried different non-dimensionalization options and found a set<br>
> that gives the optimal initial residual (i.e., the lowest value of SNES<br>
> residual number 0). It seems like for this problem, only the<br>
> nondimensional length-scale directly impacts the SNES residual.<br>
><br>
> The result of all of this is that I currently need around 800-1000<br>
> non-linear iterations to converge on a single time-step at best. I have a<br>
> feeling that I must be doing something wrong, but I can't tell if this is a<br>
> solver issue or some other type of problem (bad mesh, poorly defined<br>
> boundary conditions, etc). Any advice would be appreciated!<br>
><br>
> Thanks for the help!<br>
> Jeff Thompson<br>
> Caltech Seismolab<br>
><br>
><br>
><br>
</div></div><div><div>> _______________________________________________<br>
> CIG-SHORT mailing list<br>
> <a href="mailto:CIG-SHORT@geodynamics.org" target="_blank">CIG-SHORT@geodynamics.org</a><br>
> <a href="http://geodynamics.org/cgi-bin/mailman/listinfo/cig-short" target="_blank">http://geodynamics.org/cgi-bin/mailman/listinfo/cig-short</a><br>
><br>
<br>
_______________________________________________<br>
CIG-SHORT mailing list<br>
<a href="mailto:CIG-SHORT@geodynamics.org" target="_blank">CIG-SHORT@geodynamics.org</a><br>
<a href="http://geodynamics.org/cgi-bin/mailman/listinfo/cig-short" target="_blank">http://geodynamics.org/cgi-bin/mailman/listinfo/cig-short</a><br>
</div></div></blockquote></div><br>
</div></div></blockquote></div><br><br clear="all"><div><br></div>-- <br>What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead.<br>
-- Norbert Wiener<br>