[CIG-SHORT] Pylith stuck in the final step "Formulation finalize."

[hkj at mail.ustc.edu.cn]

Thank you for nice advice, I already used hdf5 output before, but turning off the volume output do help a lot!

There are still two things bothering me:

1. Since I only need two time points: 30yr's and 40yr's output among the whole 40 yr's simulation, but the minimum time step is 0.01 yr. Except changing material properties, can pylith only simulate the model for  dt = 0.01 yr in the first few years, and than gradually get larger in the remaining time period?

2. Is it possible make material properties change with time during the simulation?(For example, let viscosity to be 1.0e+20 during 0~10 year and to be 1.0e+21 during 10~20 year)


Sincerely
Kejing

> -----原始邮件-----
> 发件人: "Brad Aagaard" <baagaard at usgs.gov>
> 发送时间: 2018-10-02 06:52:51 (星期二)
> 收件人: cig-short at geodynamics.org
> 抄送: 
> 主题: Re: [CIG-SHORT] Pylith stuck in the final step "Formulation finalize."
> 
> Kejing Huang,
> 
> When PyLith appears to be stuck, it usually means that at least one 
> process aborted but MPI didn't exit cleanly, so one or more processes 
> are left hanging. It sounds like there is a chance you are running out 
> of disk space.
> 
> If you are not using HDF5 output, the first thing to do is to switch 
> from VTK output (default) to HDF5 output. Secondly, write out only the 
> data fields that you are interested in; if you do not care about volume 
> output, turn it off or only write the final values. See the examples 
> (such as examples/3d/subduction) in the PyLith manual for illustrations 
> of how to do this.
> 
> Regards,
> Brad
> 
> On 09/28/2018 09:28 PM, hkj at mail.ustc.edu.cn wrote:
> > Dear Sir,
> > I'm now running a .exo model with 55000 nodes. In the pylith simulation 
> > I used total time = 50 yr with dt = 0.01yr(since I need viscosity to be 
> > small, 0.01yr is the largest time step as pylith suggested).Thus the 
> > output file is about 80GB.
> > 
> > The simulation and iteration run correctly, but it has about 20% 
> > probability to stuck at the final step after it shows "Formulation 
> > finalize".
> > 
> > I waited for about 10 hours(even longer than the whole simulation time), 
> > but it still stuck at the final step. The last few lines of output is 
> > showed below
> > 
> > /
> > /
> > /``` [OUTPUT]/
> > /......./
> > /......./
> > /......./
> > /Linear solve converged due to CONVERGED_ATOL iterations 50/
> > / >> 
> > /home/collyking/Software/Pylith/lib/python2.7/site-packages/pylith/problems/TimeDependent.py:208:run/
> > / -- timedependent(info)/
> > / -- Finishing advancing solution from t=1.57725e+09*s to t=1.57788e+09*s./
> > / >> 
> > /home/collyking/Software/Pylith/lib/python2.7/site-packages/pylith/problems/Implicit.py:242:poststep/
> > / -- implicit(info)/
> > / -- Writing solution fields./
> > / >> 
> > /home/collyking/Software/Pylith/lib/python2.7/site-packages/pylith/faults/Fault.py:191:writeData/
> > / -- faultcohesivekin(info)/
> > / -- Writing fault data./
> > / >> 
> > /home/collyking/Software/Pylith/lib/python2.7/site-packages/pylith/faults/Fault.py:191:writeData/
> > / -- faultcohesivekin(info)/
> > / -- Writing fault data./
> > / >> 
> > /home/collyking/Software/Pylith/lib/python2.7/site-packages/pylith/problems/Formulation.py:273:finalize/
> > / -- implicit(info)/
> > / -- Formulation finalize./
> > ```
> > 
> > Is it the normal case since the output file is too large?
> > 
> > Sincerely
> > Kejing Huang
> 
> _______________________________________________
> CIG-SHORT mailing list
> CIG-SHORT at geodynamics.org
> http://lists.geodynamics.org/cgi-bin/mailman/listinfo/cig-short