[aspect-devel] A weird error with MPI postprocessing

[Shangxin Liu]

Hi;

Recently, when I'm running the time-dependent cases, sometimes the jobs
will fail at the postprocessing part after running more than ~20 hours. I
paste the error here (from one of my cases):

----------------------------------------------------

Exception on MPI process <76> while running postprocessor
<N6aspect11Postprocess13VisualizationILi3EEE>:


--------------------------------------------------------

An error occurred in line <6156> of file
</home/shangxin/sources/dealii/source/base/data_out_base.cc> in function

    void dealii::DataOutInterface<dim,
spacedim>::write_vtu_in_parallel(const char*, MPI_Comm) const [with int dim
= 3; int spacedim = 3; MPI_Comm = ompi_communicator_t*]

The violated condition was:

    err==0

The name and call sequence of the exception was:

    ExcMessage("Unable to open file with MPI_File_open!")

Additional Information:

Unable to open file with MPI_File_open!

--------------------------------------------------------


Aborting!

----------------------------------------------------

This error often appears after running dozens of hours so it's hard to
debug in short time test. It seems that this error is related with writing
the visualization postprocess output to files. But if so, it means that
after some time step the postprocessing output can proceed but after
another certain time step the postprocessing will not work (In my case, the
code crashed at the time step ~3000 postprocessing). I'm using the ASPECT
and dealii from git hub and didn't modify anything in the postprocessing
code.

Any idea why this problem appears and how to solve it?

Best,
Shangxin
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.geodynamics.org/pipermail/aspect-devel/attachments/20160229/b31a2eae/attachment.html>