[aspect-devel] Missing 'restart.resume' files

[Timo Heister]

Matt,

when looking at the code I realized that we never check if the write
of that file succeeded. I created a PR here
https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_geodynamics_aspect_pull_2066&d=DwIBaQ&c=Ngd-ta5yRYsqeUsEDgxhcqsYYY1Xs5ogLxWPA_2Wlc4&r=c08Btfq4m9QEScXN3ZQwLZzzWQE7S8CYq1IYuzKV_Zk&m=ohjkgTNaUvkwB5iVUd2fwRDFowsq4ym-h4C3UqZqOFY&s=oatrJUhueiAy_fa7pAeuOnc_MMbUPx7U4KX18RPsfxg&e=  that changes that. It
probably won't help fixing the problem, but it should fail immediately
instead of silently continuing.

Assuming it is not a bug in our code, there can be several reasons why
writes can fail: a weird/slow network filesystem (check if you are
using a file system recommended to use for output if you are running
on a cluster), not enough free disk space, quotas, etc.

> At the moment I have 2 simulations
> which I need to continue running, and have no way of resuming them.

If the file doesn't exist, you won't be able to continue those runs.
Sorry for not having a "solution". I would try experimenting to find
out when this problem occurs.

Best,
Timo


On Mon, Jan 22, 2018 at 9:03 AM, Matthew Lees <mlees0209 at gmail.com> wrote:
> Hi all,
>
> I'm running simulations with checkpointing enabled, to allow computations to
> be resumed. Normally it works just fine, but sometimes I find that a run
> finishes and no 'restart.resume.z' file is created. I thought I might be
> able to resolve this by renaming 'restart.resume.z.old' to
> 'restart.resume.z' then attempting to resume the simulation, but this
> doesn't seem to work either (it resumes but not from the right point).
>
> Any ideas on how this might be solved? At the moment I have 2 simulations
> which I need to continue running, and have no way of resuming them.
>
> Many thanks,
>
> Matt Lees
> Department of Geophysics
> University of Cambridge
>
> _______________________________________________
> Aspect-devel mailing list
> Aspect-devel at geodynamics.org
> https://urldefense.proofpoint.com/v2/url?u=http-3A__lists.geodynamics.org_cgi-2Dbin_mailman_listinfo_aspect-2Ddevel&d=DwIBaQ&c=Ngd-ta5yRYsqeUsEDgxhcqsYYY1Xs5ogLxWPA_2Wlc4&r=c08Btfq4m9QEScXN3ZQwLZzzWQE7S8CYq1IYuzKV_Zk&m=ohjkgTNaUvkwB5iVUd2fwRDFowsq4ym-h4C3UqZqOFY&s=Z_TksJvYI47H6zBYBn1xLbEhSuS_Rwj1V5fIBcWZhjw&e= 



-- 
Timo Heister
http://www.math.clemson.edu/~heister/