[aspect-devel] Writing output in parallel

[Wolfgang Bangerth]

> First: the solution to use MPI I/O and merge output files is the only
> way to scale to bigger machines. You can not run with 10'000 cores and
> write out 10'000 files per timestep.
> Second: the merging of files is optional. It is a runtime parameter
> you can set. You might want to generate one file per node (instead of
> one file per core now) or you can leave it as it is today.

I think what Thomas meant to say is that somehow the data has to get 
from wherever you write it to the node that actually stores it. When you 
merge it using (synchronous) MPI I/O, you still have at least one node 
that has to wait till the writing is finished. This will take a long 
time no matter what you do (writing from each node to the central file 
system or writing via MPI I/O) and that the only way to avoid this is 
doing it in the background.

Is there an asynchronous MPI I/O interface where you hand MPI the data 
and then go off doing something else while MPI decides what to do with 
the data?

Best
  W.

------------------------------------------------------------------------
Wolfgang Bangerth               email:            bangerth at math.tamu.edu
                                 www: http://www.math.tamu.edu/~bangerth/