[aspect-devel] Writing output in parallel

[Thomas Geenen]

i am not sure if this will be very efficient on the type of cluster we have.

we have a cluster with a bunch of nodes with fast local io that are
interconnected with infiniband and have an ethernet connection for the io
etc with the master node. On the master node we have our network drive (a
large slow beast, NFS).

in the proposed solution we will be using the infiniband for the io during
the computation (assuming the io will be in the background) how will that
affect the speed of the solver? how large are the mpi buffers needed for
this type of io and is that pinned memory? do we have enough left for the
application?

if its not doing io in the background this will be a bottleneck for us
since we have a very basic disk setup on the master node

locally (on the compute nodes) we have something like 500MB/s throughput so
for a typical run on 10-20 nodes we have an effective bandwith of 5-10GB/s

i would be in favor of implementing a few IO strategies and leave it to the
user to pick the one that is most efficient for his/her hardware setup.

the low tech option i proposed before (write to local storage fast and mv
the files in the background over the ethernet connection to the slow
network drive) will probably work best for me.

cheers
Thomas



On Mon, Feb 27, 2012 at 6:40 PM, Wolfgang Bangerth
<bangerth at math.tamu.edu>wrote:

>
> > Oh, one thing I forgot to mention is that I am not sure if we want one
> > big file per time step. It might happen that paraview in parallel is
> > less efficient reading one big file. One solution would be to write
> > something like n*0.05 files, where n is the number of compute
> > processes.
>
> Yes, go with that. Make the reduction factor (here, 20) a run-time
> parameter so that people can choose whatever is convenient for them when
> visualizing. For single-processor visualization, one could set it equal
> to the number of MPI jobs, for example.
>
> Cheers
>   W.
>
> ------------------------------------------------------------------------
> Wolfgang Bangerth               email:            bangerth at math.tamu.edu
>                                 www: http://www.math.tamu.edu/~bangerth/
>
> _______________________________________________
> Aspect-devel mailing list
> Aspect-devel at geodynamics.org
> http://geodynamics.org/cgi-bin/mailman/listinfo/aspect-devel
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://geodynamics.org/pipermail/aspect-devel/attachments/20120228/24e9a198/attachment.htm