[aspect-devel] Progress in writing the mantle convection code Aspect

[Wolfgang Bangerth]

> My concern is about the checkpoint file getting too big, and also (I
> believe) the serial writing of the checkpoint file.  If you have 1e9
> particles on 1e4 processors, it seems there will be at least 30GB of
> particle data written sequentially from many points in the system,
> which may become a bottleneck during the checkpoints.  For now I'll
> just write in the checkpoint file and see how it does on large runs.

Yes, let's just go with the simple solution for now. We can stripe 
writing to this file via MPI I/O if necessary if we don't already.


> If you have N particles and processor i covers a fraction V_i of the
> total domain volume, then processor i creates N*V_i particles.  This
> creates a reasonably random looking distribution, but statistically
> it's highly non-random because each subdomain has a set number of
> particles.

That's true, but I think it's not too much of a problem. After all, we 
don't use these particles for random integration or anything else that 
requires truly statistically distributed particles. Basically, we want 
to trace where particles come from and where they go, and so the 
trajectory is more important than the details of their initial 
distribution. At least, that's my view. Do I miss something?

With a weighting function (a probability distribution), all you'd need 
to do is replace V_i above from the volume owned by processor i, i.e.
   V_i = \int_{\Omega_i}  1  dx
to the integral of the probability density,
   V_i = \int_{\Omega_i}  p(x)  dx


> That sounds fine, as long as we make sure the user knows.  Actually
> the original idea of generating particles on a single processor and
> sending them to other processors will fix all these problems,
> including the random seed issue, as well as ensuring results are
> identical regardless of the number of processors used.  I might try
> coding this up soon to see how complex it is, and with the Generator
> class it will be easy to keep this separate from other
> implementations.

Would this scale? You'd either need to send particles in lots of small 
packages, or store a lot of particles on a single processor...

Cheers
  W.



-- 
------------------------------------------------------------------------
Wolfgang Bangerth               email:            bangerth at math.tamu.edu
                                 www: http://www.math.tamu.edu/~bangerth/