[aspect-devel] Progress in writing the mantle convection code Aspect

[Katrina Arredondo]

Hi,

  I'm hoping to have active particles (or fields?) that define layers
within a plate, with assigned densities that affect the model. In CitcomS,
I model bouyant basaltic crust, harzburgite and dense eclogite layers that
move with the subducting plate and affect its total bouyancy over periods
of >30 Ma. I hope that helps.

 - Katrina Arredondo


On Tue, Oct 15, 2013 at 9:27 AM, Eric Heien <emheien at ucdavis.edu> wrote:

>
> On Oct 14, 2013, at 1:29 PM, Wolfgang Bangerth wrote:
>
> >> If you have N particles and processor i covers a fraction V_i of the
> >> total domain volume, then processor i creates N*V_i particles.  This
> >> creates a reasonably random looking distribution, but statistically
> >> it's highly non-random because each subdomain has a set number of
> >> particles.
> >
> > That's true, but I think it's not too much of a problem. After all, we
> don't use these particles for random integration or anything else that
> requires truly statistically distributed particles. Basically, we want to
> trace where particles come from and where they go, and so the trajectory is
> more important than the details of their initial distribution. At least,
> that's my view. Do I miss something?
>
> I think we'd need the scientists to weigh in on this.  For tracking
> trajectories I agree having a true random distribution isn't too important.
>  However, many users want to represent material properties with particles
> where a random distribution may be more important.
>
> >> That sounds fine, as long as we make sure the user knows.  Actually
> >> the original idea of generating particles on a single processor and
> >> sending them to other processors will fix all these problems,
> >> including the random seed issue, as well as ensuring results are
> >> identical regardless of the number of processors used.  I might try
> >> coding this up soon to see how complex it is, and with the Generator
> >> class it will be easy to keep this separate from other
> >> implementations.
> >
> > Would this scale? You'd either need to send particles in lots of small
> packages, or store a lot of particles on a single processor...
>
> Good question.  I did a few tests on Stampede and it looks like it will
> scale reasonably well.  If we assume 1e9 particles with id+XYZ (4 doubles =
> 32 bytes) per particle, we need to broadcast ~30 GB from the root process
> in many small packages.
>
> Number of tasks= 256 My rank= 0
> Bcast 131072 doubles (1.000000 MB) takes 0.001406 seconds. 711.060478
> MB/sec
> Bcast 262144 doubles (2.000000 MB) takes 0.003084 seconds. 648.465996
> MB/sec
> Bcast 524288 doubles (4.000000 MB) takes 0.007491 seconds. 533.962737
> MB/sec
>
> Number of tasks= 1024 My rank= 0
> Bcast 131072 doubles (1.000000 MB) takes 0.001876 seconds. 533.071689
> MB/sec
> Bcast 262144 doubles (2.000000 MB) takes 0.003831 seconds. 522.028270
> MB/sec
> Bcast 524288 doubles (4.000000 MB) takes 0.008793 seconds. 454.887641
> MB/sec
>
> Number of tasks= 4096 My rank= 0
> Bcast 131072 doubles (1.000000 MB) takes 0.002373 seconds. 421.451682
> MB/sec
> Bcast 262144 doubles (2.000000 MB) takes 0.004993 seconds. 400.559061
> MB/sec
> Bcast 524288 doubles (4.000000 MB) takes 0.011756 seconds. 340.251219
> MB/sec
>
> Each of these is the average of 100 broadcasts.  At the kind of transfer
> rates we see above, sending 30GB of particle data would take a minute or
> two, which would occur once during initialization.  This can be improved
> further by intelligently sending particles only to processes that have a
> chance of containing them (track bounding boxes at the root process).  And
> transmission likely won't be the slowest part, but rather determining if a
> particle is in a particular cell, which has to occur no matter what method
> we use.
>
> I'll look in the literature to see if there's a better way of doing this
> sort of thing but this seems reasonable from what I can tell.
>
> -Eric
>
> _______________________________________________
> Aspect-devel mailing list
> Aspect-devel at geodynamics.org
> http://geodynamics.org/cgi-bin/mailman/listinfo/aspect-devel
>



-- 

Katrina Arredondo
Ph.D. Candidate
Geodynamics/Subduction Zones
University of California, Davis
Department of Geology
One Shield Avenue, Davis CA 95616
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://geodynamics.org/pipermail/aspect-devel/attachments/20131015/937e4a05/attachment.html>