[aspect-devel] Open boundaries + tracers

Wed Nov 30 15:19:41 PST 2016

Hi Lev, Hi John,

Sorry for the late reply. John's suggestion should be indeed the 
solution to your problem Lev.

Currently, the most reasonable setting for the load balancing strategy 
is 'repartition' (the 'balanced repartition' method I talked about in 
the webinar) if you do not need to generate new tracers / remove old 
ones, and 'repartition, add tracers, remove tracers' if you get empty 
cells, or so largely unbalanced particle numbers that the particle 
algorithm takes too much time. This happens for example if you have 
'many' different levels of cells, and 'many' particles, with the 
definition of 'many' depending on your model size and available 
computing time ;-).

The documentation of the particle load balancing schemes is currently 
not as detailed as I would like. I need extend that at some point.

About John's question: The tracer interpolation itself is not handled in 
the "Stokes solve" part, it is happening in the "Solve composition 
system" part of the timer and is only done once per nonlinear iteration. 
My guess would be that setting up your model with tracers creates 
sharper gradients in whatever property they track, and therefore the 
Stokes solution is harder to obtain. Does the number of Stokes 
iterations change between the two methods?

Best,

Rene

On 11/29/2016 10:07 AM, John Naliboff wrote:
> Hi Lev,
>
> You should set your load balancing strategy to “remove and add 
> particles, repartition”.  If you do not include the “repartition” 
> option, the majority of your model time will likely be spent with 
> particle sorting if you are using multiple processors.
>
> For example, in one case without the ‘repartition’ option the time 
> spent in particle sorting is roughly 80-90% (16 processors, ~ 1e6 
> particles, fixed grid) as opposed to 20% with the ‘repartition’ option.
>
> On a related note, I’ve found using ‘active tracers’ in replace of 
> compositional fields increases the ’Solve Stokes system’ time by over 
> an order of magnitude.  At least this is the case in the ‘extension’ 
> models I’ve been running.
>
> I haven’t had time to track down where exactly in the "Stokes solve" 
> this time is being eaten up or look at similar time increases in other 
> models, but it will be interesting to see if this is the case in your 
> models as well.  Certainly keep on eye on it before sending any large 
> models off to a cluster.
>
> Rene - Is this possibly due to re-interpolation of particle values 
> during each stokes solve?
>
> Cheers,
> John
>
> *************************************************
> John Naliboff
> Assistant Project Scientist, CIG
> Earth & Planetary Sciences Dept., UC Davis
>
>
>
>
>
>
>> On Nov 28, 2016, at 6:22 PM, Lev Karatun <lev.karatun at gmail.com 
>> <mailto:lev.karatun at gmail.com>> wrote:
>>
>> Hi Rene,
>>
>> thanks a lot, I actually had the load balancing strategy set to 
>> "repartition", changing it to "remove and add particles" solved the 
>> problem. The manual only lists the possible choices, but doesn't 
>> explain what they do, so I originally went with the default option.
>> Could you get into some more detail on the strategies, please? Do I 
>> understand it correctly that the option "add particles" enforces the 
>> minimum number of particles in the cell, generating them whenever 
>> needed? What would be the difference between "set Load balancing 
>> strategy = add particles, remove particles" and ""set Load balancing 
>> strategy = add tracers, remove tracers"? And how will the 
>> "repartition" strategy work together with the other listed 
>> algorithms? Speaking about the webinar on particles, does 
>> "repartition" correspond to the "balanced repartitioning", and the 
>> rest (except for none) - "particle management"? And for the "Variable 
>> distribution" strategy (mentioned in the webinar), what option do I 
>> have to choose to enable the mesh adjustment according to particles 
>> (and how will it work together with the regular mesh refinement 
>> strategy?) ?
>>
>> Best regards,
>> Lev Karatun.
>>
>> 2016-11-25 17:43 GMT-05:00 Rene Gassmoeller 
>> <rene.gassmoeller at mailbox.org <mailto:rene.gassmoeller at mailbox.org>>:
>>
>>     Hi Lev,
>>
>>     Let me cc this to the mailing list so that others hear about
>>     problems and solutions as well.
>>
>>     Cool model! From what you describe it seems you are doing
>>     everything right so far. What is the value for the 'Load
>>     balancing strategy' in the Particles subsection? In order for the
>>     'Minimum tracers per cell' option to have an effect you need to
>>     at least select the option 'add tracers' (possibly combine it
>>     with others like "set Load balancing strategy = add tracers,
>>     remove tracers, repartition"). This should then generate new
>>     tracers when many others have left that cell. However keep in
>>     mind that the properties of these new particles will be
>>     interpolated from the remaining existing particles in that cell.
>>     This might or might not be a problem for your model.
>>
>>     Let me know about the parameter file option,
>>
>>     Best,
>>
>>     Rene
>>
>>     On 11/24/2016 10:36 PM, Lev Karatun wrote:
>>>     Hi Rene,
>>>
>>>     I recently added tracers to my models, they work great, thank
>>>     you for adding them! I was trying to incorporate open boundaries
>>>     (by Anne Glerum) into my models, they work just fine too.
>>>     However, when I included both of them, I ran into a problem: the
>>>     cells near the open boundary, through which constant inflow of
>>>     material is happening end up not having enough tracers (and
>>>     later not  having any tracers at all) (see attached screenshot).
>>>     I decreased the CFL number from 0.5 to 0.1, but it didn't help.
>>>     So I was wondering if you could tell me what the possible causes
>>>     are? It happens both with and without adaptive refinement, and I
>>>     have min tracers number set to 10.
>>>
>>>     Thanks in advance!
>>>
>>>     Best regards,
>>>     Lev Karatun.
>>
>>
>> _______________________________________________
>> Aspect-devel mailing list
>> Aspect-devel at geodynamics.org <mailto:Aspect-devel at geodynamics.org>
>> http://lists.geodynamics.org/cgi-bin/mailman/listinfo/aspect-devel
>
>
>
> _______________________________________________
> Aspect-devel mailing list
> Aspect-devel at geodynamics.org
> http://lists.geodynamics.org/cgi-bin/mailman/listinfo/aspect-devel

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.geodynamics.org/pipermail/aspect-devel/attachments/20161130/e643717b/attachment.html>