[aspect-devel] read parameter file with Python

Wed Feb 21 19:13:10 PST 2018

Hi Marine,

Before I say anything, I would also like to add a vote for outputting
the full prm as JSON. That would be great for this!

My current approach is similar to Timo's. I usually encode the variables
in the output directory names. For example, by making a single 'output'
directory, and saving each of the model results as something like

set Output directory = output/arg1_7/arg2_4/arg3_10

The benefit is that directory trees are a fundamental data structure
that's portable and unlikely to break with future code changes.

This also lends its self well to scripting batch runs. I use a
preprocessor called pyexpander for dynamically generating parameter
files with minor differences.

Batch model runs and analysis are a super common problem. I'd be
interested in hearing how other people handle this!

Best,
-JPH

On 02/21/2018 02:19 PM, Ian Rose wrote:
> Hi Marine and others,
> 
> It's a bit messy, but I have used Python for generating .prm files when
> doing benchmarking, or similar suites of model runs. An example of this
> can be found here:
> https://github.com/geodynamics/aspect/blob/master/benchmarks/zhong_et_al_93/run_models.py
> 
> The JSON approach is probably less error-prone than my hand-rolled prm
> parsing, however.
> 
> Hope this helps!
> Ian
> 
> On Wed, Feb 21, 2018 at 2:12 PM, Marine Lasbleis
> <marine.lasbleis at elsi.jp <mailto:marine.lasbleis at elsi.jp>> wrote:
> 
>     Thanks for the info!
> 
>     A JSON file would definitely do it. 
>     I’ll have a look how to output it. 
>     I also usually name my folders with the parameters I’m modifying,
>     but experience showed that later on, I may be interested in other
>     parameters I forget to add in the folders names, so using the
>     complete output parameters (and being able to access any parameter
>     later on) would be the best. 
> 
>     The statistics file with more added parameter may also be a good
>     option, especially as I will have a couple of parameters evolving
>     with time later on.
> 
>     About the statistics file, I’m not sure how much other use Python…
>     But so: at first, I use simply readlines(), but it is very slow for
>     large files, so now I’m using pandas (pd.read_csv()), with a little
>     trick for the header. Not sure if this is the best option, so let me
>     know if you know any other one! The long header is definitely a
>     problem with the different techniques I tried. (especially as it
>     can’t be used as column names easily)
> 
>     Best, 
>     Marine
>      
>     =====
>     ELSI Research Scientist
>     Earth Life Science Institute
>     Tokyo Institute of Technology
>     Tokyo, Japan
>     +81 70 1572 5070 <tel:+81%2070-1572-5070>
>     marine.lasbleis at elsi.jp <mailto:marine.lasbleis at elsi.jp>
> 
>     https://members.elsi.jp/~marine.lasbleis/
>     <https://members.elsi.jp/~marine.lasbleis/> 
>     http://elsi.jp/en/ 
>      
> 
>>     On 22 Feb 2018, at 01:27, Timo Heister <heister at clemson.edu
>>     <mailto:heister at clemson.edu>> wrote:
>>
>>>     (My point here is: I have about 20/30 runs, I want to take the
>>>     maximum
>>>     velocity at the end of each runs — if they reached steady state
>>>     —, and plot
>>>     it as function of two parameters from the parameter file. I could
>>>     do it “by
>>>     hand”, but I am guessing there is a easier way?)
>>
>>     The way I usually do this is that I name the output directory based on
>>     the significant parameters, see benchmarks/blankenbach/ for an
>>     example. Here we automatically generate files like case1a_ref5.stat
>>     automatically.
>>
>>     You could also write a small plugin that outputs the parameters you
>>     are interested in into the statistics, the screen, or some separate
>>     file. We do this for the examples that compute errors but nobody stops
>>     you from outputting other vital information. If you are parsing the
>>     statistics file anyways, you can grab it from the respective column.
>>
>>     This is not as clean as actually parsing the parameters, but it might
>>     be good enough for your problem.
>>
>>     -- 
>>     Timo Heister
>>     http://www.math.clemson.edu/~heister/
>>     <http://www.math.clemson.edu/~heister/>
>>     _______________________________________________
>>     Aspect-devel mailing list
>>     Aspect-devel at geodynamics.org <mailto:Aspect-devel at geodynamics.org>
>>     http://lists.geodynamics.org/cgi-bin/mailman/listinfo/aspect-devel
>>     <http://lists.geodynamics.org/cgi-bin/mailman/listinfo/aspect-devel>
>>
> 
> 
>     _______________________________________________
>     Aspect-devel mailing list
>     Aspect-devel at geodynamics.org <mailto:Aspect-devel at geodynamics.org>
>     http://lists.geodynamics.org/cgi-bin/mailman/listinfo/aspect-devel
>     <http://lists.geodynamics.org/cgi-bin/mailman/listinfo/aspect-devel>
> 
> 
> 
> 
> _______________________________________________
> Aspect-devel mailing list
> Aspect-devel at geodynamics.org
> http://lists.geodynamics.org/cgi-bin/mailman/listinfo/aspect-devel
> 