[aspect-devel] Writing a plugin for a new data format/source

[Timo Heister]

James,

> We working on a NSF-funded project (BALTO) to develop a ‘data broker’ and our PI uses Aspect in her work,, which serves as one of the use-cases for the project. We would like guidance on developing an Aspect plugin to read data not from a local ASCII file, but from a (remote) web service.

We are reading various kind of files and formats in different plugins
(initial conditions, gravity, material model, to name a few). Search
for "data file name" in https://urldefense.proofpoint.com/v2/url?u=https-3A__tjhei.github.io_aspect-2Dwww-2Dmanual_&d=DwIFaQ&c=Ngd-ta5yRYsqeUsEDgxhcqsYYY1Xs5ogLxWPA_2Wlc4&r=c08Btfq4m9QEScXN3ZQwLZzzWQE7S8CYq1IYuzKV_Zk&m=M72AauGyerMMa0b-5d6UAdfFfqi5dhWbxYokW5N_y94&s=nv5Ev6xyH8qE7UV-Ga5U9axYfwBn2t1xTVUY0x-pY-A&e= for
example. I am not sure what you are trying to do precisely.

> If a new plugin is not the best way forward, what would be the best way to extend Aspect so that it could read other data formats and/or data sources (i.e., from remote web services and not local files)?

A plugin would only allow for one of the many ways a user can read
data in to work. For example, you could write a
RemoteFileGravityAsciiData plugin. I would think a better option would
be to enhance read_and_distribute_file_content() in aspect/utilities.h
to accept URLs and downloading them as needed. This function is used
by various current and future plugins. You need to worry about a few
things though:
- on a cluster you might not have internet access
- in a parallel run you don't want every single of your 1000+
processes to download the files
- caching of large files?
Best,
Timo
--
Timo Heister
http://www.math.clemson.edu/~heister/