[aspect-devel] Writing a plugin for a new data format/source

[James Gallagher]

> On Sep 21, 2018, at 09:54, Lorraine Hwang <ljhwang at ucdavis.edu> wrote:
> 
> Hi James,
> 
> Can you define what you mean by data? Is this ASPECT input files or output files?  Can you describe a use case?

Input data.

Our collaborator has written a plugin that will read ASCII data from a local file. She would like to have that plugin read data from a web service instead.

Thanks,
James

PS. I think the answer from Timo is going to get us started


> 
> Best,
> -Lorraine
> 
> *****************************
> Lorraine Hwang, Ph.D.
> Associate Director, CIG
> 530.752.3656
> 
> 
> 
>> On Sep 21, 2018, at 8:35 AM, Timo Heister <heister at clemson.edu <mailto:heister at clemson.edu>> wrote:
>> 
>> James,
>> 
>>> We working on a NSF-funded project (BALTO) to develop a ‘data broker’ and our PI uses Aspect in her work,, which serves as one of the use-cases for the project. We would like guidance on developing an Aspect plugin to read data not from a local ASCII file, but from a (remote) web service.
>> 
>> We are reading various kind of files and formats in different plugins
>> (initial conditions, gravity, material model, to name a few). Search
>> for "data file name" in https://urldefense.proofpoint.com/v2/url?u=https-3A__tjhei.github.io_aspect-2Dwww-2Dmanual_&d=DwIFaQ&c=Ngd-ta5yRYsqeUsEDgxhcqsYYY1Xs5ogLxWPA_2Wlc4&r=c08Btfq4m9QEScXN3ZQwLZzzWQE7S8CYq1IYuzKV_Zk&m=M72AauGyerMMa0b-5d6UAdfFfqi5dhWbxYokW5N_y94&s=nv5Ev6xyH8qE7UV-Ga5U9axYfwBn2t1xTVUY0x-pY-A&e= <https://urldefense.proofpoint.com/v2/url?u=https-3A__tjhei.github.io_aspect-2Dwww-2Dmanual_&d=DwIFaQ&c=Ngd-ta5yRYsqeUsEDgxhcqsYYY1Xs5ogLxWPA_2Wlc4&r=c08Btfq4m9QEScXN3ZQwLZzzWQE7S8CYq1IYuzKV_Zk&m=M72AauGyerMMa0b-5d6UAdfFfqi5dhWbxYokW5N_y94&s=nv5Ev6xyH8qE7UV-Ga5U9axYfwBn2t1xTVUY0x-pY-A&e=> for
>> example. I am not sure what you are trying to do precisely.
>> 
>>> If a new plugin is not the best way forward, what would be the best way to extend Aspect so that it could read other data formats and/or data sources (i.e., from remote web services and not local files)?
>> 
>> A plugin would only allow for one of the many ways a user can read
>> data in to work. For example, you could write a
>> RemoteFileGravityAsciiData plugin. I would think a better option would
>> be to enhance read_and_distribute_file_content() in aspect/utilities.h
>> to accept URLs and downloading them as needed. This function is used
>> by various current and future plugins. You need to worry about a few
>> things though:
>> - on a cluster you might not have internet access
>> - in a parallel run you don't want every single of your 1000+
>> processes to download the files
>> - caching of large files?
>> Best,
>> Timo
>> --
>> Timo Heister
>> http://www.math.clemson.edu/~heister/ <http://www.math.clemson.edu/~heister/>
>> _______________________________________________
>> Aspect-devel mailing list
>> Aspect-devel at geodynamics.org
>> http://lists.geodynamics.org/cgi-bin/mailman/listinfo/aspect-devel
> 
> _______________________________________________
> Aspect-devel mailing list
> Aspect-devel at geodynamics.org
> http://lists.geodynamics.org/cgi-bin/mailman/listinfo/aspect-devel

--
James Gallagher
jgallagher at opendap.org




-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.geodynamics.org/pipermail/aspect-devel/attachments/20180924/1cfff53f/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 195 bytes
Desc: Message signed with OpenPGP
URL: <http://lists.geodynamics.org/pipermail/aspect-devel/attachments/20180924/1cfff53f/attachment.sig>