[aspect-devel] Writing a plugin for a new data format/source

[James Gallagher]

> On Sep 21, 2018, at 11:36, Wolfgang Bangerth <bangerth at colostate.edu> wrote:
> 
> 
> James,
> 
>> We working on a NSF-funded project (BALTO) to develop a ‘data broker’ and our PI uses Aspect in her work,, which serves as one of the use-cases for the project. We would like guidance on developing an Aspect plugin to read data not from a local ASCII file, but from a (remote) web service.
>> 
>> If a new plugin is not the best way forward, what would be the best way to extend Aspect so that it could read other data formats and/or data sources (i.e., from remote web services and not local files)?
> 
> Timo has already mentioned a number of issues, in particular the fact
> that on most clusters, individual nodes don't have access to the wider
> internet. This makes working with data sources located elsewhere awkward.

Yes. I’ll talk with our collaborator about this.

> 
> Off the top of my head, what would be interesting would be, for example,
> to deal with large data sets that may be stored elsewhere. Data sets
> such as CRUST 1.0 might be interesting, or the STS elevation data for
> the earth surface. In all of these cases, however, if the data is too
> large to store locally, then it is also too large to download in its
> entirety from a WebDAV (or OpenDAP) server. A more interesting method
> would then of course be if the data could be obtained by sending a query
> to some server that then processes or subsets the data stored there. I
> don't know whether that's part of you're working on.

Our server, which will be the basic of the BALTO broker, does support subsetting data.

> 
> An alternative -- maybe more interesting -- would be to not deal with
> input data but with output data. A common use case is if a simulation
> produces visualization data that is then visualized on some other
> system. Transferring this data after the run is often annoying and
> awkward. A nicer way would be if it is automatically transferred as part
> of the simulation run -- though one of course gets into the same issue
> with internet access from big clusters.

That does sound interesting but I’m not sure it is within the scope of our project.

Thanks,
James
> 
> 
> So my recommendation would be to first find a good use case and then
> think about how to implement it, rather than the other way around.
> 
> Best
>  Wolfgang
> 
> 
> --
> ------------------------------------------------------------------------
> Wolfgang Bangerth          email:                 bangerth at colostate.edu
>                            www: http://www.math.colostate.edu/~bangerth/
> _______________________________________________
> Aspect-devel mailing list
> Aspect-devel at geodynamics.org
> http://lists.geodynamics.org/cgi-bin/mailman/listinfo/aspect-devel

--
James Gallagher
jgallagher at opendap.org




-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.geodynamics.org/pipermail/aspect-devel/attachments/20180924/f396d112/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 195 bytes
Desc: Message signed with OpenPGP
URL: <http://lists.geodynamics.org/pipermail/aspect-devel/attachments/20180924/f396d112/attachment-0001.sig>