Released
SoftwareDASF: Progress API: A progress reporting structure for the data analytics software framework
Cite as:
Eggert, Daniel; Dransch, Doris (2021): DASF: Progress API: A progress reporting structure for the data analytics software framework. GFZ Data Services. https://doi.org/10.5880/GFZ.1.4.2021.007Status
I N R E V I E W : Eggert, Daniel; Dransch, Doris (2021): DASF: Progress API: A progress reporting structure for the data analytics software framework. GFZ Data Services. https://doi.org/10.5880/GFZ.1.4.2021.007There is a new version of this Software:
Abstract
DASF: Progress API is part of the Data Analytics Software Framework (DASF, https://git.geomar.de/digital-earth/dasf), developed at the GFZ German Research Centre for Geosciences (https://www.gfz-potsdam.de). It is funded by the Initiative and Networking Fund of the Helmholtz Association through the Digital Earth project (https://www.digitalearth-hgf.de/). DASF: Progress API provides a light-weight tree-based structure to be sent via the DASF RPC messaging protocol. It's generic design supports deterministic as well as non-deterministic progress reports. While DASF: Messaging Python provides the necessary implementation to distribute the progress reports from the reporting backend modules, DASF: Web includes ready to use components to visualize the reported progress.
Additional Information
The data analytics software framework DASF, developed at the GFZ German Research Centre for Geosciences (https://www.gfz-potsdam.de) and funded by the Initiative and Networking Fund of the Helmholtz Association through the Digital Earth project (https://www.digitalearth-hgf.de/), provides a framework for scientists to conduct data analysis in distributed environments.DASF supports scientists to conduct data analysis in distributed IT infrastructures by sharing data analysis tools and data. For this purpose, DASF defines a remote procedure call (RPC) messaging protocol that uses a central message broker instance. Scientists can augment their tools and data with this protocol to share them with others. DASF supports many programming languages and platforms since the implementation of the protocol uses WebSockets. It provides two ready-to-use language bindings for the messaging protocol, one for Python and one for the Typescript programming language. In order to share a python method or class, users add an annotation in front of it. In addition, users need to specify the connection parameters of the message broker. The central message broker approach allows the method and the client calling the method to actively establish a connection, which enables using methods deployed behind firewalls. DASF uses Apache Pulsar (https://pulsar.apache.org/) as its underlying message broker. The Typescript bindings are primarily used in conjunction with web frontend components, which are also included in the DASF-Web library. They are designed to attach directly to the data returned by the exposed RPC methods. This supports the development of highly exploratory data analysis tools. DASF also provides a progress reporting API that enables users to monitor long-running remote procedure calls.
Technical Information
Copyright 2021 Helmholtz Centre Potsdam GFZ German Research Centre for Geosciences, Potsdam, Germany / DASF Data Analytics Software FrameworkLicensed under the Apache License, Version 2.0 (the "License"); you may not use these files except in compliance with the License. You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.
Authors
- Eggert, Daniel
;GFZ German Research Centre for Geosciences, Potsdam, Germany - Dransch, Doris;GFZ German Research Centre for Geosciences, Potsdam, Germany
Contact
- Eggert, Daniel; GFZ German Research Centre for Geosciences; ➦
Files
