Time‐critical data management in clouds: Challenges and a Dynamic Real‐Time Infrastructure Planner (DRIP) solution. (1st April 2019)

• Record Type: Journal Article
• Authors: Koulouzis, Spiros; Martin, Paul; Zhou, Huan; Hu, Yang; Wang, Junchao; Carval, Thierry; Grenier, Baptiste; Heikkinen, Jani; de Laat, Cees; Zhao, Zhiming
• Other Names: Wu Yulei guestEditor.; Yan Zheng guestEditor.; Zhao Zhiwei guestEditor.; Al‐Dubai Ahmed guestEditor.; Li Tonglin guestEditor.; Xie Bing guestEditor.; Zhang Boyu guestEditor.
• Is Part Of: Concurrency and computation. Volume 32:Number 16(2020)
• Page Start: n/a
• Page End: n/a
• Publication Date: 2019-04-01
• Subjects: Dynamic Real‐Time Infrastructure Planner -- e‐infrastructure -- research infrastructure -- time‐critical services -- virtual research environment; Parallel processing (Electronic computers) -- Periodicals; Parallel computers -- Periodicals; 004.35
• DOI: 10.1002/cpe.5269
• Languages: English
• ISSNs: 1532-0626

Abstract

Summary: The increasing volume of data being produced, curated, and made available by research infrastructures in the environmental science domain require services that are able to optimize the delivery and staging of data for researchers and other users of scientific data. Specialized data services for managing data life cycle, for creating and delivering data products, and for customized data processing and analysis all play a crucial role in how these research infrastructures serve their communities, and many of these activities are time‐critical—needing to be carried out frequently within specific time windows. We describe our experiences identifying the time‐critical requirements of environmental scientists making use of computational research support environments. We present a microservice‐based infrastructure optimization suite, the Dynamic Real‐Time Infrastructure Planner, used for constructing virtual infrastructures for research applications on demand. We provide a case study whereby our suite is used to optimize runtime service quality for a data subscription service provided by the Euro‐Argo using EGI Federated Cloud and EUDAT's B2SAFE services, and to consider how such a case study relates to other application scenarios.