Introducing distributed dynamic data‐intensive (D3) science: Understanding applications and infrastructure. (2nd February 2017)

• Record Type: Journal Article
• Authors: Jha, Shantenu; Katz, Daniel S.; Luckow, Andre; Chue Hong , Neil; Rana, Omer; Simmhan, Yogesh
• Is Part Of: Concurrency and computation. Volume 29:Number 8(2017)
• Page Start: n/a
• Page End: n/a
• Publication Date: 2017-02-02
• Subjects: dynamic -- distributed -- data intensive -- scientific applications; Parallel processing (Electronic computers) -- Periodicals; Parallel computers -- Periodicals; 004.35
• DOI: 10.1002/cpe.4032
• Languages: English
• ISSNs: 1532-0626

Abstract

Summary: A common feature across many science and engineering applications is the amount and diversity of data and computation that must be integrated to yield insights. Datasets are growing larger and becoming distributed; their location, availability, and properties are often time‐dependent. Collectively, these characteristics give rise to dynamic distributed data‐intensive applications. While "static" data applications have received significant attention, the characteristics, requirements, and software systems for the analysis of large volumes of dynamic, distributed data, and data‐intensive applications have received relatively less attention. This paper surveys several representative dynamic distributed data‐intensive application scenarios, provides a common conceptual framework to understand them, and examines the infrastructure used in support of applications.