Distributed data structure templates for data‐intensive remote sensing applications. (3rd December 2012)

• Record Type: Journal Article
• Authors: Ma, Yan; Wang, Lizhe; Liu, Dingsheng; Yuan, Tao; Liu, Peng; Zhang, Wanfeng
• Other Names: Vaquero Luis Miguel guestEditor.; Rodero‐Merino Luis guestEditor.; Buyya Rajkumar guestEditor.; Kolodziej Joanna guestEditor.; Khan Samee Ullah guestEditor.; Gelenbe Erol guestEditor.; Talbi El‐Ghazali guestEditor.
• Is Part Of: Concurrency and computation. Volume 25:Number 12(2013:Aug.)
• Page Start: 1784
• Page End: 1797
• Publication Date: 2012-12-03
• Subjects: parallel programming -- generic programming -- data‐intensive computing -- remote sensing image processing; Parallel processing (Electronic computers) -- Periodicals; Parallel computers -- Periodicals; 004.35
• DOI: 10.1002/cpe.2965
• Languages: English
• ISSNs: 1532-0626

Abstract

SUMMARY: The remotely sensed images continuously acquired by satellite and airborne sensors are increasing dramatically. Remote sensing applications are overwhelmed with tons of remote sensing data with complex data structures. Efficient programming in parallel systems for data‐intensive applications like massive remote sensing data processing will be a challenge. We propose a generic data‐structure oriented programming template to support massive remote sensing data processing in high‐performance clusters. These templates provide distributed abstractions for large remote sensing image data with complex data structure and allow these distributed data to be accessed as a global one. Through data serialization and one‐sided message passing primitives provided by message passing interface, the distributed remote sensing data template whose sliced data blocks are scattered among nodes could offer a simple and effective way to distribute and communicate massive remote sensing data. Efficient parallel input/output directly to and from the distributed data structure will also be offered to address the input/output bottleneck caused by massive image data. Developers can take the advantage of our templates to program efficient parallel remote sensing algorithms without dealing with data slicing and communication through low‐level message passing interface APIs. Through experiments on remote sensing applications, we confirmed that our templates were productive and efficient. Copyright © 2012 John Wiley & Sons, Ltd.