Spatial cumulative sum algorithm with big data analytics for climate change detection. (January 2018)

• Record Type: Journal Article
• Authors: Manogaran, Gunasekaran; Lopez, Daphne
• Is Part Of: Computers & electrical engineering. Volume 65(2018)
• Page Start: 207
• Page End: 221
• Publication Date: 2018-01
• Subjects: Hadoop Distributed File System -- Big data -- Climate change -- Data analytics -- Weather sensor data; Computer engineering -- Periodicals; Electrical engineering -- Periodicals; Electrical engineering -- Data processing -- Periodicals; Ordinateurs -- Conception et construction -- Périodiques; Électrotechnique -- Périodiques; Électrotechnique -- Informatique -- Périodiques; Computer engineering; Electrical engineering; Electrical engineering -- Data processing; Periodicals; Electronic journals; 621.302854
• DOI: 10.1016/j.compeleceng.2017.04.006
• Languages: English
• ISSNs: 0045-7906

Abstract

Highlights: A scalable big data processing framework with a novel change detection algorithm is proposed to monitor the changes in the seasonal climate data. Hadoop Distributed File System is used to store the large volume of climate data and MapReduce framework is used to compute the seasonal averages of various climate parameters. Spatial cumulative sum algorithm is proposed in this chapter to monitor the changes in the seasonal climate. The changes identified in the seasonal climate data is compared with the results of various existing change detection approaches, such as Pruned Exact Linear Time (PELT), binary segmentation and segment neighborhood method. Abstract: Big data plays a vital role in the prediction of diseases that occur due to climate change. For such predictions, scalable data storage platforms and efficient change detection algorithms are required to monitor the climate change. However, traditional data storage techniques and algorithms are not applicable to process the huge amount of climate data. This paper presents a scalable data processing framework with a novel change detection algorithm. The large volume of climate data is stored on Hadoop Distributed File System (HDFS) and MapReduce algorithm is applied to calculate the seasonal average of climate parameters. Spatial autocorrelation based climate change detection algorithm is proposed in this paper to monitor the changes in the seasonal climate. The proposed climate change detection algorithm is compared with various existing approaches such as pruned exact linear time method, binary segmentation method, and segment neighborhood method. Graphical abstract: