Automatic Quality Control of Crowdsourced Rainfall Data With Multiple Noises: A Machine Learning Approach. Issue 11 (2nd November 2021)

• Record Type: Journal Article
• Authors: Niu, Geng; Yang, Pan; Zheng, Yi; Cai, Ximing; Qin, Huapeng
• Is Part Of: Water resources research. Volume 57:Issue 11(2021)
• Page Start: n/a
• Page End: n/a
• Publication Date: 2021-11-02
• Subjects: crowdsourcing rainfall -- machine learning -- quality control -- transferability; Hydrology -- Periodicals; 333.91
• DOI: 10.1029/2020WR029121
• Languages: English
• ISSNs: 0043-1397

Abstract

Abstract: In geophysics, crowdsourcing is an emerging nontraditional environmental monitoring approach that supports data acquisition from individual citizens. However, because of the involvement of undertrained citizens and imprecise low‐cost sensors, crowdsourced data applications suffer from different types of noises that can deteriorate the overall monitoring accuracy. In this study, we propose a machine learning approach for automatic crowdsourced data quality control (CSQC) that detects and removes noisy data inputs in spatially and temporally discrete crowdsourced observations coming from both fixed‐point sensors (e.g., surveillance cameras) and moving sensors (e.g., moving cars/pedestrians). We design a set of features from original and interpolated rainfall data and use them to train and test the CSQC models using both supervised and unsupervised machine learning algorithms. The performances of the CSQC models under various scenarios assuming no retraining are also tested (hereafter referred to as transferability). The results based on synthetic but realistic data show that the CSQC models can significantly reduce the overall rainfall estimate errors. Under the stationary assumption, the CSQC models based on both supervised and unsupervised algorithms perform well in noisy data identification and overall rainfall estimation error reduction; however, if the model is transferred to other cities with different rainfall patterns or noise compositions (without retraining), supervised multilayer perceptrons (MLPs) show the best performance. Key Points: A machine learning‐based quality control approach is proposed for crowdsourced rainfall data with discontinuities in either time or space or both The performances of the quality control models under various scenarios with or without retraining are tested The supervised multilayer perceptron turns out to be the best performing algorithm under almost all scenarios