Automated data scanning for dense networks of low-cost air quality instruments: Detection and differentiation of instrumental error and local to regional scale environmental abnormalities. (March 2018)

• Record Type: Journal Article
• Authors: Alavi-Shoshtari, Maryam; Salmond, Jennifer Ann; Giurcăneanu, Ciprian Doru; Miskell, Georgia; Weissert, Lena; Williams, David Edward
• Is Part Of: Environmental modelling & software. Volume 101(2018)
• Page Start: 34
• Page End: 50
• Publication Date: 2018-03
• Subjects: Change-point detection -- Linear multi-regression -- Sensor networks -- Data reliability; Environmental monitoring -- Computer programs -- Periodicals; Ecology -- Computer simulation -- Periodicals; Digital computer simulation -- Periodicals; Computer software -- Periodicals; Environmental Monitoring -- Periodicals; Computer Simulation -- Periodicals; Environnement -- Surveillance -- Logiciels -- Périodiques; Écologie -- Simulation, Méthodes de -- Périodiques; Simulation par ordinateur -- Périodiques; Logiciels -- Périodiques; Computer software; Digital computer simulation; Ecology -- Computer simulation; Environmental monitoring -- Computer programs; Periodicals; Electronic journals; 363.70015118
• DOI: 10.1016/j.envsoft.2017.12.002
• Languages: English
• ISSNs: 1364-8152

Abstract

Abstract: Recent improvements in low-cost air quality instrumentation make deployment of dense networks of sensors possible. However, the shear volume of data from these networks means that traditional methods for data quality control and data analysis are no longer viable. We propose a real-time data scanning routine that detects local and regional variability within the data sets. This can be used to differentiate errors resulting from instrument malfunction or calibration drifts from natural (environmentally driven) regional changes in ambient concentrations. Our case study considered hourly-averaged ozone data from Texas and from two networks in Vancouver. We used 7 and 28 days of data for the algorithm initialisation with simulated and real instrumental changes. The algorithm output can be used as part of a limited resource maintenance schedule for sensor networks, and to improve understanding of air quality processes and their relation to environmental and public health data. Highlights: The algorithm output indicates instrumental errors, local and regional variations. It makes minimal assumptions about data availability and reliability. It is not restricted to certain data types or network characteristics. Linear multi-regression models and change-point detection techniques are used. It is successfully tested on hourly-averaged ozone data of two different networks.