Robust and automatic data cleansing method for short-term load forecasting of distribution feeders. (1st March 2020)

• Record Type: Journal Article
• Authors: Huyghues-Beaufond, Nathalie; Tindemans, Simon; Falugi, Paola; Sun, Mingyang; Strbac, Goran
• Is Part Of: Applied energy. Volume 261(2020)
• Publication Date: 2020-03-01
• Subjects: Distribution systems -- Outlier detection -- Binary segmentation -- Kalman smoothing -- Multi-step forecasts; Power (Mechanics) -- Periodicals; Energy conservation -- Periodicals; Energy conversion -- Periodicals; 621.042
• DOI: 10.1016/j.apenergy.2019.114405
• Languages: English
• ISSNs: 0306-2619

Abstract

Highlights: Large-scale short-term load forecasting of medium voltage distribution feeders. Automatic data cleansing of medium voltage distribution feeder time series. Comparative study of data quality enhancement strategy for distribution feeders. Nested validation procedures for model evaluation of time series data. Bias-variance trade-off in the enhancement of data quality of distribution feeders. Abstract: Distribution networks are undergoing fundamental changes at medium voltage level. To support growing planning and control decision-making, the need for large numbers of short-term load forecasts has emerged. Data-driven modelling of medium voltage feeders can be affected by (1) data quality issues, namely, large gross errors and missing observations (2) the presence of structural breaks in the data due to occasional network reconfiguration and load transfers. The present work investigates and reports on the effects of advanced data cleansing techniques on forecast accuracy. A hybrid framework to detect and remove outliers in large datasets is proposed; this automatic procedure combines the Tukey labelling rule and the binary segmentation algorithm to cleanse data more efficiently, it is fast and easy to implement. Various approaches for missing value imputation are investigated, including unconditional mean, Hot Deck via k-nearest neighbour and Kalman smoothing. A combination of the automatic detection/removal of outliers and the imputation methods mentioned above are implemented to cleanse time series of 342 medium-voltage feeders. A nested rolling-origin-validation technique is used to evaluate the feed-forward deep neural network models. The proposed data cleansing framework efficiently removes outliers from the data, and the accuracy of forecasts is improved. It is found that Hot Deck (k-NN) imputation performs best in balancing the bias-variance trade-off for short-term forecasting.