A divide-and-conquer method for compression and reconstruction of smart meter data. (15th April 2023)

• Record Type: Journal Article
• Authors: Liu, Bo; Hou, Yufan; Luan, Wenpeng; Liu, Zishuai; Chen, Sheng; Yu, Yixin
• Is Part Of: Applied energy. Volume 336(2023)
• Publication Date: 2023-04-15
• Subjects: Cloud-device collaboration -- Compressive sensing -- Data compression -- Fluctuation Segment Detection -- Symbolic aggregation approximation; Power (Mechanics) -- Periodicals; Energy conservation -- Periodicals; Energy conversion -- Periodicals; 621.042
• DOI: 10.1016/j.apenergy.2023.120851
• Languages: English
• ISSNs: 0306-2619

Abstract

Highlights: A divide-and-conquer framework for smart meter data compression and reconstruction is proposed, different methods are designed for different data segments. A VAD-based fluctuation segment location method is proposed, combined with event detection to segment load data according to power patterns. Based on the CS theory, a cloud-device collaborative adaptive fluctuation segment data compression and reconstruction method is designed. A novel steady-state segment data compression and reconstruction method based on improved SAX is established, combining the DIRECT method. Comparative tests on real data from 12 households in North America and China demonstrate our work outperforms the existing solutions. Abstract: As smart grid sensors, smart meters generate abundant valuable data, laying the foundation for data-driven applications. However, the data collection brings huge communication pressure to electric utilities. In this context, considering that different types of devices have different power consumption patterns, and different types of data compression methods have their own applicable scenarios, we propose a divide-and-conquer method for compression and reconstruction of smart meter data. First, based on algorithm of voice activity detection (VAD), a load power fluctuation segment location method is proposed, which is combined with load event detection method to divide the load data into the event segments, fluctuation segments, and steady-state segments. Then, for the fluctuation segments, a cloud-device collaboration adaptive strategy based on the compressive sensing (CS) theory is designed, in which the sparse basis and measurement matrix are updated accordingly to ensure the high reconstruction accuracy in different scenarios. For the steady-state segments, a data compression method based on the improved symbolic aggregation approximation (SAX) is established, in which the dividing rectangle (DIRECT) algorithm and the irregular time partitioning method are combined to reduce the data volume for transmission without losing important information. For the event segments, the original data values are retained since the event power curves are relatively more complex and short duration. Finally, the received compressed data are reconstructed into the original power time series data in the master station on cloud to support advanced data analytics. Comparative experiments are conducted on the private and public datasets of 12 households in North America and China. The results show that our method has higher data reconstruction accuracy and compression efficiency compared to the existing methods.