Online nonparametric monitoring of heterogeneous data streams with partial observations based on Thompson sampling. (3rd April 2023)

• Record Type: Journal Article
• Authors: Ye, Honghan; Xian, Xiaochen; Cheng, Jing-Ru C.; Hable, Brock; Shannon, Robert W.; Elyaderani, Mojtaba Kadkhodaie; Liu, Kaibo
• Is Part Of: IISE transactions. Volume 55:Number 4(2023)
• Page Start: 392
• Page End: 404
• Publication Date: 2023-04-03
• Subjects: Heterogeneous data streams -- mean shift detection -- partial observations -- antirank-based CUSUM procedure -- Thompson sampling; Industrial engineering -- Periodicals; Systems engineering -- Periodicals; Industrial engineering; Systems engineering; Electronic journals; Periodicals; 670.285
• DOI: 10.1080/24725854.2022.2039423
• Languages: English
• ISSNs: 2472-5854

Abstract

Abstract: With the rapid advancement of sensor technology driven by Internet-of-Things-enabled applications, tremendous amounts of measurements of heterogeneous data streams are frequently acquired for online process monitoring. Such massive data, involving a large number of data streams with high sampling frequency, incur high costs on data collection, transmission, and analysis in practice. As a result, the resource constraint often restricts the data observability to only a subset of data streams at each data acquisition time, posing significant challenges in many online monitoring applications. Unfortunately, existing methods do not provide a general framework for monitoring heterogeneous data streams with partial observations. In this article, we propose a nonparametric monitoring and sampling algorithm to quickly detect abnormalities occurring to heterogeneous data streams. In particular, an approximation framework is incorporated with an antirank-based CUSUM procedure to collectively estimate the underlying status of all data streams based on partially observed data. Furthermore, an intelligent sampling strategy based on Thompson sampling is proposed to dynamically observe the informative data streams and balance between exploration and exploitation to facilitate quick anomaly detection. Theoretical justification of the proposed algorithm is also investigated. Both simulations and case studies are conducted to demonstrate the superiority of the proposed method.