Stochastic dynamic programming solution to transmission scheduling: Multi sensor-multi process with wireless noisy channel. (March 2023)

• Record Type: Journal Article
• Authors: Forootani, Ali; Forootani, Sara; Iervolino, Raffaele; Zarch, Majid Ghaniee
• Is Part Of: Computers & electrical engineering. Volume 106(2023)
• Publication Date: 2023-03
• Subjects: Kalman filter -- Sensor scheduling -- Markov Decision Process -- Approximate dynamic programming -- Random Forest -- Neural Network; Computer engineering -- Periodicals; Electrical engineering -- Periodicals; Electrical engineering -- Data processing -- Periodicals; Ordinateurs -- Conception et construction -- Périodiques; Électrotechnique -- Périodiques; Électrotechnique -- Informatique -- Périodiques; Computer engineering; Electrical engineering; Electrical engineering -- Data processing; Periodicals; Electronic journals; 621.302854
• DOI: 10.1016/j.compeleceng.2022.108573
• Languages: English
• ISSNs: 0045-7906

Abstract

Abstract: We investigate sensor scheduling for remote estimation when multiple smart sensors monitor multiple stochastic dynamical systems. The sensors transmit their measurements to a remote estimator through a noisy wireless communication channel. Such a remote estimator can receive multiple packets simultaneously sent by local sensors. Sensors transmit their measurements if their Signal Interference and Noise Ratio (SINR) is above a threshold. We compute the optimal policy for sensor scheduling by minimizing expected error covariance subject to total signal transmissions from all sensors. We model this problem as Markov Decision Process (MDP) with discounted cost per stage in the finite time horizon framework, then we employ stochastic Dynamic Programming as the optimization method. A novel algorithm based on sampling and machine learning techniques is proposed as the approximation. At each phase of the DP algorithm, samples are collected using a uniform probability distribution. The data is used to feed Neural Network (NN) and Random Forest (RF) models for cost function and policy approximation. The results of the proposed framework are supported by simulation examples comparing RF and NN as Approximate DP (ADP). Note that this idea builds a bridge among the recent advances in the area of data science, Machine Learning, and the ADP. Highlights: Modeling Multi Sensor-Multi Process (MSMP) sensor scheduling MDP. New Approximate Dynamic Programming algorithm based on Machine Learning techniques. Developing a Python based package to investigate the proposed NLS-ADP method. Considering simulation scenarios to verify outcomes of the proposed NLS-ADP method.