Change-point detection for piecewise deterministic Markov processes. (November 2018)

• Record Type: Journal Article
• Authors: Cleynen, Alice; de Saporta, Benoîte
• Is Part Of: Automatica. Volume 97(2018)
• Page Start: 234
• Page End: 247
• Publication Date: 2018-11
• Subjects: Convergence of numerical methods -- Decision making -- Discretization -- Dynamic programming -- Filtering -- Markov models -- Markov decision processes -- Numerical methods -- Optimal control -- Quantization; Automatic control -- Periodicals; Automation -- Periodicals; 629.805
• DOI: 10.1016/j.automatica.2018.08.011
• Languages: English
• ISSNs: 0005-1098

Abstract

Abstract: We consider a change-point detection problem for a simple class of Piecewise Deterministic Markov Processes (PDMPs). A continuous-time PDMP is observed in discrete time and through noise, and the aim is to propose a numerical method to accurately detect both the date of the change of dynamics and the new regime after the change. To do so, we state the problem as an optimal stopping problem for a partially observed discrete-time Markov decision process taking values in a continuous state space and provide a discretization of the state space based on quantization to approximate the value function and build a tractable stopping policy. We provide error bounds for the approximation of the value function and numerical simulations to assess the performance of our candidate policy.