Deep recurrent Gaussian processes for outlier-robust system identification. (December 2017)

• Record Type: Journal Article
• Authors: Mattos, César Lincoln C.; Dai, Zhenwen; Damianou, Andreas; Barreto, Guilherme A.; Lawrence, Neil D.
• Is Part Of: Journal of process control. Volume 60(2017)
• Page Start: 82
• Page End: 94
• Publication Date: 2017-12
• Subjects: Nonlinear system identification -- Dynamical modeling -- Robust modeling -- Gaussian Process -- Outliers -- Variational Bayes; Process control -- Periodicals; Fabrication -- Contrôle -- Périodiques; Process control; Periodicals; Electronic journals; 660.281
• DOI: 10.1016/j.jprocont.2017.06.010
• Languages: English
• ISSNs: 0959-1524

Abstract

Highlights: Proposal of the RGP- t model, a deep recurrent architecture for robust Gaussian processes modeling. Development of a variational Bayes framework for approximate inference with the RGP- t model. Evaluation of the RGP- t on benchmarking datasets for different levels of outlier contamination. Abstract: Gaussian Processes (GP) comprise a powerful kernel-based machine learning paradigm which has recently attracted the attention of the nonlinear system identification community, specially due to its inherent Bayesian-style treatment of the uncertainty. However, since standard GP models assume a Gaussian distribution for the observation noise, i.e., a Gaussian likelihood, the learning and predictive capabilities of such models can be severely degraded when outliers are present in the data. In this paper, motivated by our previous work on GP learning with data containing outliers and recent advances in hierarchical (deep GPs) and recurrent GP (RGP) approaches, we introduce an outlier-robust recurrent GP model, the RGP- t . Our approach explicitly models the observation layer, which includes a heavy-tailed Student- t likelihood, and allows for a hierarchy of multiple transition layers to learn the system dynamics directly from estimation data contaminated by outliers. In addition, we modify the original variational framework of standard RGP in order to perform inference with the new RGP- t model. The proposed approach is comprehensively evaluated using six artificial benchmarks, within several outlier contamination levels, and two datasets related to process industry systems (pH neutralization and heat exchanger), whose estimation data undergo large contamination rates. The simulation results obtained by the RGP- t model indicates an impressive resilience to outliers and a superior capability to learn nonlinear dynamics directly from highly outlier-contaminated data in comparison to existing GP models.