Modulating scalable Gaussian processes for expressive statistical learning. (December 2021)

• Record Type: Journal Article
• Authors: Liu, Haitao; Ong, Yew-Soon; Jiang, Xiaomo; Wang, Xiaofang
• Is Part Of: Pattern recognition. Volume 120(2021)
• Publication Date: 2021-12
• Subjects: Gaussian process -- Modulation -- Scalability -- Heteroscedastic noise -- Multi-modality -- Non-stationarity; Pattern perception -- Periodicals; Perception des structures -- Périodiques; Patroonherkenning; 006.4
• DOI: 10.1016/j.patcog.2021.108121
• Languages: English
• ISSNs: 0031-3203

Abstract

Highlights: New scalable Gaussian process (GP) paradigms to introduce additional modulation variables for learning rich statistical representation, e.g., het- eroscedastic noise, multi-modality and non-stationarity, from massive data. Different variational inference strategies to arrive at analytical or tight evidence lower bounds (ELBOs) for effcient and effective model training. Comprehensive comparison against state-of-the-art GP and neural net- work counterparts to showcase the superiority of scalable modulated GPs. Abstract: For a learning task, Gaussian process (GP) is interested in learning the statistical relationship between inputs and outputs, since it offers not only the prediction mean but also the associated variability. The vanilla GP however is hard to learn complicated distribution with the property of, e.g., heteroscedastic noise, multi-modality and non-stationarity, from massive data due to the Gaussian marginal and the cubic complexity. To this end, this article studies new scalable GP paradigms including the non-stationary heteroscedastic GP, the mixture of GPs and the latent GP, which introduce additional latent variables to modulate the outputs or inputs in order to learn richer, non-Gaussian statistical representation. Particularly, we resort to different variational inference strategies to arrive at analytical or tighter evidence lower bounds (ELBOs) of the marginal likelihood for efficient and effective model training. Extensive numerical experiments against state-of-the-art GP and neural network (NN) counterparts on various tasks verify the superiority of these scalable modulated GPs, especially the scalable latent GP, for learning diverse data distributions.