A two-layer aggregation model with effective consistency for large-scale Gaussian process regression. (November 2021)

• Record Type: Journal Article
• Authors: Wang, Wengsheng; Zhou, Changkai
• Is Part Of: Engineering applications of artificial intelligence. Volume 106(2021)
• Publication Date: 2021-11
• Subjects: Gaussian process -- Big data -- Aggregation models -- Multi-layer structure; Engineering -- Data processing -- Periodicals; Artificial intelligence -- Periodicals; Expert systems (Computer science) -- Periodicals; Ingénierie -- Informatique -- Périodiques; Intelligence artificielle -- Périodiques; Systèmes experts (Informatique) -- Périodiques; Artificial intelligence; Engineering -- Data processing; Expert systems (Computer science); Periodicals; 620.00285
• DOI: 10.1016/j.engappai.2021.104449
• Languages: English
• ISSNs: 0952-1976

Abstract

Abstract: To scale full Gaussian process (GP) to large-scale data sets, aggregation models divide the dataset into independent subsets for factorized training, and then aggregate predictions from distributed experts. Some aggregation models have been able to produce consistent predictions which converge to the latent function when data size approaches infinity. However, these consistent predictions will become ineffective due to the limited subset size of experts. Oriented by the transition from theory to practice, the key idea is using Generalized Robust Bayesian Committee Machine (GRBCM) with corrective function to replace experts of Generalized Product of Experts (GPoE) which focuses on global information, in order to get rid of the limitation of the experts' size. Such a nested two-layer structure enables the proposed Generalized Product of Generalized Robust Bayesian Committee Machine (GPoGRBCM) to provide effective predictions on large-scale datasets and to inherit virtues of aggregations, e.g., a slightly flawed Bayesian inference framework, distributed/parallel computing. Furthermore, we perform comparisons of GPoGRBCM against the state-of-the-art aggregation models on one toy example and six real-world datasets with up to more than 3 million training points, showing dramatic performance improvement on scalability, capability, controllability, and robustness. Highlights: The local GP model reduces the time consumption and keeps the precise prediction. Theoretically, the accuracy of predictions of novel aggregation models is compared. In the application, multiple data sets and perspectives are used to judge the model.