Overlap matrix concentration in optimal Bayesian inference. (13th May 2020)

• Record Type: Journal Article
• Authors: Barbier, Jean
• Is Part Of: Information and inference. Volume 10:Number 2(2021)
• Page Start: 597
• Page End: 623
• Publication Date: 2020-05-13
• Subjects: Bayesian inference -- overlap concentration -- replica symmetry -- low-rank tensor factorization; Mathematical models -- Periodicals; 519.605
• DOI: 10.1093/imaiai/iaaa008
• Languages: English
• ISSNs: 2049-8764

Abstract

Abstract: We consider models of Bayesian inference of signals with vectorial components of finite dimensionality. We show that under a proper perturbation, these models are replica symmetric in the sense that the overlap matrix concentrates. The overlap matrix is the order parameter in these models and is directly related to error metrics such as minimum mean-square errors. Our proof is valid in the optimal Bayesian inference setting. This means that it relies on the assumption that the model and all its hyper-parameters are known so that the posterior distribution can be written exactly. Examples of important problems in high-dimensional inference and learning to which our results apply are low-rank tensor factorization, the committee machine neural network with a finite number of hidden neurons in the teacher–student scenario or multi-layer versions of the generalized linear model.