Variational approximation error in non-negative matrix factorization. (June 2020)

• Record Type: Journal Article
• Authors: Hayashi, Naoki
• Is Part Of: Neural networks. Volume 126(2020)
• Page Start: 65
• Page End: 75
• Publication Date: 2020-06
• Subjects: Non-negative matrix factorization (NMF) -- Real log canonical threshold (RLCT) -- Learning coefficient -- Bayesian inference -- Variational Bayesian method -- Variational inference; Neural computers -- Periodicals; Neural networks (Computer science) -- Periodicals; Neural networks (Neurobiology) -- Periodicals; Nervous System -- Periodicals; Ordinateurs neuronaux -- Périodiques; Réseaux neuronaux (Informatique) -- Périodiques; Réseaux neuronaux (Neurobiologie) -- Périodiques; Neural computers; Neural networks (Computer science); Neural networks (Neurobiology); Periodicals; 006.32
• DOI: 10.1016/j.neunet.2020.03.009
• Languages: English
• ISSNs: 0893-6080

Abstract

Abstract: Non-negative matrix factorization (NMF) is a knowledge discovery method that is used in many fields. Variational inference and Gibbs sampling methods for it are also well-known. However, the variational approximation error has not been clarified yet, because NMF is not statistically regular and the prior distribution used in variational Bayesian NMF (VBNMF) has zero or divergence points. In this paper, using algebraic geometrical methods, we theoretically analyze the difference in negative log evidence (a.k.a. free energy ) between VBNMF and Bayesian NMF, i.e., the Kullback–Leibler divergence between the variational posterior and the true posterior. We derive an upper bound for the learning coefficient (a.k.a. the real log canonical threshold ) in Bayesian NMF. By using the upper bound, we find a lower bound for the approximation error, asymptotically. The result quantitatively shows how well the VBNMF algorithm can approximate Bayesian NMF; the lower bound depends on the hyperparameters and the true non-negative rank. A numerical experiment demonstrates the theoretical result. Highlights: Variational inference approximates the posterior using mean-field approximation. We give a lower bound for the approximation error by using algebraic geometry. The lower bound depends on the true non-negative rank and the hyperparameters.