Quantifying the estimation error of principal component vectors. (11th July 2019)

• Record Type: Journal Article
• Authors: Hauser, Raphael; Lember, Jüri; Matzinger, Heinrich; Kangro, Raul
• Is Part Of: Information and inference. Volume 9:Number 3(2020)
• Page Start: 657
• Page End: 675
• Publication Date: 2019-07-11
• Subjects: sample complexity -- eigenvector approximation -- principal component analysis; Mathematical models -- Periodicals; 519.605
• DOI: 10.1093/imaiai/iaz014
• Languages: English
• ISSNs: 2049-8764

Abstract

Abstract: Principal component analysis (PCA) is an important pattern recognition and dimensionality reduction tool in many applications. Principal components are computed as eigenvectors of a maximum likelihood covariance $\widehat{\varSigma }$ that approximates a population covariance $\varSigma$, and these eigenvectors are often used to extract structural information about the variables (or attributes) of the studied population. Since PCA is based on the eigendecomposition of the proxy covariance $\widehat{\varSigma }$ rather than the ground-truth $\varSigma$, it is important to understand the approximation error in each individual eigenvector as a function of the number of available samples. The combination of recent results of Koltchinskii & Lounici (2017, Bernoulli, 23, 110–133) and Yu et al. (2015, Biometrika, 102, 315–323) yields such bounds. In the present paper we sharpen these bounds and show that eigenvectors can often be reconstructed to a required accuracy from a sample of strictly smaller size order.