A generalized least-squares approach regularized with graph embedding for dimensionality reduction. (February 2020)

• Record Type: Journal Article
• Authors: Shen, Xiang-Jun; Liu, Si-Xing; Bao, Bing-Kun; Pan, Chun-Hong; Zha, Zheng-Jun; Fan, Jianping
• Is Part Of: Pattern recognition. Volume 98(2020:Feb.)
• Publication Date: 2020-02
• Subjects: Dimensionality reduction -- Graph embedding -- Subspace learning -- Least-squares; Pattern perception -- Periodicals; Perception des structures -- Périodiques; Patroonherkenning; 006.4
• DOI: 10.1016/j.patcog.2019.107023
• Languages: English
• ISSNs: 0031-3203

Abstract

Highlights: It seeks a generalized orthogonality constraint based on the PCA idea of minimizing least-squares reconstruction errors, which restrains orthogonality on data while inducing a penalty factor to scale the influence of each data point. The proposed generalized least-squares approach shares both advantages of Dimensionality Reduction (DR) and least-squares reconstruction error. Our proposed method can achieve a balance between keeping global structure by data reconstruction technique and local structure by graph embedding technique. Our proposed framework can easily be extended to supervised and semi-supervised scenarios on the existing DR frameworks. Abstract: In current graph embedding methods, low dimensional projections are obtained by preserving either global geometrical structure of data or local geometrical structure of data. In this paper, the PCA (Principal Component Analysis) idea of minimizing least-squares reconstruction errors is regularized with graph embedding, to unify various local manifold embedding methods within a generalized framework to keep global and local low dimensional subspace. Different from the well-known PCA method, our proposed generalized least-squares approach considers data distributions together with an instance penalty in each data point. In this way, PCA is viewed as a special instance of our proposed generalized least squares framework for preserving global projections. Applying a regulation of graph embedding, we can obtain projection that preserves both intrinsic geometrical structure and global structure of data. From the experimental results on a variety of face and handwritten digit recognition, our proposed method has advantage of superior performances in keeping lower dimensional subspaces and higher classification results than state-of-the-art graph embedding methods.