Representative null space LDA for discriminative dimensionality reduction. (March 2021)

• Record Type: Journal Article
• Authors: He, Zaixing; Wu, Mengtian; Zhao, Xinyue; Zhang, Shuyou; Tan, Jianrong
• Is Part Of: Pattern recognition. Volume 111(2021)
• Publication Date: 2021-03
• Subjects: Linear discriminant analysis -- Dimensionality reduction -- Feature selection -- Null space -- Overfitting -- Singularity problem; Pattern perception -- Periodicals; Perception des structures -- Périodiques; Patroonherkenning; 006.4
• DOI: 10.1016/j.patcog.2020.107664
• Languages: English
• ISSNs: 0031-3203

Abstract

Highlights: The research reveals the main problem of the classic null space LDA method: the intrinsic overfitting problem. A new approach, representative null space LDA (RNLDA), is proposed to solve the overfitting problem. Practical and efficient RNLDA algorithms and an automatic parameter setting algorithm are proposed. Abstract: Null space Linear Discriminant Analysis (NLDA) was proposed twenty years ago to overcome the singularity problem of LDA in practical applications. With two decades of technique development, many Discriminative Dimensionality Reduction (DDR) methods that outperform NLDA have been proposed. This paper provides new insight into NLDA and illustrates that NLDA is much more powerful after solving its inherent problem. The main problem of NLDA is the intrinsic overfitting problem. An ideal NLDA model is proposed to analyze its overfitting problem. Based on the ideal NLDA model, a more reasonable Representative NLDA (RNLDA) method is proposed to prevent overfitting. Two simple but efficient RNLDA algorithms are proposed to implement the RNLDA method with a theoretical proof. This study theoretically analyzed and indicated that applying the classical but simple hold-out pretraining method can automatically set the only parameter to achieve high performance. Extensive experiments with eight databases demonstrate the superior performance of the RNLDA method over state-of-the-art DDR methods.