Trace Ratio Optimization for High-Dimensional Multi-Class Discrimination. Issue 1 (2nd January 2021)

• Record Type: Journal Article
• Authors: Ahn, Jeongyoun; Chung, Hee Cheol; Jeon, Yongho
• Is Part Of: Journal of computational and graphical statistics. Volume 30:Issue 1(2021)
• Page Start: 192
• Page End: 203
• Publication Date: 2021-01-02
• Subjects: Discriminant subspace -- Groupwise soft-thresholding -- Indefinite matrix -- Linear discriminant analysis -- Ratio trace -- Sparsity; Mathematical statistics -- Data processing -- Periodicals; Mathematical statistics -- Graphic methods -- Periodicals; 519.50285
• DOI: 10.1080/10618600.2020.1807352
• Languages: English
• ISSNs: 1061-8600

Abstract

Abstract: In multi-class discrimination with high-dimensional data, identifying a lower-dimensional subspace with maximum class separation is crucial. We propose a new optimization criterion for finding such a discriminant subspace, which is the ratio of two traces: the trace of between-class scatter matrix and the trace of within-class scatter matrix. Since this problem is not well-defined for high-dimensional data, we propose to regularize the within trace and maximize the between trace. A careful investigation reveals that this optimization has an innate connection to the eigenvalue decomposition of an indefinite matrix. For the sake of better interpretability of the classifier, we also consider a sparse estimation via a group-wise soft-thresholding. Interesting relationships between the proposed method and some classical methods such as Fisher's linear discriminant analysis and its variants are discussed. Empirical examples with simulated and real datasets suggest that the proposed method works well and is often better than some existing approaches in a wide range of problems, with respect to both variable selectivity and classification accuracy. Supplementary files for this article are available online.