Weighted linear programming discriminant analysis for high‐dimensional binary classification. (4th July 2020)

• Record Type: Journal Article
• Authors: Wu, Yufei; Yu, Guan
• Is Part Of: Statistical analysis and data mining. Volume 13:Number 5(2020)
• Page Start: 437
• Page End: 450
• Publication Date: 2020-07-04
• Subjects: alternating direction method of multipliers -- binary classification -- feature screening -- linear discriminant analysis -- linear programming -- high dimensional data; Data mining -- Statistical methods -- Periodicals; 006.312
• DOI: 10.1002/sam.11473
• Languages: English
• ISSNs: 1932-1864

Abstract

Abstract: Linear discriminant analysis (LDA) is widely used for various binary classification problems. In contrast to the LDA that estimates the precision matrix Ω and the mean difference vector δ in the classification rule separately, the linear programming discriminant (LPD) rule estimates the product Ω δ directly through a constrained ℓ1 minimization. The LPD rule has very good classification performance on many high‐dimensional binary classification problems. However, to estimate β * = Ω δ, the LPD rule uses equal weights for all the elements of β * in the constrained ℓ1 minimization. It may not deliver the optimal estimate of β *, and therefore the estimated discriminant direction can be suboptimal. In order to obtain better estimates of β * and the discriminant direction, we can heavily penalize β j in the constrained ℓ1 minimization if we suspect the j th feature is useless for the classification while moderately penalize β j if we suspect the j th feature is useful. In this paper, based on the LPD rule and some popular feature screening methods, we propose a new weighted linear programming discriminant (WLPD) rule for the high‐dimensional binary classification problem. The screening statistics used in the marginal two‐sample t ‐test screening, Kolmogorov–Smirnov filter, and the maximum marginal likelihood screening will be used to construct appropriate weights for different elements of β * flexibly. Besides the linear programming algorithm, we develop a new alternating direction method of multipliers algorithm to solve the high‐dimensional constrained ℓ1 minimization problem efficiently. Our numerical studies show that our proposed WLPD rule can outperform LPD and serve as an effective binary classification tool.