K-best feature selection and ranking via stochastic approximation. (1st March 2023)

• Record Type: Journal Article
• Authors: Akman, David V.; Malekipirbazari, Milad; Yenice, Zeren D.; Yeo, Anders; Adhikari, Niranjan; Wong, Yong Kai; Abbasi, Babak; Gumus, Alev Taskin
• Is Part Of: Expert systems with applications. Volume 213:Part A(2023)
• Publication Date: 2023-03-01
• Subjects: Explainable artificial intelligence -- Feature selection -- Feature ranking -- Stochastic approximation -- Barzilai and Borwein method; Expert systems (Computer science) -- Periodicals; Systèmes experts (Informatique) -- Périodiques; Electronic journals; 006.33
• DOI: 10.1016/j.eswa.2022.118864
• Languages: English
• ISSNs: 0957-4174

Abstract

Abstract: This study presents SPFSR, a novel stochastic approximation approach for performing simultaneous k-best feature ranking (FR) and feature selection (FS) based on Simultaneous Perturbation Stochastic Approximation (SPSA) with Barzilai and Borwein (BB) non-monotone gains. SPFSR is a wrapper-based method which may be used in conjunction with any given classifier or regressor with respect to any suitable corresponding performance metric. Numerical experiments are performed on 47 public datasets which contain both classification and regression problems, with the mean accuracy and R 2 reported from four different classifiers and four different regressors respectively. In over 80% of classification experiments and over 85% of regression experiments SPFSR provided a statistically significant improvement or equivalent performance compared to existing, well-known FR techniques. Furthermore, SPFSR obtained a better classification accuracy and R-squared on average compared to utilising the entire feature set. Highlights: We propose k-best feature selection and ranking based on stochastic approximation. This method involves simultaneous random perturbation of feature weights. We use Barzilai & Borwein non-monotone gains with gradient averaging and gain smoothing. We present experiments with four classifiers and four regressors on various datasets. Over 80% of classification and regression experiments were equivalent or better.