DPP-VSE: Constructing a variable selection ensemble by determinantal point processes. (15th September 2021)

• Record Type: Journal Article
• Authors: Zhang, Chunxia; Liu, Junmin; Wang, Guanwei; Li, Guanghai
• Is Part Of: Expert systems with applications. Volume 178(2021)
• Publication Date: 2021-09-15
• Subjects: Variable selection -- Determinantal point processes -- Variable selection ensemble -- Diversity -- False negative -- False positive; Expert systems (Computer science) -- Periodicals; Systèmes experts (Informatique) -- Périodiques; Electronic journals; 006.33
• DOI: 10.1016/j.eswa.2021.115025
• Languages: English
• ISSNs: 0957-4174

Abstract

Highlights: A novel method DPP-VSE is designed to construct good variable selection ensembles. Discrete DPPs are utilized to infer a probability distribution of model size. A sample from the distribution specifies the number of variables selected by each member. The simulated and real experiments are conducted to study DPP-VSE's performance. DPP-VSE outperforms its rivals in most cases and has fewer parameters to specify. Abstract: As an effective tool to analyze high-dimensional data, variable selection is playing an increasingly important role in many fields. In recent years, variable selection ensembles (VSEs) have gained much interest of researchers due to their great potential to improve selection accuracy and to stabilize the results of traditional selection methods. Inspired by one common practice of Bayesian methods, we propose in this paper a novel technique named DPP-VSE to build a VSE by utilizing determinantal point processes (DPP) to infer a distribution of model size. By sampling from this distribution, DPP-VSE has the advantage that the number of variables for a base learner to select can be automatically determined. In contrast to other VSE strategies, it has fewer parameters for users to specify. The experiments conducted with both synthetic and real data illustrate that DPP-VSE performs best under most circumstances when being evaluated with several metrics. Hence, DPP-VSE can be seen as an effective and easy to use method to solve variable selection problems.