A novel active learning framework for classification: Using weighted rank aggregation to achieve multiple query criteria. (September 2019)

• Record Type: Journal Article
• Authors: Zhao, Yu; Shi, Zhenhui; Zhang, Jingyang; Chen, Dong; Gu, Lixu
• Is Part Of: Pattern recognition. Volume 93(2019:Sep.)
• Page Start: 581
• Page End: 602
• Publication Date: 2019-09
• Subjects: Multiple query criteria active learning -- Integration criteria strategy -- Sample query criterion -- Weighted rank aggregation; Pattern perception -- Periodicals; Perception des structures -- Périodiques; Patroonherkenning; 006.4
• DOI: 10.1016/j.patcog.2019.03.029
• Languages: English
• ISSNs: 0031-3203

Abstract

Highlights: It is the first work to realize the MQCAL method for classification task by introducing weighted rank aggregation methods. A mechanism is presented that allows a self-adaptive tradeoff between any number and kind of sample query criteria. The proposed method is of high scalability and generality, and no longer needs any empirical parameters for weights. The potentially best combination of sample query criteria and rank aggregation approaches is given through experiments. Comparing with other AL methods, our proposed methods (RMQCAL) can achieve higher accuracy with less labeling costs. Abstract: Multiple query criteria active learning methods have a higher potential performance than conventional active learning methods in which only one criterion is deployed for sample selection. A central issue related to multiple query criteria active learning methods concerns the development of an integration criteria strategy that makes full use of all criteria. The conventional integration criteria strategies adopted in relevant research facilitate the desired effects, but several limitations still must be addressed. For instance, some of the strategies are not sufficiently scalable during the design process, and the number and type of criteria involved are dictated. Thus, it is challenging for the user to integrate other criteria into the original process unless modifications are made to the algorithm. Other strategies are too dependent on empirical parameters, which can be acquired only by experience or cross-validation and thus lack generality; additionally, these strategies are counter to the intention of active learning, as samples need to be labeled in the validation set before the active learning process can begin. To address these limitations, we propose a novel multiple query criteria active learning method for classification tasks that employs a third strategy via weighted rank aggregation. The proposed method serves as a heuristic means to select high-value samples of high scalability and generality and is implemented through a three-step process: (1) the transformation of the sample selection to sample ranking and scoring, (2) the computation of the self-adaptive weights of each criterion, and (3) the weighted aggregation of each sample rank list. Ultimately, the sample at the top of the aggregated ranking list is the most comprehensively valuable and must be labeled. Several experiments generating 419 wins, 226 ties and 55 losses against other state-of-the-art multiple query criteria-based methods are conducted to verify that the proposed method can achieve superior results.