Meta-learning for dynamic tuning of active learning on stream classification. (June 2023)

• Record Type: Journal Article
• Authors: Martins, Vinicius Eiji; Cano, Alberto; Barbon Junior, Sylvio
• Is Part Of: Pattern recognition. Volume 138(2023)
• Publication Date: 2023-06
• Subjects: Meta-learning -- Active learning -- Data stream -- Concept drift; Pattern perception -- Periodicals; Perception des structures -- Périodiques; Patroonherkenning; 006.4
• DOI: 10.1016/j.patcog.2023.109359
• Languages: English
• ISSNs: 0031-3203

Abstract

Highlights: A new meta-learning framework to improve the stream classification performance under a reduced number of class labels. Meta-learning for Active Learning tuning as a regression problem. New meta-features to handle the trade-off between query frequency and classification performance. A discussion about when ignoring samples improves the classification performance for incremental trees. Abstract: Supervised data stream learning depends on the incoming sample's true label to update a classifier's model. In real life, obtaining the ground truth for each instance is a challenging process; it is highly costly and time consuming. Active Learning has already bridged this gap by finding a reduced set of instances to support the creation of a reliable stream classifier. However, identifying a reduced number of informative instances to support a suitable classifier update and drift adaptation is very tricky. To better adapt to concept drifts using a reduced number of samples, we propose an online tuning of the Uncertainty Sampling threshold using a meta-learning approach. Our approach exploits statistical meta-features from adaptive windows to meta-recommend a suitable threshold to address the trade-off between the number of labelling queries and high accuracy. Experiments exposed that the proposed approach provides the best trade-off between accuracy and query reduction by dynamic tuning the uncertainty threshold using lightweight meta-features.