Meta-learning for dynamic tuning of active learning on stream classification. (June 2023) - Martins, Vinicius Eiji; Cano, Alberto; Barbon Junior, Sylvio
Highlights: A new meta-learning framework to improve the stream classification performance under a reduced number of class labels. Meta-learning for Active Learning tuning as a regression problem. New meta-features to handle the trade-off between query frequency and classification performance. A discussion about when ignoring samples improves the classification performance for incremental trees. Abstract: Supervised data stream learning depends on the incoming sample's true label to update a classifier's model. In real life, obtaining the ground truth for each instance is a challenging process; it is highly costly and time consuming. Active Learning has already bridged this gap by finding a reduced set of instances to support the creation of a reliable stream classifier. However, identifying a reduced number of informative instances to support a suitable classifier update and drift adaptation is very tricky. To better adapt to concept drifts using a reduced number of samples, we propose an online tuning of the Uncertainty Sampling threshold using a meta-learning approach. Our approach exploits statistical meta-features from adaptive windows to meta-recommend a suitable threshold to address the trade-off between the number of labelling queries and high accuracy. Experiments exposed that the proposed approach provides the best trade-off between accuracy and query reduction by dynamic tuning the uncertainty threshold using lightweight meta-features.