On better detecting and leveraging noisy samples for learning with severe label noise. (April 2023)

• Record Type: Journal Article
• Authors: Miao, Qing; Wu, Xiaohe; Xu, Chao; Zuo, Wangmeng; Meng, Zhaopeng
• Is Part Of: Pattern recognition. Volume 136(2023)
• Publication Date: 2023-04
• Subjects: Severe label noise -- Lipschitz regularization -- Adaptive modeling and detection of label noise -- Semi-supervised learning; Pattern perception -- Periodicals; Perception des structures -- Périodiques; Patroonherkenning; 006.4
• DOI: 10.1016/j.patcog.2022.109210
• Languages: English
• ISSNs: 0031-3203

Abstract

Highlights: We suggest a Lipschitz regularization based learning method with noisy labels, which prevents the model from over-fitting to noisy labels especially with severe noise. We present a novel method for adaptive modeling and detection of label noise to dynamically identify the clean samples. We combine the strongly-augmented strategy with semi-supervised learning and improve the robustness and stability of the model effectively. Abstract: Despite the success of learning with noisy labels, existing approaches show limited performance when the noise level is extremely high, since deep neural networks (DNNs) are easily overfit to the training set with corrupted labels. In this paper, we introduce Lipschitz regularization to prevent the DNNs from over-fitting to noisy labels quickly. Meanwhile, to better detect and leverage the noisy samples, we propose a Lipschitz regularization based framework with a combination of adaptive modeling and detection module and improved semi-supervised learning. We propose to adaptively model the real distribution of the training set, and the implicit individual clean/noisy distribution, instead of parametric models. With Bayes' rule, we then compute the posterior probability of a sample being clean, which provides a dynamic threshold for the detection of noisy labels. To reduce training instability caused by less labeled data with severe label noise, we improve the semi-supervised learning by combining the advantages of Mixup and FixMatch. It can not only increase the diversity of unlabeled samples, but also improve the generalization capability of the DNNs to avoid over-fitting. Experiments on several benchmarks demonstrate that our approach achieves comparable results with the state-of-the-art methods in the less-noisy environment, and obtains a substantial improvement ( ∼ 8% and ∼ 6% in accuracy on CIFAR-10 and CIFAR-100 respectively) with severe noise.