Preserving hard clean samples for robust learning with noisy labels in deep neural networks

Abstract

Sample selection is an effective method for robust learning in the presence of label noises. However, existing approaches that rely on small loss values to identify clean samples can commit the error of excluding clean samples with large losses, called hard clean samples. These hard clean samples play a crucial role in shaping high-quality decision boundaries and excluding them can lead to degraded generalization performance. Toward overcoming these limitations, this paper introduces a novel sample selection strategy called KALM, which utilizes an iterative and powerful model generation and filtering strategy based on softmax probabilities and loss values obtained from deep neural network outputs. KALM preserves challenging and correct-labeled samples while effectively removing label noises, contributing to the construction of a high-performing classifier. Notably, KALM does not rely on expensive prior information such as noise rates or clean validation data, and it produces robust performance across various noise types and ratios. Experimental results on CIFAR-10, CIFAR-100, and Clothing1M datasets consistently highlight the superior performance achieved by KALM compared to existing approaches.