Mitigating dataset bias for robust deep learning: from clean dataset to the practical noisy dataset

Abstract

The demand for machine learning models in various industries is growing rapidly due to recent advancements in artificial intelligence research. However, the effectiveness of deep neural network models heavily depends on the quality of the training data they receive. If the target labels are strongly correlated to target features, it is resulted in the model with prejudice. These datasets are known as biased datasets, and this issue is referred to as the dataset bias problem. Therefore, having a well-structured training dataset is crucial for training robust deep neural networks. However, acquiring such datasets in real-world scenarios is challenging and expensive because it often requires significant human effort, such as additional labeling. To address this issue and minimize human involvement, researchers have been studying training methods that can prevent bias in models trained on biased datasets. This thesis involves an extensive investigation that examines biased training datasets and even explores scenarios involving different types of noise. Firstly, we propose an algorithm called PGD, which is a per-sample gradient-based resampling method. It constructs balanced-mini-batches to mitigate bias in biased datasets with accurate labels. Secondly, we describe DENEB, a training method based on entropy that aims to reduce dataset bias with noisy labels. DENEB leverages the entropy of the softmax function to alleviate the impact of noisy labels. Lastly, we introduce ORBIS, a method that boosts the debiasing of existing debiasing techniques by utilizing unlabeled or potentially corrupted labeled open datasets.