Image restoration of mixed degradation using transformer-based two-stage U-Nets

Abstract

In the industry, to apply image restoration technology that converts low-quality images into high-quality ones, it is necessary to restore images with complex and mixed degradation. However, existing deep learning-based methods have been focused on restoring images with a single type of degradation. In particular, Transformer-based methods for image restoration of single degradation are effective but require a new approach for images with mixed degradation. In this thesis, we propose a model using Transformer-based two-stage U-Nets for image restoration of mixed degradation that is suitable for the real world. The proposed two-stage U-Net Transformer (TUT) effectively restores images with mixed degradation by dividing a complex problem into gray-scale image restoration and color-scale image restoration, thereby outperforming existing Transformer-based image restoration models. In particular, we designed spatial-wise Transformer-based U-Net for gray-scale image restoration and channel-wise Transformer-based U-Net for color-scale image restoration. This approach effectively performed image restoration of mixed degradation. Moreover, to address shortcomings of existing Transformer-based models and maximize performance, we introduced spatial-wise and channel-wise modulators. Additionally, various loss functions were used to optimize image restoration of mixed degradation. Lastly, we proposed synthetic data pre-processing techniques capable of representing both spatial and color degradation. This approach enabled more detailed representations of real-world degradation compared to an existing synthetic degradation dataset. Experimental results showed that the proposed TUT outperformed existing Transformer-based image restoration models on various evaluation datasets.