Unsupervised image restoration from unpaired images in medical and remote sensing applications

Abstract

The presence of noise or artifacts in image data causes problems when utilizing images. In medical imaging, since the image quality affects the accuracy of disease diagnosis, noise or artifacts in medical images adversely affect the clinical diagnosis. In remote sensing areas, noise or artifacts in satellite imagery interfere with remote sensing applications as they degrades the quality of the scene. Therefore, image restoration, which improves the quality of images by removing noise or artifacts from image data, is an important research topic for efficient image utilization. Recently, many deep learning-based image restoration methods has been proposed and demonstrates their successful performance. Many successful deep learning-based image restoration methods are supervised learning-based methods that require structurally matched input and target image pairs. However, since structurally matched images pairs are difficult to obtain in medical imaging and satellite images, it is hard to apply the existing supervised learning-based image restoration methods in the real situation. In this study, we propose unsupervised image restoration methods from unpaired images in medical and remote sensing applications. First, we propose an unsupervised denoising method for satellite imagery using wavelet directional CycleGAN (WavCycleGAN). To overcome the shortcomings of the existing CycleGAN, such as the loss of high frequency components that can occur in a situation where the training data is insufficient, an unsupervised wavelet directional learning method is proposed. Second, we propose a wavelet subband discriminator for efficient chest X-ray image restoration. In contrast to WavCycleGAN, which requires a wavelet transform both during training and during testing, the proposed wavelet subband discriminator-based CycleGAN is an image restoration method that only requires a wavelet transform in the training process. Furthermore, our wavelet subband discriminator can be used in the switchable CycleGAN structure, which produces visually pleasing intermediate output images without producing artifacts for the task requiring reconstruction of all frequency bands. Then we propose unsupervised image restoration methods based on federated learning that can learn without direct data exchange between the server and clients in a situation where data privacy needs to be protected, such as when dealing with medical images or satellite images. In this study, we propose a federated CycleGAN which learn image translation in an unsupervised manner while maintaining the data privacy. In contrast to the existing CycleGAN, which requires that all data belong to a server, an unsupervised learning is carried out while maintaining data privacy by only sharing the weights of common models and local gradients between the server and clients. In addition, we propose a federated CUT that improves performance while significantly reducing communication costs compared to CycleGAN. We hope this study will be helpful in the field of image restoration for medical and remote sensing applications.