Super-resolution research via blur kernel estimation and non-local features

Abstract

however, these are essential because the optimally generated kernel may be narrower than a point spread function (PSF) except when the PSF is ideal low-pass filter. ③ Previous studies also did not consider that GANs are affected by the thickness of edges as well as PSF. Thus, in this dissertation, 2) we propose a degradation and ranking comparison process designed to induce GAN models to became sensitive to image sharpness, and developed a scale-free kernel correction technique using Gaussian kernel approximation including a thickness parameter. Moreover, 3) we applied the estimated kernel information to the proposed robust network to complete the SR model and expand the valid range of its operation. Although the proposed method was trained on images degraded by a blur kernel as well as a bi-cubic kernel, it outperformed the state-of-the-art blind SR methods that were trained on a blur kernel only on a blurry evaluation set. Experimental results show that the proposed SR model exhibited an expanded valid SR range in that it preserved reconstruction performance on a bi-cubic evaluation set (i.e., , a clean image).

The difficulty of applying image super-resolution (SR) imaging methods increases substantially when image degradation information is unknown, which also strictly limits the valid range of trained SR. In particular, the blur kernel mismatching problem is a well-known phenomenon that results in artifacts or blurry output, which affects most learning-based convolutional SR networks. To expand the valid range of trained SR methods, in this dissertation, we propose an SR network that alleviates the artifacts caused by kernel mismatching. Furthermore, we also propose a more accurate blur kernel extraction method, and combine the two methods to construct an SR model that is valid for various sharpness domains. When the training dataset is blurrier than the evaluation image, an over sharpening problem occurs in conventional convolutional SR methods because they only depend on fixed relations to nearby features. Thus, in this dissertation, 1) we propose an SR network that is robust to sharpness domain changes in the training and evaluation stages with non-local modules. In the proposed SR network, each feature can obtain SR information directly from distant similar features, i.e., , from the evaluation image itself as well as from the training dataset. The non-local architecture of the proposed network transforms each feature to resemble similar features by soft clustering, and by that mechanism, avoids the over sharpening problem and alleviates image artifacts. Moreover, the prediction and application of degradation information as well as network robustness are very important for SR methods to perform well over a wide sharpness domains. Recently, generative adversarial networks (GANs) have been the most successful unsupervised kernel estimation methods. However, they still involve several problems. ① Their sharpness discrimination ability has been noted as being too weak, causing them to focus more on pattern shapes than sharpness. ② In some cases, kernel correction processes were omitted