Single image super-resolution for realistic faces and textures

Abstract

In deep learning based single image super-resolution (SISR), perceptually realistic super-resolution (SR) approaches such as the super-resolution generative adversarial network (SRGAN) tend to produce rather unrealistic human faces while producing realistic natural textures in SR images. Since the human visual system is found to be very sensitive to human faces, their image quality may be critical in perceiving SR images containing faces. In this work, we aim to build a novel training framework for a SISR network to achieve plausible looking faces with high perceptual quality as well as natural textures in an arbitrary image. To address this problem, we adopt two schemes in the training stage: a face training scheme to effectively handle faces of various sizes in images and a new training strategy in the up-scaled image domain. These allow the network to provide more realistic face SR performance regardless of the size of faces in the image while providing realistic SR performance in the texture area. Via quantitative and qualitative performance comparisons with a state-of-the-art method pursuing perceptual realism of textures, it is verified that the proposed methods noticeably improve the perceptual quality in the face region in images, while maintaining similar performance in the remaining regions.