Improving the Performance of Face Super-Resolution with Stochastic Attributes Modeling

Abstract

Face super-resolution (SR) is the task restores an input low-resolution (LR) face image to a high-resolution (HR) face image. When the HR image is downsampled into the LR image, the image loses the existing information. For this reason, there are a number of HR images corresponding to the LR image. Although there have been many advances in this area, previous approaches have not eliminated the uncertainty caused by stochastic attributes which only can be probabilistically inferred and therefore their result images were blurry because the network has a duty to reflect all possibilities in the single output image. In this paper, we propose a novel method to remove the uncertainty that has not been eliminated on previous works in the learning process. First of all, we separate the information of LR images into stochastic and deterministic attributes related information using a Residual Encoding Framework. For the extracted stochastic attributes, we select the most likely attributes by sampling and added it to the middle of face SR network process to eliminate the uncertainty that the network faces in the restoration process. Furthermore, we propose a Input Conditional Attribute Predictor and train it separately to be able to predict the stochastic attributes only from LR images. We evaluate the proposed method on UTKFace and CelebA datasets, and as a result confirm that our method can successfully eliminate the uncertainty in learning process and restore the stochastic attributes more clearly.