Low-complexity super-resolution using edge-orientation based direct linear mapping

Abstract

With the advent of Ultra High Definition (UHD) TVs, super-resolution (SR) methods have drawn more at-tention than before, as they can be used to generate a high-resolution (HR) image from a low-resolution (LR) image, and ultimately UHD from Full High Definition (FHD) contents. The conventional SR methods general-ly consist of two-step phases: (i) a simple interpolation step to generate initial images of target resolutions

and (ii) a quality-enhancement step that incorporate more elaborate methods aiming at enhancing the quali-ties of the interpolated images. Among them, the quality-enhancement steps often require very high compu-tational load, which then prohibits very high quality SR from being feasible for large-resolution images with real-time applications such as FHD-to-UHD up-scaling. Moreover, the simple interpolation step requires extra frame buffers to store intermediate interpolated frames for the next quality-enhancement step, which makes it difficult for SR to be implemented in low-complexity hardware. To overcome these limitations, we present a novel and fast SR method based on edge-orientation-based direction linear mapping from LR to HR images, where the interpolation step and the quality-enhancement step are combined into one unified up-scaling structure. We call our unified SR method as Super-Interpolation (SI) in this thesis. By utilizing edge-orientation-based pre-learned kernels, the proposed SI performs SR directly from the initial resolution of an input image to the target resolution of an up-scaled output image, without requiring any intermediate interpo-lated image and a bicubic interpolation algorithm. The proposed SI method involves training and up-scaling phases: (i) In the off-line training phase, training LR image patches are clustered based on their edge orientations, and each LR patch is then assigned to one of edge-orientation (EO) class indexes. For each EO class, a class-dependent linear mapping function is learned between training LR patches and their corresponding HR patches. Each pre-learned linear mapping is then able to directly up-scale LR image patches of the corresponding EO class into high-quality HR image patches during the on-line up-scaling phase

(ii) In up-scaling phase, the EO of each LR input image patch is calculated to select the linear mapping function of an EO class. Then, its corresponding HR output image patch is generated by applying the linear mapping function for the LR input image patch. To verify the effec-tiveness of our proposed SI method, we perform a large amount of experiments with common image sets as well as many FHD/UHD images. As shown in the experimental results, the proposed SI method outperforms the state-of-the-art methods in terms of PSNR/SSIM with average 0.28 dB and 0.23 point higher, with its pro-cessing speed 450 times and 32 times faster than a sparse coding based SR (SCSR) method and a multiple linear mapping (MLM) method, respectively. This shows that the proposed SI proves to be an elegant SR al-gorithm for practical implementations.