(A) study on stereo image super-resolution using a convolutional neural network with progressive parallax coherency learning

Abstract

Recently, stereo cameras have been widely packed in smart phones and autonomous vehicles thanks to their low costs and small-sized packages. Nevertheless, the acquisition of high resolution (HR) stereo images is still a challenging problem since more complex hardware and larger storage space are inevitably required. While the traditional stereo image processing tasks have mainly focused on stereo matching, stereo super-resolution (SR) has drawn relatively less attention. Some deep learning-based stereo image SR works have recently shown promising results. However, they have not fully exploited binocular parallax in stereo SR, which may lead to unrealistic visual perception. In this thesis, we present a novel and computationally efficient convolutional neural network (CNN) based deep SR network for stereo images by maintaining parallax coherency between the left and right SR images. The proposed network progressively learns parallax coherency via a novel recursive parallax coherency (RPC) block with shared parameters. The RPC block is effectively designed to extract prior parallax information for the left image SR from its right view input images and vice versa. Furthermore, we propose a parallax coherency loss to reliably train the proposed network.