Depth estimation using sequential information for online 3D reconstruction

Abstract

We address the depth estimation problem in order to achieve a high-quality 3D reconstruction of the scene. In this thesis, we focus on two major problems of depth estimation including depth completion from sparse measurements and depth prediction in multi-view stereo. We propose novel deep learning architectures for these problems. Our proposals are mainly based on the sequential property of the input such as video or a sequence of images. They utilize the estimated depths of neighboring views to compensate estimation of the reference view. As a result, the proposed methods can produce temporal consistent depth. We also propose an uncertainty estimation for the predicted depth to efficiently remove the outliers when performing 3D reconstruction. Extensive experiments show that our frameworks achieve significant improvement compared with the state-of-the-art baselines in both offline and online fashion.