Single view 3D reconstruction using deep learning based on human visual depth perception

Abstract

3D information plays significant role in various computer vision and robotics tasks. 3D reconstruction can be done in multiple ways, such as stereo matching, structure from motion or sensor fusion. However, when it comes to single view 3D reconstruction, many of the existing deep learning-based methods rely heavily on collecting dataset. Yet, in human visual depth perception system, there are several cues that help human to perceive depth, such as perspective, relative size, occlusion, etc. This thesis aims to suggest robust and data-efficient model-based single view 3D reconstruction pipeline, that successfully models various human monocular depth perception cues into deep learning-based methodology. We first try this idea on several specific tasks to show the effectiveness of the visual depth cues. Each of the cues and following tasks are, 1) relative size and absolute size cues for dense LiDAR simulation with 2D laser observation, 2) aerial perspective and texture gradient cues for CNN-based simultaneous dehazing and depth estimation, 3) occlusion cue for depth completion using Plane-Residual representation and 4) perspective and elevation cues for single view scene scale estimation using scale field. Finally, by combining some of the cues and corresponding inspired methods, this thesis proposes a human-inspired single view depth estimation approach that is more robust and data-efficient than previous data-driven approach.