Monocular video depth estimation with planar constraint and semantic prior for real-time 3d reconstruction

Abstract

We address the problem of 3D scene reconstruction from monocular video. Classical methods of scene reconstruction suffer from high computational complexity, while learning-based methods have not yet provided a general solution. In this work, we propose a novel algorithm for estimating consistent dense depth maps from learning-based depth prior with planar constraint and a full framework 3D scene reconstruction that consists of three main parts: 1) time efficient sparse visual SLAM optimization algorithm, 2) dense depth estimation and 3) weighted depth fusion. Unlike previous works, our framework provides real-time and robust performance that works in generalized, challenging and texture-poor scenes without inference-time fine-tuning. The experiments on unseen on training indoor datasets show that our framework outperforms state-of-the-art methods in terms of ”in the wild” accuracy and speed.