Accurate depth inference of multi-view stereo using deep neural network

Abstract

We propose an accurate depth inference of multi-view stereo using deep neural network, which aims to compute accurate depths on both fine structures and overall scene. Inspired by learning-based approach of plane sweeping algorithm, we design Deep Neural Networks (DNN) targeting both preserving sharp boundaries and inferring reasonable depths even in homogeneous texture regions. Our main contributions to achieve the goal is design of separate patch matching network depending on their tasks by arranging local and global patch matching in parallel, and also reducing the memory consumption and inference time into half while maintaining the performance. We trained our network using well-known MVS benchmarks, and validated our performance on ETH3D benchmark, which revealed that ours showed more accurate depth prediction compared to state-of-the-art learning-based multi-view stereo algorithms. As a result, our system enables high-quality depth predictions, which possibly leads to denser and more precise 3D dense reconstruction.