Scanline resolution-invariant depth completion using a single image and sparse LiDAR point cloud

Abstract

Most existing deep learning-based depth completion methods are only suitable for high (\eg 64-scanline) resolution LiDAR measurements, and they usually fail to predict a reliable dense depth map with low resolution (4, 8, or 16-scanline) LiDAR. However, it is of great interest to reduce the number of LiDAR channels in many aspects (cost, weight of a device, power consumption). In this paper, I propose a new depth completion framework with various LiDAR scanline resolutions, which performs as well as methods built for 64-scanline resolution LiDAR inputs. For this, I define a consistency loss between the predictions from LiDAR measurements of different scanline resolutions. (i.e. 4, 8, 16, 32-scanline LiDAR measurements) Also, I design a fusion module to integrate features from different modalities. Experiments show our proposed method outperforms the current state-of-the-art depth completion methods for input LiDAR measurements of low scanline resolution and performs comparably to the methods(models) for input LiDAR measurements of 64-scanline resolution on the KITTI benchmark dataset.