Depth image super resolution method for time-of-flight camera based on machine learning

Abstract

Time-of-flight (ToF) measurement sensor is widely used to measure 3D depth. However, conventional ToF cameras has relatively low resolution compared to the RGB camera. To utilize such depth image of low resolution effectively in various research fields, low resolution depth image of ToF sensor should be increased. Meanwhile, ToF sensor also has problem related saturated pixels and missing pixels. A novel depth completion algorithm is proposed in this paper to improve the 3D depth image of ToF camera in terms of image resolution and abnormal pixels. Specifically, low resolution depth images and relatively high resolution RGB images are fused in machine learning architecture. The performance of this proposed depth completion algorithm is demonstrated under various experimental conditions.