Deep learning-based sequential image contrast enhancement for enhancing visual odometry

Abstract

This paper proposes a novel image processing network designed to enhance image contrast for improved performance in visual odometry. Pose estimation results in visual odometry are often susceptible to challenges arising from image illumination condition and the quantity of extracted features. To address these issues, the proposed network integrates dilated convolution layers and Convolutional Graphed Recurrent Units (GRU), leveraging spatio-temporal information within the image sequences. Significantly, the loss function is delicately designed to enhance image contrast while preserving both structural and sequential informational content during training. Leveraging SSIM loss, contrast loss, and optical flow loss, the training process aims to enhance image contrast to amplify feature points while preserving innate image details, contributing to improved visual odometry results. The network’s training is conducted by unsupervised learning with a custom image dataset collected from alleys and parking lots inside the N1 building at KAIST. The proposed network has three principal contributions: robust tracking, enhanced accuracy, and real-time performance. Validation of the network’s performance contains a comparison of pose estimation results between raw and processed images using open-source visual SLAM. Robustness is validated through comparisons between raw and processed images, using sets of images acquired in tunnel environments. Tracking performance of visual slam has improved, demonstarting the ability of proposed network. The ETH3D dataset, a well-established benchmark, is also employed to validate the network’s enhanced tracking capabilities. Furthermore, the experiments utilizing KITTI dataset highlight an enhancement in accuracy of visual odometry, proving the network’s proficiency in enhancing the contrast of image. Lastly, a UAV flight experiment is conducted to validate the real-time applicability of the proposed network. Navigation, collision avoidance, and control modules are implemented in the embedded computer, proving real-time pose estimation results. This comprehensive examination demonstrates the ability of the proposed image processing network for enhancing visual odometry.