Camera-radar fusion for backward projection based 3d object detection

Abstract

Accurate perception for surroundings of autonomous driving car is crucial for ensuring safety. Various sensors are used in 3D object detection to precisely measure the 3D positions and categorize the types of objects around vehicles. Recently, researches that utilize camera and radar, which have complementary characteristics and cost-effective than LiDAR, have gained great attention. In particular, studies using radar in the process of transforming images from perspective view to bird’s eye view show promising performance. However, forward projection based camera radar fusion only operates once which makes over dependent to depth distribution, and obtained bev features are sparse. Additionally, studies based on backward projection have not fundamentally addressed the false positive problems caused by the absence of depth information. In this paper, to mitigate aforementioned limitations, we utilize a radar that provides accurate information in the range direction, minimizing the depth ambiguity of the model based on backward projection. Firstly, our network utilizes both the predicted depth distribution from images and the radar occupancy to transform the context information of the perspective view image more accurately into the bird’s-eye view space. Furthermore, by extracting radar context information in the width and depth directions and performing attention operations, we ensure that even for the same ray, different depth information is considered. Quantitative results show that our proposed method enhances all 3D object detection performance metrics due to better depth discrimination. Moreover, in qualitative results, observing the direction of light beams from the perspective of the ego vehicle shows a reduction in false positive. The proposed method in this thesis is expected to contribute significantly to the advancement of camera-radar fusion research for 3D object detection.