Millimeter-wave radar and RGB-D camera sensor fusion for pedestrian detection and tracking

Abstract

One of the key aspects of modern-day robotics research is the development of the perceptual capabilities of agents, robots need to understand their surroundings in order to best reason or infer about a given situation. Chief among the areas of perception is that of detection and tracking of people. For purposes of Human Robot Interaction (HRI), several platforms have been developed that include tracking of pedestrians, mapping of environment and tracking of cyclists and incoming cars in the case of self-driving cars. In this work we employ a heterogeneous sensor fusion approach to moving person tracking by integrating ideas from the vast literature on military target tracking technology. We perform experiments in a variety of settings (single target and multi target, varying illumination, varying distances and fields of view, through a wooden wall as well as dense and light clutter). The results show that using multiple complementary sensors instead of single ones greatly enhances the performance of the tracking system under the stated variations.