Robust visual place recognition-based localization for navigation and preclinical magnetic particle imaging

Abstract

Visual place recognition is a crucial research topic that can be utilized in various ways concerning location recognition in computer vision and robotics. This dissertation deals with a novel visual place recognition (VPR) method robust to urban environments crowded with dynamic objects. Furthermore, its key components are utilized for visual localization of navigation and distortion correction research of medical tomography images. First, we propose a robust visual place recognition method that suppresses the effect of dynamic objects by self-supervised learning in an urban environment with many dynamic things. Visual place recognition searches for images most similar to an input query image among a geo-tagged database and outputs its place. However, its accuracy is severely degraded when images include many dynamic objects that change over time, such as vehicles and pedestrians. To this end, we propose a new self-supervised de-attention mechanism that suppresses the influence of dynamic objects in images. In addition, sharpened triplet marginal loss is proposed to improve the global descriptor discrimination of the VPR, and its effectiveness is visualized. Subsequently, the re-ranking process using geometric verification based on deep local features follows. Finally, we apply the above three new approaches to the NetVLAD backbone, widely used in image-based location recognition, and train and test it with public datasets. To overcome the lack of datasets crowded with dynamic objects such as vehicles and people, we propose a clutter augmentation method that augments the density of dynamic objects in images. Second, we employ the proposed robust visual place recognition method for the global localization of a robot in a city. Next, localization for robot navigation is also an essential research topic in robotics. Generally, expensive 3D Lidar sensors and pre-map building are used for localization instead of GNSS (Global Navigation Satellite System) with severe position errors in shaded areas and indoors. However, because astronomical costs are required to introduce public robot services at the city level with such an expensive localization method, studies on cost-effective robot localization are crucial for robot navigation. To this end, we introduce a robot camera image and free street view images into the query and database, respectively, and predict the robot’s location on a free online map. Monte Carlo localization (MCL) is utilized for global localization, and the VPR location result is introduced into its sensor model. However, suppose the domain difference between the learning and the test images is severe or similar scenes are repeated, such as a walking path or a long corridor. In that case, the reliability of the VPR is decreased. To cope with this, we define the visible region based on the predicted location and restrict only the results within the visible region to valid sensor observations. Lastly, we utilize the geometrical verification method of the proposed VPR for 2D tomography image registration of a novel magnetic particle imaging device. A magnetic particle imaging (MPI) system has recently attracted attention as a medical diagnosis device using a safe tracer without radiation exposure. Similar to research in the future vehicle interdisciplinary field, novel MPI development research requires extensive interdisciplinary convergence research in electricity, electronics, physics, materials, pharmacy, medicine, hospital clinical, robotics, and computer vision. Therefore, this study is possible only in a few advanced countries, such as the United States and Germany, where the technology has evenly reached the completion stage. We have successfully developed a novel point-of-care compact MPI for the first time in Korea. In this dissertation, we introduce it and employ the proposed geometric verification method of VPR for its image processing. In particular, we present a method for calibrating the distortion of 2D tomography images accumulated from the manufacturing stage by homography estimation based on fiducial markers and restoring it in a three-dimensional (3D) MPI image.