Fusing Lidar Data and Aerial Imagery with Perspective Correction for Precise Localization in Urban Canyons

Abstract

This paper addresses a vehicle localization method that fuses aerial maps and lidar data in urban canyon environments where global positioning system (GPS) signals are inaccurate. The boundaries of buildings are extracted from the aerial map and they are matched to point cloud data provided by the lidar. However, most aerial maps contain perspective projection distortions which can be significant in urban canyons with tall buildings. In this study, a new method to correct such projection distortion is proposed and it is applied to precise localization by fusing the corrected map and lidar data. In order to achieve this, the semantic segmentation of an aerial image is performed using a convolutional neural network, and the mutual information between the lidar measurements and the building boundaries is obtained to measure their similarity. A particle filter framework is employed to localize the vehicle and match the map using the mutual information as the weight of a particle. An experimental dataset is then used to validate the feasibility of the proposed method.