Urban feature based localization and 3D mapping using multi-sensor fusion in large scale environment

Abstract

This thesis proposes the method for generating three-dimensional (3D) spatial information (3D point cloud) with high accuracy in a complex urban environment. The 3D spatial information can be utilized to estimate accurate positions of autonomous vehicles and indoor navigation robots, and it also has the advantage of intuitively conveying information about space. The first part of this thesis proposes a sensor system for reconstructing 3D spatial information using multiple sensor data and SLAM algorithms in complex urban environments. This system stably saves data from multiple sensors, and it builds the accurate 3D spatial information using saved data and graph-based Simultaneous Localization and Mapping (SLAM) algorithm. The data obtained using the system are provided through data set homepage for research of SLAM, and estimated vehicle pose and reconstructed 3D point cloud are also provided with the raw sensor data. The second part of this thesis introduces calibration, which is the most important part of sensor fusion systems. The sensor system includes Light Detection and Ranging (LiDAR) sensors to measure distance for surrounding environment, cameras to obtain RGB image data, navigation sensors (Inertial Measurement Unit (IMU), Fiber Optic Gyro (FOG), Encoder, Global Positioning System (GPS)) to estimate vehicle pose. As data of each sensor has its coordinate system, a transformation between each sensor coordinate should be defined to fuse them. The calibration is the process to estimate transformation between each coordinate system, and it is not possible to reconstruct 3D spatial information accurately without the calibration process. This thesis proposes an extrinsic calibration method between camera and LiDAR without overlap between each data, and also introduces calibration of sensor data (intensity calibration of LiDAR). The sensor system can produce locally accurate 3D maps with cumulative error correction using the SLAM algorithm, but the global accuracy of the 3D map depends mainly on GPS data. The reliability of the GPS data is usually decreased in complex urban environments because of multi-path problems by high-rise buildings. The GPS data affects the accuracy of 3D maps

therefore the highly complex environment can be a constraint to generate globally accurate 3D maps. The third and fourth part of this thesis propose the method to reduce the error of global pose estimation by utilizing road features extracted from the image. The third part introduces algorithms to generate loop closure when the vehicle returns to the same place using local sub-map, which is defined using road features from monocular camera images. The fourth part proposes a global localization algorithm, which estimates the global position of the vehicle by matching road features from the stereo image with High Definition (HD) map data. The HD map used in the thesis has high accuracy because it was extracted from true-ortho aerial images. The localization algorithm using HD maps can be exploited for autonomous vehicles to estimate global position in complex urban scenarios, and it also can be utilized for our system to achieve high global accuracy of the 3D map.