Deep learning based approaches for 3D object detection and instance segmentation on point clouds

Abstract

This dissertation considers a 3D object detection and instance segmentation method based on point clouds. 3D object detection is a task that predicts bounding boxes including objects in point cloud datasets. 3D instance segmentation is a task that segment each pixel or point into a object that the point belongs to. Despite several previous works, these tasks have some limitations : (1) While PointNet-based permutation-invariant network extracts features of point cloud for object detection, independent network are used to extract the feature of grouped points without interaction between grouped points. (2) Bounding box detection is sensitive to rotation error and translation error for flat objects. (3) Incorrect semantic prediction propogates to final instance prediction in one-hot semantic based grouping instance segmentation. To mitigate these problems, we propose (1) Group-Point Attention(GPAN) to enhance feature representation of 3D point clouds, adapting cross-attention between base point and grouped points to encourage interaction. (2) SphereSeg by predicting spheres replacing bounding boxes that are invariant to object orientation and more robust to localization error. (3) SoftGroup to improve the quality of instance proposals, further refine proposals for accurate 3D instance segmentation. Experiments on various 3D point cloud datasets show the efficacy of the proposed methods.