Point cloud descriptor robust to point density variation by using domain-adversarial siamese CNN and its applications to robots

Abstract

Point cloud descriptors, which represent the characteristics of a point cloud data numerically, are the fundamental elements in many 3D vision applications. Although traditional point cloud descriptors that are handcrafted or learned with machine learning techniques have been used in many vision applications, their performances are still affected by environmental conditions of targeting applications and the specification of a sensor used, which is known as generalization problem. Particularly for 3D applications, it is still challenging to handle high noise and variance in 3D data. Recent works have applied deep learning techniques for developing point cloud descriptors to resolve this difficulty, but they lack consideration of practical problems in using point cloud descriptors in robot applications. Especially when an application has a high variance of viewpoint and sensing distance, the point density of the same surface of an object is varied. Then, the performance of the system may be deteriorated because a descriptor cannot return consistent description. To secure applicability of a point cloud descriptor to robot applications, this study aims for developing a novel point cloud descriptor that satisfies following conditions. 1. It should be able to handle noise characteristics that appear in the real world. 2. It is assumed that no object models are pre-given. 3. It should be applicable to point cloud directly without a pre-process like mesh construction. 4. It should show consistent performance on the variation of point density. 5. It should show better performance than other descriptors proposed by researchers. 6. Its feasibility to robot applications should be confirmed in an aspect of computational cost. As the first step for the above objectives, public 3D databases are surveyed and their applicability for this research is checked. Because there is no public database that includes images of various point density, point cloud images are collected and detailed process of collecting is studied. In the second step, algorithms that fill up missing topology information and convert it into a structured data representation that can be used as input to a neural network are proposed. The comparison between the algorithms is proceeded. Thirdly, the effect of domain-adversarial feature learning is examined to secure the consistency of the description on the variation of point density. In the fourth step, the performance of the proposed descriptor is compared to the ones proposed by other researchers. These are Point Feature Histogram(PFH), Fast Point Feature Histogram(FPFH), Spin Image, Signatures of Histograms for OrienTations(SHOT), Point Pair Feature(PPF), Local Voxelized structure(LoVS), Local Feature Statistics Histgoram(LFSH), 3D Histgoram of Point Distribution(3DHoPD), 3dMatch, PointNet, and PointNet++. The result proved that the proposed descriptor outperforms than the others for five test datasets

images with low density, images with medium density, images with high density, images acquired by other sensor and images acquired in another environment. In addition, the robust performance to the variation of interest volume size and noise level is confirmed. Lastly, robot applications with the proposed descriptor are implemented to show its feasibility.