Ambiguity-aware multi-object pose optimization toward visually-assisted robot manipulation

Abstract

For a robot manipulator to effectively manipulate a target object, recognition of target objects placed in the workspace of the robot manipulator is essential. In this thesis, we construct a robot system using one robot manipulator and a RGB-D sensor attached to the end effector of the robot manipulator. Before this robot system manipulates the target object, it scans the target objects placed in the workspace. At this time, we can obtain the robot joint angles and RGB-D data. By using all these sensor data, we propose a method to prepare a basis for robot manipulation on the multi-object by detecting and estimating the 6D object poses placed in the workspace. The technical summary of this thesis is as follows. First, we propose a method for the 6D object pose estimation. To effectively manipulate an object in 3D space, estimating the orientation of the target object is more crucial than the translation of the target object. Therefore, we introduce a new concept explaining the orientation of the object, a rotation primitive. The rotation primitive concentrates and emphasizes the orientation information. Using this rotation primitive, we propose a novel 6D object pose estimation method called PrimA6D. In the experimental result, we verified that the proposed method performs better than other existing methods in the benchmark dataset. Second, we propose an ambiguity-aware 6D object pose estimation method, PrimA6D++, as a generic uncertainty prediction method. In the object pose estimation field, we usually considered two types of objects

asymmetric objects and symmetric objects. For most cases of asymmetric objects, all three rotation axes can be uniquely defined, so there is no posture ambiguity. For the symmetric objects, we cannot define the rotation axes uniquely, but can only define a dominant axis. To solve this problem, most existing methods provide prior information on the object shape in advance, which is arduous to obtain in reality. Furthermore, there are cases in which an asymmetric object appears as a symmetric object due to camera viewpoint change. Therefore, a generic method for ambiguity-aware 6D object pose estimation could be more viable in the field of robotics when such prior information is unavailable. In this thesis, we propose a new network that predicts three rotation axis primitive images, each corresponding to the orientation axis of the object. In addition, the uncertainty for each rotation axis primitive image is estimated via unsupervised learning. Based on these uncertainties, we discern object ambiguity caused by shape symmetricity and occlusion by rejecting unreliable rotation axis primitive images. For the evaluation, we present examples of awaring the object ambiguity, and we verified that the proposed method performs better than other existing methods in the benchmark dataset. Third, we formulate the problem as Object-SLAM by introducing the camera pose factor and object pose factor to refine the multi-object poses with camera poses. In the evaluation, we verified that the proposed method performs better than other existing methods in the benchmark dataset. We demonstrate real-time scene recognition capability for visually-assisted robot manipulation. Fouth, By using a robot system which composes a robot manipulator and a RGB-D sensor, we execute the robotic pick-and-place in real-world. We verified the the method presented in this study can be easily used for robotic pick-and-place.