Geometry-aware Interactive AR Authoring using a Smartphone in a wearable AR Environment

Abstract

This paper presents an augmented reality (AR) authoring system that enables an ordinary user to easily build an AR environment by manipulating and placing 3D virtual objects. The system tracks users' hand motions via an RGB-D camera which built-in an optical see-through (OST) head mounted display (HMD), and interactive features applied to virtual objects by the pre-defined hand gestures. In addition, the virtual objects and dynamic paths are placed by using a smartphone in a real-world space. To implement this system, three cored technologies are needed: (i) segmentation of regions of spaces and real-world objects (ii) hand tracking and gesture recognition for manipulating virtual objects, (iii) dynamic path placing of virtual objects using a smartphone wearing an OST HMD. We implement a prototype of the proposed system for testing its feasibility. To the end, we expect that our system enables simplify AR technology usage to ordinary users who are unfamiliar with professional AR authoring tools.