Collaborative Haptic interaction with dynamic remote environments

Abstract

The importance of haptic sensation in recognizing and understanding spatial information is well understood, and a lot of virtual reality and telepresence applications have introduced haptic technologies from early on. Although these attempts were effective in reinforcing the presence of virtual or remote targets, providing haptic feedback of inconstant dynamic environments still remains a challenging topic. In this study, we address the current challenges and demands of haptic telepresence systems by improving and integrating two independent approaches: real-time haptic rendering of unstructured spatial data and collaborative interaction with a remote partner. Contact information, which is a key element of haptic rendering, is directly estimated from streaming point cloud data without surface reconstruction for securing a real-time manner. On top of that, haptic guidance cues which restrict or promote the user’s motion can be activated by both pre-defined triggers and impromptu intervention from a helper. Since these physical and virtual constraints are defined by heterogeneous representations according to their respective characteristics, we also provide an integrated interface that simplifies proxy optimization regardless of the details underneath each representation. Based on the aforementioned ideas, we implement a proof-of-concept prototype and conduct a user study to evaluate the effectiveness and performance of the proposed approach. As a result, we confirm that our proposed approach is capable of stable haptic interaction with real-world spatial data and also improves the performance of remote exploration tasks significantly. With an adequate level of spatial sensing and haptic display technologies, we believe that this integration of physical and virtual constraints can give a meaningful clue for perceiving and exploring remote environments.