Virtual environment design methodology for natural hand interaction

Abstract

While the human hand is the most natural tool for interacting with objects, hand interaction with virtual objects is still a challenging research field. Owing to the popularity of various hand tracking interfaces, there have been numerous applications developed to provide intuitive hand interaction with the virtual world. However, existing hand interaction techniques show limitations with respect to the naturalness of manipulation, satisfaction of visual sensation and interactivity in an immersive environment. We propose an interaction inter-face that consists of a new glove device for finger tracking and a robust and intuitive grasping interaction strategy. Although users start with great anticipation in virtual environment using such novel interfaces, they end up with dissatisfaction due to difficulties of manipulation or physical tiredness coming very short. Although the task itself is rather trivial in a real life situation, it requires much effort in the virtual environment. We address this awkwardness as ‘VR interaction-induced fatigue symptom’ and hypothesize its causes based on our observations. We argue that the source of the fatigue comes from the restricted sensory information of the VR interfaces, and that users try to accommodate the missing sensory feedback by excessive motion leading to wrong posture or bad timing. We demonstrate our hypothesis by conducting experiments of two types of virtual interaction scenarios: object transport and 3D selection. Then, by analyzing the behaviors of users’ action collected from our experiment, we derive essential factors to be considered in designing VR applications, and propose a conceptual interaction model for orchestrating virtual grasping. Based on the preliminary study related to symptoms caused in virtual environment, we claim that it is necessary to establish a mechanism about how or what factors of human hand interaction behaviors analytically affect our great virtual experience. To actually deliver...