Natural, unobtrusive, and ergonomic interaction with virtual reality user interface

Abstract

The natural and unobtrusive virtual reality user interface (NUVRUI) could gradually replace the current desktop/mobile workspace, but since VR UI is fundamentally different due to unique input/output devices, many new usability problems would arise. Therefore, comprehensive research that embraces major interactions with the VR UI needs to be conducted, especially for the mid-air gestural input. Six studies targeted for three major interactions (locomotion and distant/near selection) were conducted. The first study elicited and evaluated walking-in-place (WIP) gestures for forward, sideways, and backward walking directions from users, proposing Stepping-in-place (SIP) for forward and Step one foot + SIP/Rock/Stay for other directions. The second study proposed a WIP technique for omnidirectional locomotion based on a single RGB camera utilizing OpenPose, inducing more stable walking and better subjective evaluation at non-frontal tracking. The third study proposed the ViewfinderVR technique, a virtual viewfinder panel with the Through-the-lens metaphor, resulting in better performance and lower perceived workload than traditional raycasting for selecting distant objects. The fourth study proposed a variant technique named TouchView that the virtual panel large enough to cover the whole view follows the head to minimize panel configuration. The fifth study investigated the effect of four pseudo-haptic features: proximity feedback, protrusion, hit effect, and penetration blocking to compensate for the lack of haptic feedback of mid-air button interaction, where all features positively affected user experience. The sixth study utilized pseudo-haptic and thumb-to-finger self-haptic pinch approaches on a virtual keyboard and achieved comparable typing performance and perceived workload, but better user experience and preference without additional wearable devices. The research outcome could enhance the scientific understanding of VR UI and human interactions in VR. In addition, the established novel interaction framework and research methods to effectively interact with the NUVRUI will serve as a useful reference for developing VR applications with better usability.