Influence of Cross-Modal Correspondence between Auditory and Visual Stimuli on Vection Perception in Virtual Reality

Abstract

Vection is an optical illusion that enhances the sense of self-translation or self-rotation, augmenting a sense of moving or rotating without any physical movement. This sense is closely related to visually induced motion sickness (VIMS), a critical issue of current virtual reality (VR) applications. In order to find solutions for VIMS, lots of research is being conducted to explore vection. Since future VR content utilizes multiple senses to provide an immersive and comfortable experience, it is essential to explore the effect of multisensory stimuli on vection perception for detailed understanding. However, the effect of multisensory stimuli is not explored as much as uni-modal stimuli, and there could be various factors such as congruency, location, and combination methods. Therefore, in this paper, we experimented (n=32) with five intensity levels of visual stimuli and three congruence statuses of audio stimuli in VR. The result showed an interaction effect in vection detection time between congruent audio and no audio conditions. Also, it showed that the incongruent audio condition has augmented a stronger sense of vection than no audio condition. Based on the result, we propose the effect of multisensory stimuli and its rotation direction congruency on vection perception for future exploration.