Hybrid tactile feedback-based haptic interaction on a flat touch surface

Abstract

With the widespread adoption of touchscreens, and the importance of haptic interaction on the touch surface has become significant, much effort has been made to reproduce the haptic properties of virtual objects on the surface. However, many researchers have focused primarily on how to reproduce a single piece of information (either geometry or texture) with a single tactile feedback (either friction or mechanical vibration), while real objects have multiple tactile factors. Recently, several researchers have proposed the concept of simulating multiple tac-tile information at the same time, but there is a limitation that it is difficult to transfer multiple information com-pletely due to insufficient consideration of masking effect. Masking effect means that the perception of one stim-ulus is affected by other stimulus. When two stimuli are provided at the same time, the absolute threshold (absolute limen: AL) or the difference threshold (difference limen: DL) of each stimulus are changed due to the masking effect. If the masking effect is not properly taken into account when designing multiple types of tactile feedback, the information delivered to the user will significantly be distorted. Since absolute and difference threshold are the most fundamental resource in haptic feedback design, their changes induced by masking effects must be con-sidered precisely. To this end, in this dissertation, a hybrid tactile feedback concept that utilizes both electrovi-bration (EV) and mechanical vibration (MV) to simultaneously reproduce geometry and texture information on a flat touch surface is proposed. From a completely different point of view compared to other studies, the disserta-tion focuses on the bi-directional masking effect in the hybrid tactile feedback and discusses ways to overcome it. First, the experimental setup to generate both electrovibration and mechanical vibration was constructed and, through analytical and quantitative evaluation, it was confirmed that the hybrid tactile feedback could be uni-formly provided on the entire surface in response to the user’s active touch. Psychophysical experiments were designed to observe the masking effect contributes to the change in absolute and difference threshold of each of electrovibration and mechanical vibration. The masking effect of mechanical vibration on the perception of elec-trovibration, and the opposite case were investigated. In this dissertation, the masking effect on both sides (mask-ing effect on the perception of EV by MV, MV by EV) is defined as a “bi-directional masking effect”. As a result, it was found that the absolute threshold of the electrovibration was increased by more than 10 dB depending on the frequency (120 ？ 270 Hz) and intensity (10 ？ 20 dB SL) of the masking stimulus, mechanical vibration. In addition, the absolute threshold of the mechanical vibration increased in a significantly larger extent by the masking stimulus, electrovibration. On the other hand, it was confirmed that the difference threshold of each of elec-trovibration and mechanical vibration has a constant value (1.21 dB for the electrovibration, 1.98 dB for the me-chanical vibration) regardless of the frequency and intensity of the masking stimulus. Based on these observations, additional psychophysical experiments were performed to obtain the masking function of the absolute threshold for both electrovibration and mechanical vibration, and then, the generalized form of the masking function was derived. The masking function exists in the form of a conditional equation, which is defined as different functions in two sections separated by a specific intensity. Considering the masking functions, a rendering algorithm is proposed such that two types of tactile feedback can transmit two independent tactile information (e.g. geometry and texture). In addition, it was mathematically proved that the proposed algorithm can be universally applied to the common situation of combining two stimuli. Finally, through user test, it was verified that it is possible to reproduce both geometry and texture information simultaneously by applying the proposed rendering algorithm in the hybrid tactile feedback design. We contend that this approach is the first in-depth investigation of hybrid (or multiple) tactile feedback design. Therefore, the findings of this dissertation will be a key criterion for the design of a more realistic haptic interface. In addition, the proposed rendering method is not limited to the com-bination of EV and MV (geometry and texture), and can be widely applied to other combination of tactile feedback or tactile factors. Based on this dissertation, we expect that numerous studies for pursuing realistic haptic interac-tion with virtual environment will be conducted more actively.