(The) effect of sound on haptic perception

Abstract

In this paper, overall approaches to human multimodal perception are divided into three sections. First, an evaluation of human sensibility on perceived texture was researched as a basic research on cutaneous perception. After the essential elements of human sensitivity related to texture perception were found, various types of sandpaper were utilized in terms of how the perceived roughness is related to other elements. From the results, it was found that perceived roughness is closely related to ruggedness and prickliness. Following this, attempts were made to discover how perceived roughness and its elements are affected by amplitude and frequency of a tactile display system. The experiment showed important relationships among them. Second, effects of the sound energy level in accordance with the frequency band on haptic perception are revealed after the attempts to discover the effects of auditory cues on haptic perception. It was found that some of the sensation elements such as ruggedness and roughness are closely related to the intensity of a low frequency band of a sound(~600Hz). Third, three kinds of sound synthetic techniques (SSTs) based on sinusoidal frequency modulations with irregular primitive function algorithms were suggested. And the investigations on benefits of real time SSTs for haptic interaction were followed. Performance is evaluated in two ways; 1) unimodal analysis of the synthesized sound and 2) multimodal analysis between haptic and the sound. After the unimodal analysis, it was found that meaningful spectrum contents can be generated using suggested SSTs. And, several pieces of evidence that an auditory-haptic multimodal perception will increase the human sensitivity of the texture perception are found after the multimodal analysis between haptic and sound.