Adaptation strategies based on error back-propagation for improved speech recognition

Abstract

The performance of automatic speech recognition system needs improvements under both clean and noisy environments. Adaptation scheme used error back-propagation is useful to achieve high recognition accuracy. We can consider this strategies with two aspects. One is the test phase and the other is the training phase for speech recognition. In the test phase, we present the selective attention scheme especially with audio-visual integration. And the unified training scheme is proposed in the training phase.

Speech is inherently bimodal, relying on cues from the acoustic and visual speech modalities for perception and production. The McGurk effect demonstrates that when humans are presented with conflicting acoustic and visual stimuli, the perceived sound may not exist in either modality. This effect has formed the basis for modeling the complementary nature of acoustic and visual speech by encapsulating them into the relatively new research field of audio-visual speech recognition (AVSR). Especially, for acoustically noisy speech recognition a new algorithm is presented to integrate audio and visual information for better recognition performance. Actually human begins utilize the visual cues such as lips` movements in acoustically noisy environments in order to understand speech better. In order to improve the performance of audio-visual speech recognition systems many works have been reported. These efforts may be categorized into two approaches such as research on robust feature extraction espcially for visual signals and that on audio-visual integration. Here we are interested in the latter.

The developed algorithm utilizes top-down selective attention, which is an important information processing module of human perception. Selective attention model which are brought from psychological researches is proposed to recognize isolated word in noisy environments. This model is applied to Hidden Markov Models (HMM) as classifiers. The selective atten...