Speech recognition using hybrid time-delay neural network/hidden markov model

Abstract

In this dissertation, we porpose a hybrid modular time-delay neural network (TDNN)/hidden Markov model(HMM) architecture and a new parameter smoothing method to increase the recognition accuracy of a speech recognition system. In order to obtain the benchmark performances of the porposed methods, we implemented two kinds of baseline speech recognition systems. One is based on HMM and the other is based on TDNN. The database consists of 75 phonetically balanced Korean words, and 44 context-independent phonemes are chosen as the recognition unit. Speaker-independent phonemebased word recognition was done in our experiments. First, a new hybrid modular TDNN/HMM architecture for the speech recognition system is proposed. In this architecture, TDnn and HMM are effectively integrated using the fuzzy mapping concept. Modular construction of TDNN is used in our system to expand the TDNN to handle all phonemes. To deal with the temporal structure of phonemic features, we divide the input layer of our modular TDNN into two states in a time sequence. The first hidden layer of each phoneme subclass network has also a tied-connected window for each state to preserve the time-shift invarance property. This structure allows the networks to capture the temporal structure of phonemic features with two feature detectors. Our system consists of 11 phoneme subclass networks and a vowel/consonant classification network. Each phoneme subclass network is trained separately using the training data from its own set only, and connection weights between layers of subclass networks are retrained. TDNN and HMM are integrated in our system by feeding outpit vectors from the second hidden layer of each phoneme subclass network to HMM. The HMM algorithm is modified to accommodate these outputs. Therefore, our system takes advantage of both TDNN and HMM, and it can treat the temporal structure of phonemic features. Simulation results shows that dividing each layer of subclass networks into two...