(A) study on the continuous hand gesture recognition system for the Korean sign language

Abstract

Sign language is a representative example of hand gestures with linguistic structure and is important for communication among the hearing impaired. When a hearing-impaired person expresses gesture in the form of sign language, it is not a word-by-word gesture, but a continuous gesture. It is remarked that most studies about sign language recognition have concentrated on isolated sign word. While some studies have dealt with recognizing continuous sign language, they focused on simple connected types of isolated sign words. Continuous sign language gestures include not only sign words but also linking gestures (LGs) between words. To recognize continuous gestures, it is necessary to segment them into individual sign words. Thus, any method based solely on sign word recognition is not sufficient for real-world implementation. This thesis considers two conspicuous problems in for a continuously inputted hand motion pattern in Korean Sign Language (KSL). One is gesture segmentation problem, which means segmenting meaningful gestures (MGs) from a continuous sign language. LGs can occur between two consecutive words or between two consecutive sentences. This problem means to remove LGs from a continuous language, and to determine start and end point of MG. The other problem is a gesture matching (classification) problem, which is to decide what the gesture is by using extracted features. There are two different LGs. We call linking gesture between two consecutive sentences as $LG^#I$, while those between the words is $LG^#II$. When a signer has tendency to pause intentionally between sentences, but does not pause too much between words in a sentence. To notice pause of gesture, we have examined the motion of hand, that is, ``speed`` and ``change of speed`` of the hands. While sentence gesture and $LG^#I$ have different speed feature, there is almost no difference on speed features between $LG^#II$ and sign word gestures since the speed of $LG^#II$ is as fast as si...