Recognition of hand gesture for virtual environments using hidden markov model and colored petri nets

Abstract

In this study, we propose a framework for recognition of hand gesture for virtual world interaction. Recent researches dealing with hand gesture recognition problem tend to be limited in certain local aspects of the problem. Among the issues, the major problem addressed in this study is how to recognize the space-time variable patterns of nonlinear arm movement and to integrate with other attributes to find a proper interpretation. Our proposed approach employs a hidden Markov model(HMM) network to recognize three-dimensional nonlinear arm movement patterns. HMM is a statistical model that can automatically extract knowledge from samples and can cope with time- scale variance and shape variance while preserving the order of arm movement. Encoding step is devised so as to reduce rotational and translational variances of gesture conduction. A two-dimensional essential trajectory is extracted by finding a global gesture plane and then encoded as a sequence of directional changes across time. The encoding step also provides additional feature of modeling the pause occurrences in the gesture. Codified information is then fed into the HMM network which is responsible for segmentation and recognition of continuous arm movement. HMMs are interconnected each other to form a network by passing through juncture HMMs. Juncture HMM is devised to provide a property of modeling a few connections of the pseudo linear trajectory of connecting movement. On the other hand, the integration of gestural attributes are modeled with colored petri net(CPN). This modeling not only provides the representation of synchrony between hand gestural attributes but also helps the recognition of arm movement patterns in combination with the HMM network, thus leading to the whole interpretation of the hand gesture. As the testbed for the proposed recognition approaches, an interior layouting application was constructed. Gestural interaction is incorporated into the application by the situation ...