In this thesis, a real-time recognizing system is studied for human machine interface. Specifically, the problem of vision-based human hand tracking is studied. In this study, the real-time implementation issue and a means of how to convey various human intentions to a machine are investigated. For this, a special color vision system for real-time image processing is designed. Also, for recognition of hand gestures, expandable gesture sets are introduced. This system overcomes some disadvantages of a hand tracking device that does not show other objects besides the hand motion To show applications of the proposed system, human machine interface to PUMA560 is implemented and experimentally tested.