Multicue-based automatic face detection/tracking with its application to user intention reading

Abstract

Video detection/tracking of human faces becomes essential for a wide range of applications, including human-robot interaction, smart home, and construction of video databases. Since some form of automatic detection/tracking mechanism is employed when massive information of high-level context is processed as in facial expression recognition, intention reading, and behavioral understanding, it is desirable to keep the computational complexity of the face detection/tracking process as low as possible, maintaining a high detection rate and low false positive rate under various environmental conditions. However, previous approaches naively combined face detection module and face tracking module or do not show good performance in terms of detection rate, the number of false positives, and computational cost due to the ineffective integration of multiple cues. In particular, high computational cost and many false positives, and converging to a local maximum due to the face-like object are the critical problems of conventional techniques in face detection process and face tracking process, respectively. In this thesis, a unified framework, called $“Multicue-based Dynamic Cascade Structure (MUDCAST)”$ is thus proposed for fast and robust automatic face detection/tracking in the environment where there are objects with features similar to those of the face. The proposed face detection/tracking system consists of three stages, namely, a pre-attentive stage, an assignment stage, and a post-attentive stage, which are cascaded and integrated with multimodal cues for effective and efficient processing of data. Here, the valid cues are selected in consideration of the trade-off between detection rate and the number of false positives occurred when we detect/track the face with the selected cues. Differing from typical automatic face detection/tracking structure, MUDCAST rapidly extracts face candidate regions in the pre-attentive stage, and then it assigns them to face de...