Unified stereo-based 3d head tracking using online illumination modeling and its application to face recognition

Abstract

This paper investigates the estimation of 3D head poses and its application to face recognition with a partial ellipsoid model. A human face provides a variety of different communicative functions such as identification, the perception of emotional expression, and lip-reading. For these reasons, many applications in robotics and computer vision fields require tracking and recognizing a human face. Since face space as a subspace of image space is highly nonlinear and non-convex, face recognition evaluation reports and other independent studies indicate that the performance of many state-of-the-art face recognition methods deteriorates with changes in pose, illumination, and other disturbing factors, among which pose variation is the most difficult one to deal with. Therefore, face registration is the key of robust face recognition. If we can register face images with pose variations into frontal views, the recognition task would be much easier. To align a face image into a canonical frontal view, we need to know the 3D pose information of a human head. In this dissertation, a simple method for modeling a human head as a partial 3D ellipsoid is firstly introduced. A 3D ellipsoid is a reasonable approximation to the shape of the actual head and can be easily initialized by a simple 2D face detector. For face detection and model initialization, a novel approach, which is based on the discriminative MspLBP features selected by a boosting technique called Ada-LDA, is presented. Generally, 3D model-based tracking with a single camera is not robust to fast and large out-of-plane rotations and especially movement in depth (along camera Z-axis). To cope with large out-of-plane rotations and depth-motion, we extend conventional 3D model-based tracking with a single camera to a stereo-based framework. To achieve robust motion estimation even under time-varying lighting conditions, we incorporate online illumination correction into the aforementioned framework. We appr...