Deep facial landmark learning by component priors

Abstract

Facial landmark detection (FLD) is vital for diverse applications such as face recognition, tracking, expression recognition, and 3D modeling, etc. Over the past few decades, FLD has been researched as one of the major topics in computer vision. Although FLD has been significantly advanced, it is not solved perfectly because of large variations (e.g., pose variations, expression variations, illumination variations, and occlusion) in face images. In order to achieve robust FLD performance, we propose a novel deep facial landmark learning by component priors, which trains two deep convolutional neural networks (DCNNs): DCNN-I and DCNN-II, which localize the landmarks in facial inner components, and facial contour, respectively. The landmarks in facial contour is significantly more difficult to be learned than those of facial inner components, because facial contour has less texture information and more noise from the background compared to the facial inner components. Therefore, two independent DCNN-I, and DCNN-II are trained and the whole system has a chance to learn more specific to each facial inner component and facial contour. In order to further improve the detection performance of DCNN-I and DCNN-II, we propose a novel structure of DCNN-I and learning strategy of DCNN-II. To consider overall shape and local facial components simultaneously, we separate the higher layers of DCNN-I into branches to detect the landmarks of each facial inner components. From the observation that performance for detecting landmarks in facial contour is improved when learning the facial contour landmarks jointly with the landmarks in facial inner components, the one convolutional neural network, that has branches of higher layers for each facial component including facial inner components and facial contour, is learned. After learning, we take the weights of the lower layers and the higher layers corresponding to the facial contour, as the DCNN-II. Extensive experiments conducted on the challenging 300-W dataset shows that the proposed method outperforms the state-of-the-art FLD approaches.