Automatic Face Annotation in Personal Photo Collections Using Context-Based Unsupervised Clustering and Face Information Fusion

Abstract

In this paper, a novel face annotation framework is proposed that systematically leverages context information such as situation awareness information with current face recognition (FR) solutions. In particular, unsupervised situation and subject clustering techniques have been developed that are aided by context information. Situation clustering groups together photos that are similar in terms of capture time and visual content, allowing for the reliable use of visual context information during subject clustering. The aim of subject clustering is to merge multiple face images that belong to the same individual. To take advantage of the availability of multiple face images for a particular individual, we propose effective FR methods that are based on face information fusion strategies. The performance of the proposed annotation method has been evaluated using a variety of photo sets. The photo sets were constructed using 1385 photos from the MPEG-7 Visual Core Experiment 3 (VCE-3) data set and approximately 20 000 photos collected from well-known photo-sharing websites. The reported experimental results show that the proposed face annotation method significantly outperforms traditional face annotation solutions at no additional computational cost, with accuracy gains of up to 25% for particular cases.