Robust video fingerprinting for content-based video identification

Abstract

Video fingerprints are short feature vectors that uniquely characterize one video clip from another. The goal of video fingerprinting is to identify a given video query in a database by measuring the distance between the query fingerprint and the fingerprints in the database. The video fingerprinting systems generally consist of three parts: fingerprint extraction, database search, and fingerprint matching. In this dissertation, various issues of fingerprint extraction and matching are addressed. First, a video fingerprinting method based on a novel fingerprint called the centroid of gradient orientations is proposed. The proposed fingerprint is not only pairwisely independent but also robust against common video processing steps including lossy compression, resizing, frame rate change, etc. A threshold used to reliably determine a fingerprint match is theoretically derived by modelling the proposed fingerprint as a stationary ergodic process, and the validity of the model is experimentally verified. Second, a video fingerprinting method based on a novel machine learning algorithm called the symmetric pairwise boosting is proposed. Given a set of training data, the symmetric pairwise boosting learns a binarization method which can convert a real-valued fingerprint into a robust and pairwisely independent binary fingerprint. Third, a video fingerprinting method based on the discriminant analysis of affine covariant regions is proposed. To achieve robustness against geometric transformations such as cropping and rotation, fingerprints are extracted from local regions covariant with affine transformations. Robust local fingerprints are extracted using a novel discriminant analysis algorithm called the 2-dimensional oriented principal component analysis. For reliable matching of local fingerprints, only spatio-temporally consistent matches are taken into account. The performance of all the proposed methods is experimentally evaluated using a database of video clips...