Robust correspondence search using local and global constraints

Abstract

Finding correspondences across images is the essence of many computer vision problems such as object tracking and recognition, camera-pose estimation, and 3D scene-structure reconstruction. What makes the correspondence problem difficult is repeated local scene structure and texture, which are the main sources of ambiguity when using local observation. In this thesis, we aim at finding feature correspondences across images under significant geometric and photometric changes, in the presence of such sources of ambiguity. We propose two kinds of approaches using local and global constraints respectively. In the first approach using local constraints, we formulate feature matching as a global optimization problem. We propose a viewpoint-adaptive objective function based on local pairwise constraints requiring local affine transformations to be well preserved across feature correspondences. We also propose an optimization algorithm that not only maximizes the objective function but also enforces one-to-one correspondence constraints. Through experiments involving various kinds of image pairs, we show that the proposed objective function is effective for reducing ambiguity under geometric and photometric changes between images, in contrast to previous objective functions that aim at preserving across images the distance or orientation between features. We also show that the proposed optimization algorithm better maximizes the objective function than a state-of-the-art algorithm. In the second approach using global constraints, we represent a global image-to-image transformation with multiple homographies and their associated statistical values, in order to deal with local deformations. We propose an expectation maximization algorithm for learning the global transformation; we detect and create feature correspondences using a learned global transformation. This second approach is combined with the first approach, maximizing correct correspondences and minimiz...