3D structure recovery and motion segmentation using uncalibrated cameras

Abstract

This thesis explores motion segmentation and 3D structure recovery from the image sequences that are captured by uncalibrated cameras. Recently, the needs of the extraction of information from the video sequences arise in many areas such as multimedia application, virtual reality, and augmented reality. The main difficulty to process video sequences comes from the unknown parameters of camera and multiple motions in the scene. This thesis presents a segmentation algorithm, which separates given sequences into two groups. One group is the area that corresponds to the static background under the moving camera, and it can be represented with uncalibrated 3D motion model and projective depth at each pixel. The other group is the area that corresponds to the multiple moving objects. These initial cues can be used effectively for the further processing. Robust direct motion estimation algorithm that considers the depth discontinuity is presented, and it uses an uncalibrated 3D motion model that is the most general motion model under the pin-hole camera assumption with uncalibrated cameras. Dominant motion corresponding to the ego-motion of camera is extracted by the proposed algorithm. Moving objects can be extracted by investigating the difference of the brightness value between the original and compensated image. But, this can hardly detect the homogenous regions on the moving objects. To solve this problem we employ Markov Random Field formulation that effectively uses the contextual information of the image. The results computed by above algorithm are on the projective domain. Calibration information is needed to convert it to the Euclidean. For this, two calibration algorithms are presented. One is a calibration algorithm that uses one type of scene constraint of known angles between lines. The other is an improved self-calibration algorithm using the additional constraints on the position of the principal points. The proposed calibration algorithm by known an...