Depth estimation for light field camera using multi-cue integrated cost volume

Abstract

With the recent introduction of commercial light field cameras, depth estimation using a light field camera draw significant interest of researchers. However, since the baseline of a light field camera is restricted to the size of lens aperture and its sub-aperture images are quite noisy, depth map estimation of a light field camera is a challenging problem. In this thesis, a robust depth map estimation method using multi-cue integrated cost volume is suggested. By using the cost volume concept, fast computation of depth map is achieved. To alleviate the ambiguity problem that comes from weakly textured region and narrow baseline of sub-aperture images, the pixels with reliable depth values are picked out and used to correct nearby unreliable depth values. A focus cue is exploited to filter out the pixels with unreliable depth values. A pixel with a depth value that falls within a confidence interval defined by the focus cue is classified as the anchor pixel having a reliable depth value. Using the estimated reliable depth values, unreliable depth values are corrected with an assumption that nearby pixels with similar color values may have similar depth values. Such an assumption significantly improves the performance of a non-convex optimization step. A discrete-continuous optimizer is adopted to obtain the final depth map by minimizing an objective function consisting of the color-gradient consistency and the depth difference between nearby pixels. The discrete-continuous optimization is suitable for efficiently combining the discrete cost volume structure and the continuous depth smoothness term at the same time. The performance of the proposed method is systematically validated on both indoor and outdoor datasets. The experimental results show that the proposed method outperforms other existing methods.