View-dependent scene appearance synthesis using inverse rendering from light fields

Abstract

Light field imaging is a technology capable of acquiring abundant ray information in 3D, and has the advantage of acquiring viewpoint-dependent information compared to a conventional 2D camera that can record only incident light from a fixed viewpoint. This view-dependent information contains observations necessary to reconstruct three-dimensional information in space. In this dissertation, we propose an inverse-rendering based system capable of synthesizing the appearance that preserves the view-point dependence even at the unobserved point by using the dense sample ratio of the wide-baseline light field. This system acquires the light field data using 32 cameras arranged in a grid, and the scene is effectively expressed by calculating the geometric information and reflection property parameters considering the reflectivity of the object. Through this system, it is possible to synthesize the scene from a new viewpoint while reserving information on the actual reflection characteristics of an object even using a commercial mesh-based renderer.