A Novel Depth-Based Virtual View Synthesis Method for Free Viewpoint Video

Abstract

Free-viewpoint rendering (FVR) has become a popular topic in 3-D research. A promising technology in FVR is to generate virtual views using a single texture image and the corresponding depth image. A critical problem that occurs when generating virtual views is that the regions covered by the foreground objects in the original view may be disoccluded in the synthesized views. In this paper, a depth based disocclusion filling algorithm using patch-based texture synthesis is proposed. In contrast to the existing patch-based virtual view synthesis methods, the filling priority is driven by the robust structure tensor that efficiently reflects the overall structure of an image part and a new confidence term that produces fine synthesis results even near the foreground boundaries. Moreover, the best-matched patch is searched in the background regions and finally it is chosen through a new patch distance measure. Significant superiority of the proposed method over the state-of-the-art methods is presented by comparing the experimental results.