Reflection removal using general-motion parameterization with truncated nuclear norm

Abstract

We propose a novel method for removing reflection from multiple images obtained through a short video clip. Motion difference between transmitted and reflection layer is exploited as a key observation for separating layers in each image. Due to an ill-posed nature of reflection separation problem, most of previous approaches model variables with prior assumption or restricted search space so as to shape the problem into a well-posed one. For example, in many works, motion field is modeled by a constrained motion, e.g., homography, epipolar line, etc., to reduce its degree of freedom. We observe that it is redundant to estimate motion field of both layers. Also, the motion estimation of reflection layer itself does not noticeably improve quality of a final separation. Therefore, we parameterize motions only on transmitted layers with an unconstrained search space. This allows us to significantly reduce the number of optimizing variables, as well as to make the proposed method compliant to general scene structures without being restricted by plane or rigid scene assumptions. Additionally, we regularize coherence of transmitted layers by a soft rank-1 penalty, i.e., truncated nuclear norm with rank-1, so that motion fields and other variables are led to better estimation even with a small number of input frames. Combining these key contributing components, we present a joint iterative optimization framework with the guidance of image-structural priors and demonstrate plausible quality of restoration that is competitive to other existing methods.