Spatio-temporal analysis of image sequence

Abstract

An image sequence contains various spatio-temporal information. To accomplish a robust image analysis this kind of information is very useful. In this paper the spatio-temporal information of image sequence is used on edge detection and optical flow estimation. Additional criteria to the spatio-temporal edge detection is suggested; the temporal compensation and the motion independency. And two ways of using spatio-temporal information on edge detection are introduced. The first estimates the spatial gradient or the spatial Laplacian with spatio-temporal filters, and the second considers the spatial edge as a temporal slice of the spatio-temporal edge (TSSTE). Since the image sequence is not continuous on time axis, the TSSTE results in thick edges when the image motion is larger than one pixel per frame. Several post-processing techniques are suggested to extract an exact edge. Some experiments show that these kinds of edge detectors result in more robust edges. The spatio-temporal estimation of gradient also helps optical flow estimation. Applying the spatio-temporal gradient estimation to the traditional gradient method brings significantly improved results. A new spatio-temporal image model is also derived. This is a quite different model which has an ability to represent the temporal variance of the image flow. For some kinds of image sequences, the proposed model showed more robust results. But the proposed model did not show satisfiable results especially to the moving motion boundary. This is deeply related with the traditional motion boundary problem and remains for future work.