A study on moving object detection and tracking with partial decoding in H.264|AVC bitstream domain

Abstract

An object detection and tracking technique has been an important issue traditionally in the field of computer vision and video processing since it enables efficient analysis of video contents. It can be utilized not merely for surveillance systems but also for interactive broadcasting services. However, most of current object detection and tracking techniques which utilize only raw pixel data are not practical due to tremendously high computational complexity. Furthermore, most of videos tend to be communicated in the form of encoded bitstreams in order to enhance the transmission efficiency. In that case, the pixel domain approach requires additional computation time to fully decode the encoded bitstream. In the meantime, H.264|AVC technology has been a popular compression tool for videos due to its high coding efficiency and the availability of its real-time encoding devices. Fortunately, the H.264|AVC bitstream contains encoded information such as motion vectors, residual data, and macroblock types which can be directly utilized as effective clues for object detection and tracking. The traditional compressed domain algorithms which make use of such encoded information have shown fast computation time with low computational complexity. However, these algorithms are available only under limited circumstances. In addition, they are difficult to be followed by the color extraction of objects or the object recognition which distinguishes one object from other objects. In this thesis, two methods for moving object detection and tracking with partial decoding in H.264|AVC bitstream domain are introduced. While one approach is the semi-automatic method which users can initially select a target object in stationary or non-stationary scenes, another approach is the automatic method which all moving objects are automatically detected and tracked especially in stationary scenes. The former is beneficial to metadata authoring tools which generate additional content...