Robust Multi-Object Tracking With Local Appearance and Stable Motion Models

Abstract

Multi-object tracking (MOT) has been steadily studied for video understanding in computer vision. However, existing MOT frameworks usually employ straightforward appearance or motion models and may struggle in dynamic environments with similar appearance and complex motion. In this paper, we present a robust MOT framework with local appearance and stable motion models to overcome these two hindrances. The framework incorporates object and local part detectors, a feature extractor, a keypoint extractor, and a data association method. For the data association, we utilize five types of similarity metrics and a cascaded matching strategy. The local appearance model is suggested to be used additionally with global appearance features of full bounding boxes to obtain discriminative features even for objects with a similar appearance. At the same time, the stable motion model considers the core of the body as the central point of the object and subdivides the body using a novel 12-tuple Kalman state vector to analyze complex motion. As a result, our new tracker achieves state-of-the-art performance on the DanceTrack test set, surpassing all other listed tracking systems in terms of both detection and tracking quality metrics, obtaining 61.3 HOTA, 82.3 DetA, 45.8 AssA, and 91.7 MOTA. The source code is available at https://github.com/Jubi-Hwang/Robust-MOT-with-Local-Appearance-and-Stable-Motion-Models.