Feature tracking from multiple views using the split and merge method

Abstract

In this dissertation, we propose a new method for tracking multiple features from multiple views over time. Features are assumed to move independently. This problem is typically formulated as a multidimensional matching problem (MMP), which is NP-complete with computational complexity $O((n!)^{2m-1})$, where n and m are the number of features and views, respectively. Our method reduces the computational complexity to conceived $O(n^6m^2)$. A mathematical framework, called quantitative relational structure (QRS), is to formulate the multi-views and motion correspondence problem. QRS encodes the correspondence across the view as well as across sequence. Energy function to be optimized in defined QRS. This multidimensional matching problem is reduced to a manageable one by adopting the split-merge strategy. The split is the one that divides multi-view into multiple two views. This stage solves stereo-motion correspondence problem. We apply the concept of graduated assignment algorithm, called extended assignment algorithm. Final outcomes of the split are $^mC_2$ stereo-motion correspondence and corresponding energy values. Merge is the process in that sliced stereo-motion correspondences problems are merged to multi-views and motion correspondence one. This method proceeds repeatedly. A structured cluster is proposed. It is very important for the merge and can imply a feature independently moving. A set of structured clusters indicates multi-views and motion correspondence. The overall process is the following. The merge process is started by selecting two views $v_1$ and $v_2$ having maximal energy values among $m_C_2$ stereo-motion correspondences. Then another view $v_3$ is chosen among the rest. Then three stereo-motion correspondences are merged into multi-views $(v_1.v_2,v_3)$ and motion correspondence. This process is repeated until all views are merged or no more increase the energy values. Experimental results are presented to illustrate the performance...