As environments become smart in accordance with
advances in ubiquitous computing technology, researchers are
struggling to satisfy users’ diverse and sophisticated demands.
The aim of the present work is to enable multiple persons in a
interactive virtual environment to simultaneously and
conveniently interact with virtual agents. To this end, we propose
a real-time system that robustly tracks multiple persons in virtual
environments and recognizes their actions through image
sequences acquired from a single fixed camera. The proposed
system is compromised of three components: blob extraction,
object tracking, and human action recognition. Given an image,
we extract blobs using the Mixture of Gaussians algorithm with a
hierarchical data structure and we additionally remove shadows
and highlights in order to obtain a more accurate object silhouette.
We then track multiple objects using a motion-based object model
and an inference graph for handling grouping and fragment
problems. Finally, we model an action as a Motion History Image
(MHI) based on given object tracks, normalize the MHI, reduce
the MHI using PCA, and classify an action using a multi-layer
perceptron. To evaluate the performance of the proposed system,
we employed it in an augmented reality application where
multiple persons can interact with a virtual pet. The results
confirm that reliable object tracking is achieved and multiple
persons’ actions can be recognized for applications in interactive
virtual environments.