Early recognition of multi-user common intention based on multi-dimensional contexts in a smart space

Abstract

As the smart space is extending from the smart home to the smart city, more and more users and IoT devices will be involved in the future smart space. To address this new challenge, we explored how we can utilize the abundant data from the space to enhance user convenience with spontaneous services. In many cases, a group of people enter a smart space to conduct some activities for their common intention. If the system can recognize the intention as earlier as possible, the system will be able to provide more appropriate and optimized services for users, e.g., presentation control service and smart environment auto-configuration for the intention of ‘have a meeting.’ As the initial endeavor, we set out to recognize the Multi-User Common Intention (MCI) even in its early stage. To recognize MCI as early and accurately as possible, it is necessary to obtain enough appropriate contexts to learn the MCI in the very early stage for the machine learning system. For that, we figured out and utilized Multi-Dimensional Contexts, such as User-Activity, Ambient Status, Social Relationship and Time-Zone contexts, which cover sufficient inter-relational information among users and devices in the early stage. To utilize the Multi-Dimensional Contexts, we devised a novel task model that prescribes the formal structure of input and output for the machine learning to determine the MCI. For intention recognition, we customized a long short-term memory (LSTM) model to deal with various time-series data and static data, which is called DSM-LSTM. We evaluated the proposed methodology of the Early Recognition of MCI, using IoT data from a real smart office testbed, and our experiments showed high accuracy (85-90%) of intention recognition in 2-3 minutes after users enter a space. Based on our extensive experiments and analyses, we found that multi-users’ Social Relationship context, such as the group diversity index, was the most significant factor in early recognition of the MCI. These results confirm that our system can support the early MCI recognition for many smart space services and applications.