Contextual relationship-based activity segmentation on an event stream in the IoT environment with multi-user activities

Abstract

The human activity recognition in the IoT environment plays the central role in the ambient assisted living, where the human activities can be represented as a concatenated event stream generated from various smart objects. From the concatenated event stream, each activity should be distinguished separately for the human activity recognition to provide services that users may need. In this regard, accurately segmenting the entire stream at the precise boundary of each activity is indispensable high priority task to realize the activity recognition. Multiple human activities in an IoT environment generate varying event stream patterns, and the unpredictability of these patterns makes them include redundant or missing events. In dealing with this complex segmentation problem, we figured out that the dynamic and confusing patterns cause major problems due to: inclusive event stream, redundant events, and shared events. To address these problems, we exploited the contextual relationships associated with the activity status about either ongoing or terminated/started. To discover the intrinsic relationships between the events in a stream, we utilized the LSTM model by rendering it for the activity segmentation. Then, the inferred boundaries were revised by our validation algorithm for a bit shifted boundaries. Our experiments show the surprising result of high accuracy above 95%, on our own testbed with various smart objects. This is superior to the prior works that even do not assume the environment with multi-user activities, where their accuracies are slightly above 80% in their test environment. It means that our work is feasible enough to be applied in the IoT environment.