Detecting user attention to video segments using interval EEG features

Abstract

To manage voluminous viewed videos, which US adults watch at a rate of more than five hours per day on average, an automatic method of detecting highly attended video segments during video viewing is required to access them for fine-grained sharing and rewatching. Most electroencephalography (EEG) based studies of user state analysis have addressed the recognition of attention-related states in a specific task condition, such as drowsiness during driving, attention during learning, and mental fatigue during task execution. In contrast to attention in a specific task condition, both inattention and normal attention are meaningless to viewers in terms of managing viewed videos, while detecting high attention paid to video segments would make a valuable contribution to an automatic management system of viewed videos based on viewer attention. To the best of our knowledge, this is the first EEG-based study of detecting viewer attention paid to video segments. This study describes how to collect video-induced EEG and attention data for video segments from viewers without bias to specific genres and how to construct a subject-independent detection model for the top 20% of viewer attention. The attention detection model using the proposed interval EEG features from 14 channels achieved the best average F1 score of 39.79% with an average accuracy of 52.96%. Additionally, this paper proposes a channel-based feature selection method that considers both the performances of single-channel models and their physical locations for investigating the group of channels relevant to attention detection. The attention detection models using the interval EEG features from all four or some of the channels located in the fronto-central, parietal, temporal, and occipital lobes of the left hemisphere achieved the best F1 score of 39.60% with an average accuracy of 48.70%. It is shown that these models aehieve better performance than models using the features from all four or some of their symmetric channels in the right hemisphere and models using the features from six channels located in the anterior-frontal and frontal lobes of the left and right hemispheres. This paper shows the feasibility of subject-independent and genre-independent attention detection models using a wireless EEG headset with optimized channels: these models can be applied to an intelligent video management system based on viewer attention in real-world scenarios. (C) 2018 Elsevier Ltd. All rights reserved.