Multimodal emotion recognition using multi-attribute aggregation for attributes with uncertainties = 불확실성을 고려한 다중 속성 통합 프레임워크 기반 멀티모달 감성인식

Cited 0 time in webofscience Cited 0 time in scopus
  • Hit : 316
  • Download : 0
Affective interaction between humans and robots/machines is a cherished goal for socially intelligent robots/machines. Ability to recognize human emotional states is an essential prerequisite to such affective interactions. Therefore, this dissertation addresses the issue of human emotion recognition by first individually processing and then aggregating different modes of human communication through a classification and aggregation framework. Specifically, the proposed framework analyzes the speech acoustics, facial expressions, and body language using unimodal emotion classifiers. The speech emotion is classified using a deep neural network (DNN) while Facial and body language emotion classifiers are implemented using classifiers implemented through supervised fuzzy adaptive resonance theory (ARTMAP). The speech emotion classifier uses acoustic features, the facial emotion classifier uses features based on facial animation parameters (FAP), and body language emotion classifier uses head and hands motion capture data to formulate body language features. These unimodal evaluations are then aggregated using a fuzzy integral for interval type-2 fuzzy-valued attributes (FIIFA). FIIFA is proposed in this dissertation as a novel aggregation framework for attribute evaluations with linguistic and numeric uncertainties. Moreover, FIIFA also utilizes reliability based preferences for the unimodal evaluations. The dissertation proposed to generate these reliabilities based preferences from the accuracies of the unimodal classifiers for each emotion. The framework was tested and compared against the existing state-of-the-art. The results show that the proposed framework significantly outperforms the existing techniques. Furthermore, because of late fusion, the functionality of the proposed approach is robust to unavailability all but one of the modes of communication.
Kim, Jong-Hwanresearcher김종환researcher
한국과학기술원 :전기및전자공학부,
Issue Date

학위논문(박사) - 한국과학기술원 : 전기및전자공학부, 2017.2,[iv, 61 p. :]


Emotion recognition; Multi-attribute aggregation; Affective human robot interaction (HRI); Fuzzy Integral; Default ARTMAP; 감정인식; 다중 속성 통합; 감정적 상호 작용; 퍼지 적분

Appears in Collection
Files in This Item
There are no files associated with this item.


  • mendeley


rss_1.0 rss_2.0 atom_1.0