Title
Context in Human Emotion Perception for Automatic Affect Detection: A Survey of Audiovisual Databases
Author
Dudzik, B.J.W. (TU Delft Interactive Intelligence)
Jansen, Michel Pierre (University of Twente)
Burger, Franziska (TU Delft Interactive Intelligence)
Kaptein, F.C.A. (TU Delft Interactive Intelligence)
Broekens, D.J. (Universiteit Leiden)
Heylen, Dirk K.J. (University of Twente)
Hung, H.S. (TU Delft Pattern Recognition and Bioinformatics)
Neerincx, M.A. (TU Delft Interactive Intelligence)
Truong, Khiet P. (University of Twente)
Date
2019-09-01
Abstract
An important aspect of human emotion perception is the use of contextual information to understand others' feelings even in situations where their behavior is not very expressive or has an emotionally ambiguous meaning. For technology to successfully detect affect, it must mimic this human ability when analyzing audiovisual input. Databases upon which machine learning algorithms are trained should capture the context of social interactions as well as the behavior expressed in them. However, there is a lack of consensus about what constitutes relevant context in such databases. In this article, we make two contributions towards overcoming this challenge: (a) we identify two principal sources of context for emotion perceptions based on psychological theory, and (b) we provide an overview of how each of these has been considered in published databases covering social interactions. Our results show that a similar set of contextual features are present across the reviewed databases. Between all the different databases researchers seem to have taken into account a set of contextual features reflecting the sources of context seen in psychological theory. However, within individual databases, these features are not yet systematically varied. This is problematic because it prevents them from being used directly as resources for the modeling of context-sensitive affect detection. Based on our findings, we suggest improvements for the future development of affective databases.
Subject
Audiovisual Databases
Automatic Affect Detection
Context
Human Emotion Perception
Survey
To reference this document use:
http://resolver.tudelft.nl/uuid:7d644f30-00b2-4e76-b88d-b0183c5ae36d
DOI
https://doi.org/10.1109/ACII.2019.8925446
Publisher
IEEE
Embargo date
2022-04-08
ISBN
9781728138886
Source
2019 8th International Conference on Affective Computing and Intelligent Interaction, ACII 2019
Event
8th International Conference on Affective Computing and Intelligent Interaction, ACII 2019, 2019-09-03 → 2019-09-06, Cambridge, United Kingdom
Series
2019 8th International Conference on Affective Computing and Intelligent Interaction, ACII 2019
Bibliographical note
Green Open Access added to TU Delft Institutional Repository ‘You share, we take care!’ – Taverne project https://www.openaccess.nl/en/you-share-we-take-care Otherwise as indicated in the copyright section: the publisher is the copyright holder of this work and the author uses the Dutch legislation to make this work public.
Part of collection
Institutional Repository
Document type
conference paper
Rights
© 2019 B.J.W. Dudzik, Michel Pierre Jansen, Franziska Burger, F.C.A. Kaptein, D.J. Broekens, Dirk K.J. Heylen, H.S. Hung, M.A. Neerincx, Khiet P. Truong