On Head Motion for Recognizing Aggression and Negative Affect during Speaking and Listening

On Head Motion for Recognizing Aggression and Negative Affect during Speaking and Listening

Fitrianie, S. (TU Delft Interactive Intelligence)
Lefter, I. (TU Delft System Engineering)

2023

Affective aggression is a form of aggression characterized by impulsive reactions driven by strong negative emotions. Despite the extensive research in the area of automatic emotion recognition, affective aggression is a phenomenon that has received less attention. This study investigates the use of head motion as a potential indicator of affective aggression and negative affect. It provides an analysis of head movement patterns associated with various levels of aggression, valence, arousal and dominance, and compares behaviors and recognition performance under speaking and listening conditions. The study was conducted on the Negative Affect and Aggression database - a multimodal corpus of dyadic interactions between aggression regulation training actors and non-actors, annotated for levels of aggression, valence, arousal, and dominance. Results demonstrate that head motion features can serve as promising indicators of affect during both speaking and listening. Valence and arousal prediction achieved better performance during speaking, while aggression and dominance were better predicted during listening. Significant increases in the magnitude of pitch angular acceleration were associated with escalation along all four annotated dimensions. Interestingly, higher escalation was accompanied by a significant increase in the total number of movements during speaking, but a significant decrease of the number of movements was observed as escalation increased along listening intervals. These findings are particularly relevant as head motion can be used solely or potentially as a supplementary modality when other modalities such as speech or facial expressions are unavailable or altered.

Affective Aggression
Affective Computing
Emotion
Head Motion
NAA database
Speaking and Listening Behavior.

http://resolver.tudelft.nl/uuid:db96ce73-7c10-4be7-a72d-52c3d3d8e79f

https://doi.org/10.1145/3577190.3614169

Association for Computing Machinery (ACM)

9798400700552

ICMI 2023 - Proceedings of the 25th International Conference on Multimodal Interaction

25th International Conference on Multimodal Interaction, ICMI 2023, 2023-10-09 → 2023-10-13, Paris, France

ACM International Conference Proceeding Series

Institutional Repository

conference paper

© 2023 S. Fitrianie, I. Lefter