Print Email Facebook Twitter Exploring the Detection of Spontaneous Recollections during Video-viewing In-the-Wild using Facial Behavior Analysis Title Exploring the Detection of Spontaneous Recollections during Video-viewing In-the-Wild using Facial Behavior Analysis Author Dudzik, B.J.W. (TU Delft Pattern Recognition and Bioinformatics) Hung, H.S. (TU Delft Pattern Recognition and Bioinformatics) Date 2022 Abstract Intelligent systems might benefit from automatically detecting when a stimulus has triggered a user's recollection of personal memories, e.g., to identify that a piece of media content holds personal significance for them. While computational research has demonstrated the potential to identify related states based on facial behavior (e.g., mind-wandering), the automatic detection of spontaneous recollections specifically has not been investigated this far. Motivated by this, we present machine learning experiments exploring the feasibility of detecting whether a video clip has triggered personal memories in a viewer based on the analysis of their Head Rotation, Head Position, Eye Gaze, and Facial Expressions. Concretely, we introduce an approach for automatic detection and evaluate its potential for predictions using in-the-wild webcam recordings. Overall, our findings demonstrate the capacity for above chance detections in both settings, with substantially better performance for the video-independent variant. Beyond this, we investigate the role of person-specific recollection biases for predictions of our video-independent models and the importance of specific modalities of facial behavior. Finally, we discuss the implications of our findings for detecting recollections and user-modeling in adaptive systems. Subject Affective ComputingCognitive ProcessingFacial Behavior AnalysisMemoriesMind-WanderingRecollectionUser-Modeling To reference this document use: http://resolver.tudelft.nl/uuid:915744c3-4ae8-4f31-bfe8-8a336759967a DOI https://doi.org/10.1145/3536221.3556609 Publisher Association for Computing Machinery (ACM) ISBN 9781450393904 Source ICMI 2022 - Proceedings of the 2022 International Conference on Multimodal Interaction Event 24th ACM International Conference on Multimodal Interaction, ICMI 2022, 2022-11-07 → 2022-11-11, Bangalore, India Series ACM International Conference Proceeding Series Part of collection Institutional Repository Document type conference paper Rights © 2022 B.J.W. Dudzik, H.S. Hung Files PDF 3536221.3556609.pdf 1.18 MB Close viewer /islandora/object/uuid:915744c3-4ae8-4f31-bfe8-8a336759967a/datastream/OBJ/view