Print Email Facebook Twitter CEAP-360VR Title CEAP-360VR: A Continuous Physiological and Behavioral Emotion Annotation Dataset for 360 VR Videos Author Xue, Tong (Beijing Institute of Technology; Centrum Wiskunde & Informatica (CWI)) El Ali, Abdallah (Centrum Wiskunde & Informatica (CWI)) Zhang, T. (TU Delft Multimedia Computing; Centrum Wiskunde & Informatica (CWI)) Ding, Gangyi (Beijing Institute of Technology) Cesar, Pablo (TU Delft Multimedia Computing; Centrum Wiskunde & Informatica (CWI)) Date 2021 Abstract Watching 360 videos using Virtual Reality (VR) head-mounted displays (HMDs) provides interactive and immersive experiences, where videos can evoke different emotions. Existing emotion self-report techniques within VR however are either retrospective or interrupt the immersive experience. To address this, we introduce the Continuous Physiological and Behavioral Emotion Annotation Dataset for 360 Videos (CEAP-360VR). We conducted a controlled study (N=32) where participants used a Vive Pro Eye HMD to watch eight validated affective 360 video clips, and annotated their valence and arousal (V-A) continuously. We collected (a) behavioral (head and eye movements; pupillometry) signals (b) physiological (heart rate, skin temperature, electrodermal activity) responses (c) momentary emotion self-reports (d) within-VR discrete emotion ratings (e) motion sickness, presence, and workload. We show the consistency of continuous annotation trajectories and verify their mean V-A annotations. We find high consistency between viewed 360 video regions across subjects, with higher consistency for eye than head movements. We furthermore run baseline classification experiments, where Random Forest classifiers with 2s segments show good accuracies for subject-independent models: 66.80% (V) and 64.26% (A) for binary classification; 49.92% (V) and 52.20% (A) for 3-class classification. Our open dataset allows further experiments with continuous emotion self-reports collected in 360 VR environments, which can enable automatic assessment of immersive Quality of Experience (QoE) andmomentary affective states. Subject 360 videocontinuous annotationdatasetemotionhead and eye movementHMDphysiological signalsvirtual reality To reference this document use: http://resolver.tudelft.nl/uuid:d959fe58-0455-4d11-b104-dd9d35866923 DOI https://doi.org/10.1109/TMM.2021.3124080 Embargo date 2023-07-01 ISSN 1520-9210 Source IEEE Transactions on Multimedia, 25, 243-255 Bibliographical note Green Open Access added to TU Delft Institutional Repository ‘You share, we take care!’ – Taverne project https://www.openaccess.nl/en/you-share-we-take-care Otherwise as indicated in the copyright section: the publisher is the copyright holder of this work and the author uses the Dutch legislation to make this work public. Part of collection Institutional Repository Document type journal article Rights © 2021 Tong Xue, Abdallah El Ali, T. Zhang, Gangyi Ding, Pablo Cesar Files PDF CEAP_360VR_A_Continuous_P ... Videos.pdf 6.71 MB Close viewer /islandora/object/uuid:d959fe58-0455-4d11-b104-dd9d35866923/datastream/OBJ/view