Investigating Affective Responses toward In-Video Pedestrian Crossing Actions using Camera and Physiological Sensors

Investigating Affective Responses toward In-Video Pedestrian Crossing Actions using Camera and Physiological Sensors

Rao, Shruti (Centrum Wiskunde & Informatica (CWI))
Ghosh, Surjya (Birla Institute of Technology and Science, Pilani)
Pons Rodriguez, Gerard (Centrum Wiskunde & Informatica (CWI))
Röggla, Thomas (Centrum Wiskunde & Informatica (CWI))
El Ali, Abdallah (Centrum Wiskunde & Informatica (CWI))
Cesar, Pablo (TU Delft Multimedia Computing; Centrum Wiskunde & Informatica (CWI))

2022

Automatically inferring drivers' emotions during driver-pedestrian interactions to improve road safety remains a challenge for designing in-vehicle, empathic interfaces. To that end, we carried out a lab-based study using a combination of camera and physiological sensors. We collected participants' (N=21) real-time, affective (emotion self-reports, heart rate, pupil diameter, skin conductance, and facial temperatures) responses towards non-verbal, pedestrian crossing videos from the Joint Attention for Autonomous Driving (JAAD) dataset. Our findings reveal that positive, non-verbal, pedestrian crossing actions in the videos elicit higher valence ratings from participants, while non-positive actions elicit higher arousal. Different pedestrian crossing actions in the videos also have a significant influence on participants' physiological signals (heart rate, pupil diameter, skin conductance) and facial temperatures. Our findings provide a first step toward enabling in-car empathic interfaces that draw on behavioural and physiological sensing to in situ infer driver emotions during non-verbal pedestrian interactions.

driver emotion recognition
empathic car
pedestrian behaviour

http://resolver.tudelft.nl/uuid:98b8de6e-d4c2-4274-bf5d-bf9ca41db4db

https://doi.org/10.1145/3543174.3546842

Association for Computing Machinery (ACM), New York

978-1-4503-9415-4

AutomotiveUI '22: Proceedings of the 14th International Conference on Automotive User Interfaces and Interactive Vehicular

14th International ACM Conference on Automotive User Interfaces and Interactive Vehicular Applications, AutomotiveUI 2022, 2022-09-17 → 2022-09-20, Seoul, Korea, Republic of

Institutional Repository

conference paper

© 2022 Shruti Rao, Surjya Ghosh, Gerard Pons Rodriguez, Thomas Röggla, Abdallah El Ali, Pablo Cesar