Augmenting Media Experiences with Affective Haptics

Within our Distributed and Interactive Systems research group, we focus on affective haptics, where we design and develop systems that can enhance human emotional states through the sense of touch. Such artificial haptic sensations can potentially augment and enhance our mind, body, and (virtual) social connections. In three works—voice...

Transparent AI Disclosure Obligations: Who, What, When, Where, Why, How

Advances in Generative Artificial Intelligence (AI) are resulting in AI-generated media output that is (nearly) indistinguishable from human-created content. This can drastically impact users and the media sector, especially given global risks of misinformation. While the currently discussed European AI Act aims at addressing these risks through...

BreatheWithMe: Exploring Visual and Vibrotactile Displays for Social Breath Awareness during Colocated, Collaborative Tasks

Sharing breathing signals has the capacity to provide insights into hidden experiences and enhance interpersonal communication. However, it remains unclear how the modality of breath signals (visual, haptic) is socially interpreted during collaborative tasks. In this mixed-methods study, we design and evaluate BreatheWithMe, a prototype for...

FeelTheNews: Augmenting Affective Perceptions of News Videos with Thermal and Vibrotactile Stimulation

Emotion plays a key role in the emerging wave of immersive, multi-sensory audience news engagement experiences. Since emotions can be triggered by somatosensory feedback, in this work we explore how augmenting news video watching with haptics can influence affective perceptions of news. Using a mixed-methods approach, we design and evaluate...

Reflecting on Hybrid Events: Learning from a Year of Hybrid Experiences

The COVID-19 pandemic led to a sudden shift to virtual work and events, with the last two years enabling an appropriated and rather simulated togetherness - the hybrid mode. As we return to in-person events, it is important to reflect on not only what we learned about technologies and social justice, but about the types of events we desire,...

Affective Driver-Pedestrian Interaction: Exploring Driver Affective Responses toward Pedestrian Crossing Actions using Camera and Physiological Sensors

Eliciting and capturing drivers' affective responses in a realistic outdoor setting with pedestrians poses a challenge when designing in-vehicle, empathic interfaces. To address this, we designed a controlled, outdoor car driving circuit where drivers (N=27) drove and encountered pedestrian confederates who performed non-verbal positive or...

From Video to Hybrid Simulator: Exploring Affective Responses toward Non-Verbal Pedestrian Crossing Actions Using Camera and Physiological Sensors

Capturing drivers’ affective responses given driving context and driver-pedestrian interactions remains a challenge for designing in-vehicle, empathic interfaces. To address this, we conducted two lab-based studies using camera and physiological sensors. Our first study collected participants’ (N = 21) emotion self-reports and physiological...

Weakly-supervised Learning for Fine-grained Emotion Recognition using Physiological Signals

Instead of predicting just one emotion for one activity (e.g., video watching), fine-grained emotion recognition enables more temporally precise recognition. Previous works on fine-grained emotion recognition require segment-by-segment, fine-grained emotion labels to train the recognition algorithm. However, experiments to collect these...

Is that My Heartbeat? Measuring and Understanding Modality-Dependent Cardiac Interoception in Virtual Reality

Measuring interoception ('perceiving internal bodily states') has diagnostic and wellbeing implications. Since heartbeats are distinct and frequent, various methods aim at measuring cardiac interoceptive accuracy (CIAcc). However, the role of exteroceptive modalities for representing heart rate (HR) across screen-based and Virtual Reality (VR...

Towards socialVR: Evaluating a novel technology for watching videos together

Social VR enables people to interact over distance with others in real-time. It allows remote people, typically represented as avatars, to communicate and perform activities together in a shared virtual environment, extending the capabilities of traditional social platforms like Facebook and Netflix. This paper explores the benefits and...

Designing Real-time, Continuous QoE Score Acquisition Techniques for HMD-based 360°VR Video Watching

Watching HMD-based 360° video has become in-creasing popular as a medium for immersive viewing of photo-realistic content. To evaluate subjective video quality, researchers typically prompt users to provide an overall Quality of Experience (QoE) score after viewing a stimulus. However, since users can adjust their viewport throughout a 360°...

Investigating Affective Responses toward In-Video Pedestrian Crossing Actions using Camera and Physiological Sensors

Automatically inferring drivers' emotions during driver-pedestrian interactions to improve road safety remains a challenge for designing in-vehicle, empathic interfaces. To that end, we carried out a lab-based study using a combination of camera and physiological sensors. We collected participants' (N=21) real-time, affective (emotion self...

Understanding and Designing Avatar Biosignal Visualizations for Social Virtual Reality Entertainment

Visualizing biosignals can be important for social Virtual Reality (VR), where avatar non-verbal cues are missing. While several biosignal representations exist, designing effective visualizations and understanding user perceptions within social VR entertainment remains unclear. We adopt a mixed-methods approach to design biosignals for...

Ethical Self-Disclosing Voice User Interfaces for Delivery of News

Voice User Interfaces (VUIs) such as Alexa and Google Home that use human-like design cues are an increasingly popular means for accessing news. Self-disclosure in particular may be used to build relationships of trust with users who may reveal intimate details about themselves. This information can be (mis)used by algorithms to tailor and...

Exploring Emotion Responses toward Pedestrian Crossing Actions for Designing In-vehicle Empathic Interfaces

While affective non-verbal communication between pedestrians and drivers has been shown to improve on-road safety and driving experiences, it remains a challenge to design driver assistance systems that can automatically capture these affective cues. In this early work, we identify users' emotional self-report responses towards commonly...

Few-shot Learning for Fine-grained Emotion Recognition using Physiological Signals

Fine-grained emotion recognition can model the temporal dynamics of emotions, which is more precise than predicting one emotion retrospectively for an activity (e.g., video clip watching). Previous works require large amounts of continuously annotated data to train an accurate recognition model, however experiments to collect such large...

SensiBlend: Sensing Blended Experiences in Professional and Social Contexts

Unlike traditional workshops, SensiBlend is a living experiment about the future of remote, hybrid, and blended experiences within professional and other social contexts. The interplay of interpersonal relationships with tools and spaces—digital and physical—has been abruptly challenged and fundamentally altered as a result of the COVID-19...

CEAP-360VR: A Continuous Physiological and Behavioral Emotion Annotation Dataset for 360 VR Videos

Watching 360 videos using Virtual Reality (VR) head-mounted displays (HMDs) provides interactive and immersive experiences, where videos can evoke different emotions. Existing emotion self-report techniques within VR however are either retrospective or interrupt the immersive experience. To address this, we introduce the Continuous...