Thermotion: an exploration of facilitating emotion perception with wearable thermal displays

Abstract

The project aims at exploring tangible ways to communicate emotion. To narrow down the scope, I lay the focus on emotion communication in remote settings, a context when nonverbal communication is usually limited. From interviews, the neutral expression is found to be the main factor hindering a sufficient understanding of others' emotional state. Therefore, the scope is narrowed down to a topic related to the real-world problem: Contributing to the emotion interpretation from the neutral verbal expression.
While verbal communication cannot convey emotion sufficiently, tangible modalities like haptics, thermal feedback are promising extra channels to explore. Communicating affection with the physical stimulus is not only supported by theoretical studies like embodying emotion theory but also practiced through prototyping and experimenting from HCI filed. Within this project, thermal stimulus, a modality highly related to emotional experience without causing privacy issue, is selected for the study. Thus, the research can be further framed as: How thermal stimulus contribute to emotion interpretation from the neutral verbal expression? Considering the application perspective and the requirement about wearables from CWI, the thermal stimulus should be designed as wearables. The research is conducted in the path of the controlled experiment to generate solid findings. Voice message, a medium less explored in the area of remote communication, is chosen as the case for study. Therefore, the final research question is formulated as follows: How can wearable thermal display contribute to emotion interpretation from the neutrally spoken voice messages?
The project follows the path of research through design. First, through literature review and quick user test, a pre-study leads to some initial decisions for the experiment: 1) using the upper chest as the body location. 2) using woven silk as the contact medium. 3) setting four thermal stimuli (high intensity warm 38°C, low intensity warm 35°C, high intensity cold 26°C, low intensity cold 29°C).
Afterward, the first experiment is conducted to find the recognizable and acceptable on-body thermal stimulus with a suitable contact medium (skin or fabric) for the second experiment. The study with 12 participants collects the ratings in terms of subjective intensity and comfort. The findings indicate that low intensities cannot be used if the thermal stimuli should be detectable for most people. For the cool stimuli, the intensity is decreased from -6°C to -4°C to ensure participant comfort. Besides, the fabric is not used for the second experiment because of its delay effect on thermal sensation.
Between the two experiments, a study is conducted to validate that the neutral tone can make a normal voice message perceived more neutral. Twelve neutrally spoken voice stimuli are generated by an AI voice generator and compared with the original normally spoken voices (from a validated database) in terms of valence and arousal. The result shows that the neutral expression does make a sentence with either positive or negative content perceived more neutral.
The second experiment is to explore how thermal stimulus can affect emotion perception from nuetrally spoken voice messages. Participants listen to eight voice messages and experience the thermal stimuli at the same time. Afterward, they give their ratings in terms of valence and arousal from the voice messages. Each voice message is paired with three thermal conditions (warm, cold, baseline). The result shows that warm stimulus augments the perceived positiveness of positive messages while cold stimulus augments the negativeness of negative messages.
However, augmenting the emotion perception doesn't necessarily mean the accuracy of emotion perception. The contribution and limitation of the project are discussed in the end.