Embodied Conversational Agent for Mental Health Intervention

Abstract

Embodied Conversational Agents (ECA) seek to provide a more natural means of interaction for a user through verbal and non-verbal properties of human face-to-face communication. For this reason, these systems are found to bring benefits in different mental health related interventions. However, a key challenge in developing agents to replace the human interlocutor in a dyadic conversation, is to simulate appropriate attentive listening behaviors. In this thesis work, we explored different backchannel strategies and studied their effects in terms of likability and engagement. We built a fully embodied conversational agent with three different levels of backchannel strategies and ran a within-subject study with a convenience sample of 24 participants. The results showed that the amount of emotional words in the speech of users increased if the attentive listening capabilities of the agent were improved. In addition, the capability to trigger both verbal and nonverbal backchannels with proper timing was found to be a relevant feature in terms of improved speech rate and emotional words. Contrary to our hypothesis, backchannels based on actual emotion and sentiment analysis of the speech content were not found to be significantly influential on the quality of interaction. Multi-modal approaches are suggested for future works in order to overcome limitations of this work due to potential lack in emotion detection accuracy.