Cross-modal approach for conversational well-being monitoring with multi-sensory earables

Conference paper (2018)

Authors

Chulhong Min Nokia Bell Labs
Alessandro Montanari Nokia Bell Labs
Akhil Mathur Nokia Bell Labs
Seungchul Lee Korea Advanced Institute of Science and Technology
Well-being Earable Multi-sensory
More Info
expand_more

Published Date

2018

Language

English

Reuse Rights

Other than for strictly personal use, it is not permitted to download, forward or distribute the text or part of it, without the consent of the author(s) and/or copyright holder(s), unless the work is under an open content license such as Creative Commons.

Abstract

We propose a cross-modal approach for conversational well-being monitoring with a multi-sensory earable. It consists of motion, audio, and BLE models on earables. Using the IMU sensor, the microphone, and BLE scanning, the models detect speaking activities, stress and emotion, and participants in the conversation, respectively. We discuss the feasibility in qualifying conversations with our purpose-built cross-modal model in an energy-efficient and privacy-preserving way. With the cross-modal model, we develop a mobile application that qualifies on-going conversations and provides personalised feedback on social well-being.