Automatic multimodal detection of team cohesion in meetings

Abstract

In this thesis the automatic multimodal detection of social and task cohesion in meetings is studied. The presence of social and task cohesion has positive benefits on employee well-being, creativity and productiveness, and can therefore be used to assess meeting quality. Conversational partners imitate each other's body language and speech characteristics to have smoother interactions, to increase liking, and it can cause a coordination of expectations. We hypothesize that social and task cohesion are therefore positively related to the imitation of both body language and speech characteristics. As the group-level alignment of non-verbal speech behaviour has been previously linked to social and task cohesion in meetings, this thesis investigates the relationship between cohesion in meetings and both motion and posture mimicry from accelerometer and video data. Motion mimicry is described using accelerometer features previously used for the detection of friendly and romantic attraction in pairs. We propose a method to convert these features to group-level descriptors of mimicry. The same quantifications of mimicry are also applied to video-based motion quantifiers that have been previously used to detect team cohesion. Appearance is described using HOG descriptors from densely-sampled feature points that are tracked over time. To our knowledge this is the first-time appearance similarity is used to estimate cohesion. We also test if a multimodal mimicry model performs better than a unimodal mimicry model to investigate if the different forms of mimicry contain complementary information. Our group-level movement mimicry features detect social cohesion with average an area under the ROC-curve of 0.64 for social cohesion, and the accelerometer-based movement mimicry features specifically detect task cohesion with an average AUC of 0.63. The multimodal combination of the different features does improve over the unimodal models with an area under the ROC-curve of 0.68 for social cohesion and 0.65 for task cohesion. This shows both that movement mimicry is an indicator of verbal expressions of cohesion, and that measuring mimicry in different modalities can better model cohesion than a unimodal model. Further experiments are recommended for general appearance feature mimicry, specifically in the method used to measure mimicry, to confirm or deny if these are related to cohesion.