Social interaction plays a key role in assessing teamwork and collaboration. It becomes particularly critical in team performance when coupled with isolated, confined, and extreme conditions such as undersea missions. This work investigates how social interactions of individual members in a small team evolve during the course of a long duration mission. We propose to use a topic model to mine individual social interaction patterns and examine how the dynamics of these patterns have an effect on self-assessment of mood and team cohesion. Specifically, we analyzed data from a 6-person crew wearing Sociometric badges over a 4-month mission. Our results show that our method can extract the latent structure of social contexts without supervision. We demonstrate how the extracted patterns based on probabilistic models can provide insights on common behaviors at various temporal resolutions and exhibit links with self-report affective states and team cohesion. ... Read more

Continuous monitoring with unobtrusive wearable social sensors is becoming a popular method to assess individual affect states and team effectiveness in human research. A large number of applications have demonstrated the effectiveness of applying wearable sensing in corporate settings; for example, in short periodic social events or in a university campus. However, little is known of how we can automatically detect individual affect and group cohesion for long duration missions. Predicting negative affect states and low cohesiveness is vital for team missions. Knowing team members' negative states allows timely interventions to enhance their effectiveness. This work investigates whether sensing social interactions and individual behaviors with wearable sensors can provide insights into assessing individual affect states and group cohesion. We analyzed wearable sensor data from a team of six crew members who were deployed on a four-month simulation of a space exploration mission at a remote location. Our work proposes to recognize team members' affect states and group cohesion as a binary classification problem using novel behavior features that represent dyadic interaction and individual activities. Our method aggregates features from individual members into group levels to predict team cohesion. Our results show that the behavior features extracted from the wearable social sensors provide useful information in assessing personal affect and team cohesion. Group task cohesion can be predicted with a high performance of over 0.8 AUC. Our work demonstrates that we can extract social interactions from sensor data to predict group cohesion in longitudinal missions. We found that quantifying behavior patterns including dyadic interactions and face-to-face communications are important in assessing team process. ... Read more

With the tremendous progress in sensing and IoT infrastructure, it is foreseeable that IoT systems will soon be available for commercial markets, such as in people's homes. In this paper, we present a deployment study using sensors attached to household objects to capture the resourcefulness of three individuals. The concept of resourcefulness highlights the ability of humans to repurpose objects spontaneously for a different use case than was initially intended. It is a crucial element for human health and wellbeing, which is of great interest for various aspects of HCI and design research. Traditionally, resourcefulness is captured through ethnographic practice. Ethnography can only provide sparse and often short duration observations of human experience, often relying on participants being aware of and remembering behaviours or thoughts they need to report on. Our hypothesis is that resourcefulness can also be captured through continuously monitoring objects being used in everyday life. We developed a system that can record object movement continuously and deployed them in homes of three elderly people for over two weeks. We explored the use of probabilistic topic models to analyze the collected data and identify common patterns. ... Read more

Gaze information provides indication of users focus which complements remote collaboration tasks, as distant users can see their partner’s focus. In this paper, we apply gaze for co-located collaboration, where users’ gaze locations are presented on the same display, to help collaboration between partners. We integrated various types of gaze indicators on the user interface of a collaborative search system, and we conducted two user studies to understand how gaze enhances coordination and communication between co-located users. Our results show that gaze indeed enhances co-located collaboration, but with a trade-off between visibility of gaze indicators and user distraction. Users acknowledged that seeing gaze indicators eases communication, because it let them be aware of their partner’s interests and attention. However, users can be reluctant to share their gaze information due to trust and privacy, as gaze potentially divulges their interests. ... Read more

In this paper we propose a novel method of estimating verbal expressions of task and social cohesion by quantifying the dynamic alignment of nonverbal behaviors in speech. As team cohesion has been linked to team effectiveness and productivity, automatically estimating team cohesion can be a useful tool for assessing meeting quality and broader team functioning. In total, more than 20 hours of business meetings (3-8 people) were recorded and annotated for behavioral indicators of group cohesion, distinguishing between social and task cohesion. We hypothesized that behaviors commonly referred to as mimicry can be indicative of verbal expressions of social and task cohesion. Where most prior work targets mimicry of dyads, we investigated the effectiveness of quantifying group-level phenomena. A dynamic approach was adopted in which both the cohesion expressions and the paralinguistic mimicry were quantified on small time windows. By extracting features solely related to the alignment of paralinguistic speech behavior, we found that 2-minute high and low social cohesive regions could be classified with a 0.71 Area under the ROC curve, performing on par with the state-of-the-art where turn-taking features were used. Estimating task cohesion was more challenging, obtaining an accuracy of 0.64 AUC, outperforming the state-of-the-art. Our results suggest that our proposed methodology is successful in quantifying group-level paralinguistic mimicry. As both the state-of-the-art turn-taking features and mimicry features performed worse on estimating task cohesion, we conclude that social cohesion is more openly expressed by nonverbal vocal behavior than task cohesion ... Read more

Mutual gaze is a powerful cue for communicating social attention
and intention. A plethora of studies have demonstrated the fundamental roles of
mutual gaze in establishing communicative links between humans, and enabling
non-verbal communication of social attention and intention. The amount of mutual gaze between two partners regulates human-human interaction and is a sign of social engagement. This paper investigates whether implementing mutual gaze in robotic systems can achieve social effects, thus to improve human robot interaction. Based on insights from existing human face-to-face interaction studies, we implemented an interactive mutual gaze model in an embodied agent, the social robot head Furhat. We evaluated the mutual gaze prototype with 24 participants in three applications. Our results show that our mutual gaze model improves social connectedness between robots and users. ... Read more