Meetings are a vital part of discussions and negotiations. Unfortunately, individuals often leave with a vague understanding of the topics covered during the meeting and tend to forget even more of what transpired as time goes on. Driven by previous research that attempts to solve the issue by using architectural shapes as a way of removing ambiguity along with recent advancements in Automatic Speech Recognition (ASR) and Natural Language Processing (NLP) this research attempts to improve user understanding of key topics discussed in meetings by combining ASR models with NLP tools to create a visual summary that would improve user understanding of key topics covered during meetings. To achieve this the research utilizes the speech-to-text transcription and speaker identification capabilities of the WhisperX model with noun phrase extraction features provided by Spacy and key topic recognition functionality of Microsoft's DeBERTa model. Finally, the data is presented as a node-based graph utilizing the D3.js library. The results show that the system is able to identify between 33% - 58% of meeting key topics. This shows the potential of combining ASR models with NLP tools for creating concise meeting summaries but also raises new questions such as why some topics were missed, how the system performance can be improved, and how to design an optimal user interface for such a task.

Within the field of negotiations, a recent publication is a paper called the NegotiAct[9], which analyzed existing coding schemes of negotiations and introduced an improvement on them that promises a viable way to analyze negotiations in depth. In this research, my goal is to develop a workflow for gathering information and analyzing it with the NegotiAct. To this end the Colored Trails Game [7] is used to design an experiment that simulates real-life negotiations. The Colored Trails Game is a game where each player, through negotiating tries to maximize their own score under limited resources. The game at its core offers the opportunity for both cooperation and competitiveness.
In the experiment a total of 15 participants took part and it was run a total of 20 times, resulting in 3 hours and 10 minutes of recordings. Encoding them with the NegotiAct resulted in the discerning of a total of 87 offers made, 24 offers accepted, 26 offers rejected, and 16 requests for offer modifications. Based on this coherent mapping it can be concluded that the Colored Trails Game is a suitable choice for the workflow of gathering data to be put through the analysis of the NegotiAct.

Meetings represent a key component of collabora- tion in the workplace, serving purposes like brain- storming, discussion, and negotiation. Despite their importance, reaching a consensus among partici- pants can frequently be difficult because different people can leave the debate with different perspec- tives. In order to promote efficient communication and decision-making in organisational contexts, the use of the Shape Language is proposed. The Shape Language consists of shapes that people can use in meetings in order to represent abstract ideas, that would be difficult to represent by only words. In order to track how people interact with these ob- jects, computer vision tools can be used. This study aims to explore the current existing computer vi- sion tools for segmenting and classifying objects in meetings, aiming to find limitations in how well these models are able to recognize objects in the context of meetings and negotiations. Results of this study show that after fine-tuning four models on the custom dataset, they can recognize the three shapes provided as classes in most of the cases, but still make mistakes when assigning classes, or miss objects that they should classify all together, which show limitations of these modern tools.

Human activity recognition plays an interesting and important role nowadays as there are a variety of use cases. It is utilized in health monitoring, in the development of human-computer interaction system and in security monitoring. However current methods involve usage of privacy sensitive data and impractical sensors for everyday usage. To tackle this problem, we aim to answer the research question "How to maximize the capabilities of in-mouth sensors for human activity recognition?". The main contributions of this paper are the classification of different gestures using an in-mouth device, implementation of a classifier directly onto a microcontroller and the evaluation whether the models can generalize to multiple people. To investigate this, we experimented with popular classical machine learning classifiers: Decision Tree, K-Nearest Neighbors, Support Vector Machine, Logistic Regression and Random Forest classifiers. The results shows that the F1-score of all classification problems are above 80% using the various classifiers along with different parameters.

Multiactivity analysis investigates one's coordination of actions within a social context, such as gestures and speech, usually using video recordings of the social activity, to further understand the rules of human behaviour. This paper focuses specifically on the coordination between speaking and drinking activities within a social setting, and explores the possibility of automatically identifying these events using audio captured from a drinking glass. As social interactions occur in vastly different contexts, this paper also investigates the effect that background noise might have on the accuracy of identifying these events. Different parameters and audio features were compared. Linear classification models LR and SVM with a linear kernel were able to achieve 100% accuracy for all sample lengths between 2 and 8 seconds using the first 20 PCA components from 60 audio features. The best performing feature in identifying speaking and drinking events was MFCCs, achieving an F₁ score of 99.4% on average across models with a training sample length of 3 seconds. Background noise had different effects on classification accuracy depending on the type, with music lowering the F₁ score to 74.3%, noisy room audio to 64.7%, and podcast audio simulating the presence of other speakers to 59.6% using MFCCs and a 3-second sample length

This research investigates the detection of gestures using a torso-worn accelerometer sensor. Using the Conflab dataset, we focus on gestures during conversations in mingling scenarios. Due to significant variability in gesture styles among individuals, traditional methods face challenges in building personalized models. Our experiments demonstrate that Transductive Parameter Transfer (TPT), an adaptive transfer learning method, can more effectively model these individual differences in gesturing. To gain insights into individual expressiveness, we classify gestures into three classes: 'no gesture,' 'normal,' and 'large' gestures. TPT performed an average AUC score of 0.84 in binary classification and 0.77 in multiclass classification. These findings highlight the potential of using a single torso-worn accelerometer to understand social behavior in naturalistic settings.

This paper presents a smart drinking cup proto- type platform for social sensing studies. Recording the dynamics of unscripted human interactions in social settings can be challenging and often requires the participants to wear external hardware. This leads to greater participant discomfort and behavioural changes. To address this, we propose a prototype for a smart sensing cup: a multi-sensor data collection device integrated into a drinking cup. A hydrostatic pressure sensor is used for the volume and temperature measurements of the con- tents of the cup. Orientation data is gathered with a three-axis accelerometer, measuring the pitch of the cup. Detection of whether the cup is being held by a participant is ensured using a capacitance sensor. The sampling rate for all sensors is 10 Hz and can store up to an hour of data on the device. Using the sensor data we are able to detect whether a cup is being held, identify drinking events and fluid intake and give live feedback on this using an LED.

Understanding children's social interaction patterns is critical for their cognitive development; however, existing psychological studies often focus on dyadic interactions, overlooking the complexities of group dynamics. This study extends the concept of homophily—the tendency for individuals to interact with similar others—by exploring its implications within group settings. We introduce F-formations, a well-adopted notion from the computational field for detecting interaction groups in adults, which has yet to be extensively studied in children. By identifying this research gap, we aim to investigate how F-formations can be applied to children's social interactions.
Our case study was conducted in a preschool setting, specifically in the Starfish classroom, which includes children with hearing loss. Several key findings emerged from this study. First, we observed that F-formations can be detected in children's interactions, though some formations violate the definition of F-formations. Second, while the homophily effect was not evident in dyadic studies concerning children's hearing conditions, it was observed in group settings. We found that children prefer to form more homogeneous and smaller groups, with those who have hearing loss spending more time with peers who share similar characteristics. Overall, our results suggest that integrating group interaction data into traditional dyadic analyses provides valuable insights into children's social behavior, highlighting the importance of studying group dynamics alongside dyadic interactions.

The Socially Perceptive Computing Lab (SPCL) at Delft University of Technology has developed a device called the Midge. The aim of this device is to record data of social interactions at conferences. This paper aims to characterise how battery life is affected by different sensor settings on the Midge as well as how much data is generated in a given period of time. Based on the findings from the performed tests, formulae for both run time and data generation were devised.

The Midge is a wearable badge created by the Socially Perceptive Computing Lab, Pattern Recognition and Bioinformatics group of the Delft University of Technology, with as goal to analyse human behaviour. The badge has a digital motion processor (DMP) that can determine its orientation. This DMP makes use of an inertial measurement unit (IMU), that houses an accelerometer, a gyroscope and a magnetometer, to calculate its movement in a 3D-space. For both of the accelerometer and gyroscope the Full Scale Range (FSR) can be changed, in addition to the frequency. In this paper, the effects of both elements are analysed to determine if they influence the accuracy of the data gathered. The results show that the changing the FSR does not influence the accuracy of neither the two sensors nor the performance of the DMP. On the other hand, it was found that changing the frequency does influence the performance of the Midge. Even though the frequency did not affect the measurements of the accelerometer and gyroscope directly, the performance of the DMP was affected. The DMP performed best with a frequency of 150 Hz. Using a higher frequency also captured local extremes and turning points from the sensors and the interpreted orientation more precisely.

In our daily life people encounter many social interactions, for example in the supermarket, at work and in schools. Currently the most reliable way to find social interactions in groups, is to manually annotate the data. Manual annotation takes a lot of time and human resources and as the information stream goes faster and faster, the manual labour cannot keep up. Therefore an automated program to replace this is desirable. One method to automate this, is by looking at the proximity of people, which has been done by Dikker [1]. The results look promising, however they are sensitive to errors. The F1-score (equation 3) is only 0.625 and the precision (equation 5) is 0.696. This means that there are still a lot of false positives. With orientation data one can determine if the people in question are facing each other before they are assigned to be in social interaction. With a view frustum (cone shape) it can be determined who are facing at each other by checking if the cones overlap. This cone can take different sizes, making the view area bigger or smaller. Because the absolute rotation were only used, it was used to find out who cannot face each other. Using the proximity results as a baseline, the impossible orientations were removed. With the goal to reduce the amount of false positives. From the results it can be concluded that a small cone will perform worse compared to the baseline. The bigger the angle of the cone becomes, the closer it gets to the results from Dikker [1]. However, in none of the results it turns out that using absolute orientations improves the current work significantly.

Detecting social interactions through wireless wearable Bluetooth devices is increasing in popularity. Devices use the signal strength to other detected devices to estimate the proximity between people and group them together based on the Dominant set algorithm. Dominant sets are a maximal clique of nodes with an edge-weight based on the affinity between the nodes. Nevertheless, the signal is heavily influenced by external factors, which increase in an crowded environment. This paper introduces three different noise reduction filters that try to detect the kind of noise and therefore improve the detection of surrounded devices. Further, this paper looks at the overall impact of proximity on the resulting RSSI values. Knowing this relationship helps to normalize the values and therefore eliminates the need to apply noise reduction. Using a dataset of 48 sensors recorded in a conference setting with a specific designed sensor the low frequency pass filter gets an accuracy score of 81.8 % with a cut-off frequency of 0.07 Hz. It performs best when considering a conversation window of 20 seconds. Here, only 2/3 of the detected groups has to coincide with the actual formed group at a specific timestamp. Furthermore, the orientation of the participants to each other has heavy influence on the resulting RSSI values and therefore a normalization based on only proximity cannot be done.

The goal for this paper is to find out what the smart badge provided by the Social Perceptive Computive Lab (SPCL) group is and what it contains. The sensors that are used in the smart badge are the Accelerometer, Gyroscope and Magnetometer. The main question of this paper is ”What is the benefit of using full 9-DOF IMU data in predicting speaking status, as opposed to using only accelerometer signals?”. The three senors all contribute in their own way and complement each other to give an estimate about the speaking status. The ability to estimate the speaking status using the smart badge opens up the potential for analyzing more about the social aspects of people without the need to record what they are saying.

The Midge is a sensor device developed by the Socially Perceptive Computing Lab (SPCL) at Delft University of Technology (TU Delft). This device is used to monitor human behaviour in social settings using several sensors. In this paper, the accuracy of the Inertial Measurement Unit (IMU) chip used in the Midge sensor package was evaluated. The IMU is responsible for sensing motion in multiple directions using an accelerometer, gyroscope and magnetometer. This experiment was performed by comparing several Midge devices with each other as well as comparing them with a modern smartphone. First will be explained how the control software was updated to run on modern hardware (computers) and the work that was done to reliably convert the generated binary data to readable data (parsing). Then the process of creating the test setup, performing the tests and analyzing the data will be explained.

This thesis has researched how the Midge compares to a modern mobile phone regarding the accuracy and reliability of the rotation vector from the DMP in both devices. The rotation of the main axis of the Midge accurately matches that of the modern mobile phone, which means that the accuracy and reliability of the rotation vector from the DMP of both devices are mostly similar. The rotation of the secondary axes of the Midges do not exactly match that of the mobile phone. Additionally, the rotation of the secondary axes differs for each Midge. These differences might suggest that the DMP of the Midges are less accurate at detecting small changes. Further research is required to draw a definite conclusion.

When analysing social interactions, manual labour is often required to identify what is happening. An automated method of detecting who is interacting with who would already prove to be a significant help. This paper looks at how automated interaction detecting can be established. We look at methods of detecting proximity and look deeper into detecting F-formations using proximity. An f-formation is a group of people who are standing together with the intention of conversing. We show that it is possible to detect f-formations using a data-set containing proximity information and f-formations as ground truths. Our results show that using only proximity data from this dataset; we can detect f-formations better than the baseline provided in that dataset.