We present a low-cost, camera-based tactile sensor that leverages the photoelastic effect—interference fringes that appear under stress—to estimate contact force, position, and shape. Each fringe image is recorded at 50 Hz and processed by a multi-task neural network that predicts (i) the normal force (Fz ), (ii) the 2D contact location (x, y), and (iii) the shape class of the object. Two sensor variants were developed: Sensor 1, a layered design with fewer visible fringes, and Sensor 2, an integrated structure with improved fringe clarity. Both were evaluated using a ResNet-18 and a lightweight custom CNN, under three augmentation pipelines: grayscale images with 10 noisy augmented samples each, RGB images with 3 noisy augmentations, and RGB images with 3 clean (noise-free) augmentations. The base dataset includes nearly 15,000 synchronised samples of high-frequency fringe images and force signals. With augmentation, this was expanded to around 45,000 or 150,000 samples depending on the pipeline. The best results were achieved using Sensor 1 and ResNet-18 trained on grayscale images with 10 augmentations per input image. This configuration yielded a force MSE of 0.0213 N2, a contactpoint RMSE of 0.4462 mm, and 96.24% shape classification accuracy. Notably, even RGB images with only three augmentations per sample reached similar performance levels. These findings highlight that full-colour input and lightweight augmentation remain effective for accurate, scalable tactile sensing. Our modular learning pipeline generalises across sensor variants and data regimes, enabling robust, highfrequency tactile inference suitable for real-world deployment.

This research addresses the challenge of deploying real-time drinking gesture detection in messy, "in-the-wild" environments. We propose and evaluate two computationally inexpensive systems, one using a Random Forest classifier, another using a 1-Dimensional Convolutional Neural Net (1D-CNN) classifier. Both are trained on 2D skeleton data, or features derived from that solely that data. Tested on the Conflab social interaction dataset, our method is designed to handle sparse labels and significant data occlusion. This study reports on the performance of this light-weight, video-based approach, providing a benchmark for applicability in real-world health and human-computer-interaction applications where privacy and computational efficiency are important factors. Although we were unable to create a robust and reliable classifier (f1 of 0.07 and 0.03 respectively), this work shows that there is potential for future work to succeed (roc-auc’s of 0.63 and 0.55 respectively) and provides critical insights into pitfalls to avoid when designing similar systems.

We propose a method for detecting laughter in spontaneous social interactions using chest-worn accelerometers. Our approach compares three segmentation strategies—padded, centered, different sliding win- dowssizesandevaluatesannotationmodalities: No Audio, Only Audio, and With Audio. Using time-domain features and Random Forests, we reach up to 0.962 macro F1-score. Longer windows and multimodal annotations improve performance and generalizability. Key features include axis-wise means, deriva- tives, and inter-axis correlations. These results support the potential of motion-based laughter detection in privacy-sensitive environments, while highlighting the importance of segment and label design.

This study investigates the feasibility of detecting drinking behavior in social environments using chest-mounted accelerometer data. A dataset collected during a conference is used, consisting of accelerometer data and annotated video recordings of 48 participants. After preprocessing, a random forest classifier is trained using statistical features: mean, correlation, maximum, minimum, and covariance derived from the yand z-axes of acceleration data. Evaluation through sixfold cross-validation for one participant yields an accuracy of 79%, while a leave-one-out cross-validation across participants achieves 70% accuracy. Qualitative analysis of false predictions reveals that actions like nodding, walking while drinking, or movement of the drinking hand towards the face can mimic drinking behavior. These findings demonstrate that accelerometer data contains detectable signals of drinking behavior even in noisy real-world conditions. However, further improvements require more diverse training data, consistent annotation, and possibly the inclusion of additional movement categories. The results support the potential of wearable accelerometers for drinking monitoring in social settings.

Consumer-grade fitness trackers can produce unreliable physiological data due to sensor errors. The same holds for cycling data from Wahoo Fitness, where heart rate (HR) and power readings are essential for training and performance analysis. This thesis presents a prediction-based anomaly detection framework tailored to multivariate time-series cycling data. The approach reframes anomaly detection as a personalized physiological HR prediction problem. We define anomalies as deviations between measured sensor values and their predicted values, based on contextual activity metrics (e.g., power, cadence, speed, altitude, and gradient) and user-specific embeddings. The system combines ordinary differential equations (ODEs) modeling heart rate dynamics with machine learning techniques to capture non-linear, non-stationary, and individualized relationships. The model not only detects implausible values but reconstructs them with physiologically consistent alternatives. Compared to reconstruction-based methods, which are mostly used for anomaly detection in time series data, this physiologically grounded approach better differentiates between normal variation and true anomalies. Experimental results demonstrate effective identification and correction of HR and power anomalies, contributing to improved data quality and reliability in wearable fitness applications.

This research paper proposes a deep learning model to infer segments of speaking intentions using body language captured by a body-worn accelerometer. The objective of the study is to detect instances where individuals exhibit a desire to speak based on their body language cues. The labeling scheme employed is a binary string, with “0” indicating no intention to speak and “1” indicating the presence of an intention to speak on a defined window size of 40 (corresponding to a 2 seconds segment recorded at a frequency of 100 Hz scaled down to 20 binary points per second). In this experiment, a real-life social event dataset was employed, and intensions to speak were manually annotated. A 10-minute segment from the dataset was selected and annotated using the ELAN software. The annotations included two categories: realized intentions, where individuals intended to speak and actually did so, and unrealized intentions, where individuals displayed intentions to speak but did not take their turn. The dataset consisted of 255 segments with realized intentions and 31 segments with unrealized intentions. Additionally, 255 negative samples were included, representing instances where no intention to speak was observed throughout the entire segment. To address the class imbalance inherent in the dataset, the model was evaluated using the Area Under the ROC Curve (AUC) metric using 5- fold cross validation. The model was tested on realized intentions, unrealized intentions, and a combination of both. Its performance was compared against a baseline model, which is assumed to always predict the start of the intention in the middle of the segment. Also, a new model was built that enables classification using varying window sizes as input. The classification task is performed to provide a comparison to another study [1] in which the training segments were predetermined window sizes instead of precisely annotated segments. The results of the study indicate that the deep learning models perform consistently better than the baseline on the segmentation task, and surpasses the performance of the model trained exclusively on window sizes on the classification task. This not only demonstrates the potential of body language as an informative cue for inferring speaking intentions, but also suggests that a supervised learning where intentions are identified with greater precision can lead to a superior outcome.

In this thesis the automatic multimodal detection of social and task cohesion in meetings is studied. The presence of social and task cohesion has positive benefits on employee well-being, creativity and productiveness, and can therefore be used to assess meeting quality. Conversational partners imitate each other's body language and speech characteristics to have smoother interactions, to increase liking, and it can cause a coordination of expectations. We hypothesize that social and task cohesion are therefore positively related to the imitation of both body language and speech characteristics. As the group-level alignment of non-verbal speech behaviour has been previously linked to social and task cohesion in meetings, this thesis investigates the relationship between cohesion in meetings and both motion and posture mimicry from accelerometer and video data. Motion mimicry is described using accelerometer features previously used for the detection of friendly and romantic attraction in pairs. We propose a method to convert these features to group-level descriptors of mimicry. The same quantifications of mimicry are also applied to video-based motion quantifiers that have been previously used to detect team cohesion. Appearance is described using HOG descriptors from densely-sampled feature points that are tracked over time. To our knowledge this is the first-time appearance similarity is used to estimate cohesion. We also test if a multimodal mimicry model performs better than a unimodal mimicry model to investigate if the different forms of mimicry contain complementary information. Our group-level movement mimicry features detect social cohesion with average an area under the ROC-curve of 0.64 for social cohesion, and the accelerometer-based movement mimicry features specifically detect task cohesion with an average AUC of 0.63. The multimodal combination of the different features does improve over the unimodal models with an area under the ROC-curve of 0.68 for social cohesion and 0.65 for task cohesion. This shows both that movement mimicry is an indicator of verbal expressions of cohesion, and that measuring mimicry in different modalities can better model cohesion than a unimodal model. Further experiments are recommended for general appearance feature mimicry, specifically in the method used to measure mimicry, to confirm or deny if these are related to cohesion.