Even though much research has been conducted on body worn sensors, little research has been performed on specialized (non-ear-worn) headworn sensors. These sensors could be used to perform a variety of health related research. Therefore, this thesis focusses on a new low-power specialized (non-ear-worn) headworn sensor platform and explores how it can be improved.

This research investigates the detection of gestures using a torso-worn accelerometer sensor. Using the Conflab dataset, we focus on gestures during conversations in mingling scenarios. Due to significant variability in gesture styles among individuals, traditional methods face challenges in building personalized models. Our experiments demonstrate that Transductive Parameter Transfer (TPT), an adaptive transfer learning method, can more effectively model these individual differences in gesturing. To gain insights into individual expressiveness, we classify gestures into three classes: 'no gesture,' 'normal,' and 'large' gestures. TPT performed an average AUC score of 0.84 in binary classification and 0.77 in multiclass classification. These findings highlight the potential of using a single torso-worn accelerometer to understand social behavior in naturalistic settings.

Multiactivity analysis investigates one's coordination of actions within a social context, such as gestures and speech, usually using video recordings of the social activity, to further understand the rules of human behaviour. This paper focuses specifically on the coordination between speaking and drinking activities within a social setting, and explores the possibility of automatically identifying these events using audio captured from a drinking glass. As social interactions occur in vastly different contexts, this paper also investigates the effect that background noise might have on the accuracy of identifying these events. Different parameters and audio features were compared. Linear classification models LR and SVM with a linear kernel were able to achieve 100% accuracy for all sample lengths between 2 and 8 seconds using the first 20 PCA components from 60 audio features. The best performing feature in identifying speaking and drinking events was MFCCs, achieving an F₁ score of 99.4% on average across models with a training sample length of 3 seconds. Background noise had different effects on classification accuracy depending on the type, with music lowering the F₁ score to 74.3%, noisy room audio to 64.7%, and podcast audio simulating the presence of other speakers to 59.6% using MFCCs and a 3-second sample length

Human activity recognition plays an interesting and important role nowadays as there are a variety of use cases. It is utilized in health monitoring, in the development of human-computer interaction system and in security monitoring. However current methods involve usage of privacy sensitive data and impractical sensors for everyday usage. To tackle this problem, we aim to answer the research question "How to maximize the capabilities of in-mouth sensors for human activity recognition?". The main contributions of this paper are the classification of different gestures using an in-mouth device, implementation of a classifier directly onto a microcontroller and the evaluation whether the models can generalize to multiple people. To investigate this, we experimented with popular classical machine learning classifiers: Decision Tree, K-Nearest Neighbors, Support Vector Machine, Logistic Regression and Random Forest classifiers. The results shows that the F1-score of all classification problems are above 80% using the various classifiers along with different parameters.

Tracking food intake provides a valuable source of information to gain insights in dietary habits for the health industry. Currently, the main method to track food intake to is do it manually. To take a step towards tracking food intake manually, this these aims to answer the following research question: ”How to detect chewing episodes with an IMU sensor around the ear?”. The IMU collect x, y, z axis data for the accerelometer an gyrscope. The data of the axis was down sampled to 20Hz and passed through a low-pass filter. Chewing detecting was done by determining chewing episode in time win- dows of 30 seconds. Feature were extracted from those time windows. Then random forest, decision tree, k-neareast neigbours, support vector machine and logistic regression machine learning models were used to classify the data, for which f1-scored higher than 0.8 have been achieved. Therefore it can be concluded is it possible to detect chewing episodes with a single IMU around the ear. Feature selection and performance analysis has been done, and it seem to be that features that use auto correlation and Fast Fourier Transform (FFT), play a significant role increasing the classification performance. They are computationally expensive and are not ideal for embedded system. However there is room for finding features that are less computationally expensive.

Analyzing food consumption patterns can provide valuable insights into the development of obesity and eating disorders. The detection and quantification of chewing strokes are essential to facilitate this analysis. One approach to food intake analysis involves evaluating chewing sounds generated during the eating process. These sounds were recorded by microphones placed to the user’s outer
ear canal. Aside from discovering the most accurate solution, the algorithms used must demonstrate sufficient efficiency to operate on low-power embedded ear-worn hardware. Three algorithms for automated chewing detection were evaluated with the help of two datasets. The first dataset consists of the food intake sounds from the consumption of three types of food. The second dataset consists of environmental noise. The data were manually labeled by recognizing mastication sounds’ visual and audio characteristics. Precision of over 80%
was achieved by all algorithms in the dataset consisting of only chewing sounds. Finally, an efficient solution has been developed to distinguish between speech and chewing sounds.

This research explores the correlation between chewing activities and non-chewing activities using an Arduino microcontroller. Chewing samples are recorded by attaching the microcontroller to the back of the jaw, underneath the ear. The microcontroller collects data from a microphone capturing audio data of chewing sounds, as well as an Inertial Measurement Unit (IMU) collecting motion and orientation data of jaw movements.

The collected data is processed using signal processing techniques, extracting relevant features from the microphone data, such as intensity, frequency content, and features related to acceleration, orientation, and jaw movement patterns from the IMU data. Statistical analysis, employing correlation metrics like Pearson correlation coefficient and Spearman's rank correlation coefficient, determines the correlation between the extracted features from the microphone and IMU data.

Conclusions from the analysis indicate that pre-processing and feature extraction techniques are needed to establish meaningful correlations between the IMU and microphone data. The sliding window approach shows promising results, particularly in correlating the sum of energy from the audio with the sum of gyro data, specifically in the y- and z-axes. The accelerometer data does not exhibit significant correlations, but it can be useful as a threshold for detecting the start of chewing events based on zero crossings.

Furthermore, the findings reveal that food texture and density play a larger role than anticipated in determining the correlation between chewing patterns and sensory data. The research outcomes contribute to various fields, including dentistry, nutrition, and human-computer interaction.