Automatic Detection of Mind Wandering Using Residual Network Generated Features

Automatic Detection of Mind Wandering Using Residual Network Generated Features

Demi, Arbër (TU Delft Electrical Engineering, Mathematics and Computer Science)

Zhang, X. (mentor)
Dudzik, B.J.W. (mentor)
Hung, H.S. (mentor)
Murukannaiah, P.K. (graduation committee)

Delft University of Technology

Computer Science and Engineering

CSE3000 Research Project

2022-06-22

Mind wandering occurs when a person’s attention unintentionally shifts away from their current thought or task. Being able to automatically detect cases of mind wandering can assist applications with attention retention, and help people with maintaining focus. Many methods have been tested to deal with mind-wandering detection, but they are mainly conducted in controlled environments.
There also has been little study into the usefulness of learned features from neural networks. This paper is focused on showcasing the effectiveness of
using neural network-generated features as input for classification models. Specifically, using ResNet to generate features which are then used as input by supervised learning models for classification. These features and models were used to classify mind wandering in the Mementos data set, outside of a controlled environment or differently put as “In-the-wild“. The study shows that the extracted features could not be used to accurately detect mind wandering based on the F1-Score (Macro) measure. The results can be attributed to data imbalance, low amount of data, lack of dataset-tailored pre-processing operations, and indiscriminate features. To improve on the study, more data collection is advised and the usage of methods like re-sampling and data augmentation to deal with data imbalance. And lastly, experimentation with neural network training and transforming the data into a time series format to better represent the temporal information from the data.

Mind wandering
ResNet
SVM
Computer vision
Temporal

http://resolver.tudelft.nl/uuid:c4f4b721-2baa-4018-ac0f-098ea8ba7ed7

Student theses

bachelor thesis

© 2022 Arbër Demi