Assessing learner’s distraction in a multimodal platform for sustained attention in the remote learning context using mobile devices sensors

Abstract

During a learning task, keeping a steady attentive state is detrimental for good performance. A person is subject to distraction from different sources, among which distractions originating from within him or herself or from external sources, such as ambient sound. The detection of such distraction can improve the effectiveness of a task by giving feedback when necessary. Existing researches tried to measure performance on specific activities with the use of mobile devices such as smartphones and smartwatches, and a study showed a correlation between changes of posture and distraction. This paper tackles a main question \say{How mobile devices sensors can indicate learner’s distractions in the remote learning context?}. The process to do so included the recording of raw data from the movement sensors from a smartphone and smartwatch during a reading task, which was processed to highlight movements and then used to train a Convolutional Long Short Term Memory (LSTM) model. The final produced result showed a F1 score of 0.919 on validation data and was also combined with an external model to detect distraction from ambient noise to create a multimodal model, which showed better performance than both models individually. The limitations of the data collected during the experiment and improvements for future work are also discussed.