WEDAR

WEDAR: Webcam-based Attention Analysis via Attention Regulator Behavior Recognition with a Novel E-reading Dataset

Lee, Y. (TU Delft Web Information Systems)
Chen, H. (TU Delft Web Information Systems; University of Oulu)
Zhao, Guoying (University of Oulu)
Specht, M.M. (TU Delft Web Information Systems)

2022

Human attention is critical yet challenging cognitive process to measure due to its diverse definitions and non-standardized evaluation. In this work, we focus on the attention self-regulation of learners, which commonly occurs as an effort to regain focus, contrary to attention loss. We focus on easy-to-observe behavioral signs in the real-world setting to grasp learners' attention in e-reading. We collected a novel dataset of 30 learners, which provides clues of learners' attentional states through various metrics, such as learner behaviors, distraction self-reports, and questionnaires for knowledge gain. To achieve automatic attention regulator behavior recognition, we annotated 931,440 frames into six behavior categories every second in the short clip form, using attention self-regulation from the literature study as our labels. The preliminary Pearson correlation coefficient analysis indicates certain correlations between distraction self-reports and unimodal attention regulator behaviors. Baseline model training has been conducted to recognize the attention regulator behaviors by implementing classical neural networks to our WEDAR dataset, with the highest prediction result of 75.18% and 68.15% in subject-dependent and subject-independent settings, respectively. Furthermore, we present the baseline of using attention regulator behaviors to recognize the attentional states, showing a promising performance of 89.41% (leave-five-subject-out). Our work inspires the detection & feedback loop design for attentive e-reading, connecting multimodal interaction, learning analytics, and affective computing.

Attention regulator behaviors
Neural networks
WEDAR dataset

http://resolver.tudelft.nl/uuid:bad4a727-23fb-4852-b6f6-887dea234246

https://doi.org/10.1145/3536221.3556619

Association for Computing Machinery (ACM)

9781450393904

ICMI 2022 - Proceedings of the 2022 International Conference on Multimodal Interaction

24th ACM International Conference on Multimodal Interaction, ICMI 2022, 2022-11-07 → 2022-11-11, Bangalore, India

ACM International Conference Proceeding Series

Institutional Repository

conference paper

© 2022 Y. Lee, H. Chen, Guoying Zhao, M.M. Specht