Title
EyeSyn: Psychology-inspired Eye Movement Synthesis for Gaze-based Activity Recognition
Author
Lan, G. (TU Delft Embedded Systems)
Scargill, Tim (Duke University)
Gorlatova, Maria (Duke University)
Contributor
O'Conner, L. (editor)
Date
2022
Abstract
Recent advances in eye tracking have given birth to a new genre of gaze-based context sensing applications, ranging from cognitive load estimation to emotion recognition. To achieve state-of-the-art recognition accuracy, a large-scale, labeled eye movement dataset is needed to train deep learning-based classifiers. However, due to the heterogeneity in human visual behavior, as well as the labor-intensive and privacy-compromising data collection process, datasets for gaze-based activity recognition are scarce and hard to collect. To alleviate the sparse gaze data problem, we present EyeSyn, a novel suite of psychology-inspired generative models that leverages only publicly available images and videos to synthesize a realistic and arbitrarily large eye movement dataset. Taking gaze-based museum activity recognition as a case study, our evaluation demonstrates that EyeSyn can not only replicate the distinct pat-terns in the actual gaze signals that are captured by an eye tracking device, but also simulate the signal diversity that results from dif-ferent measurement setups and subject heterogeneity. Moreover, in the few-shot learning scenario, EyeSyn can be readily incorpo-rated with either transfer learning or meta-learning to achieve 90% accuracy, without the need for a large-scale dataset for training.
Subject
Eye tracking
eye movement synthesis
activity recognition
To reference this document use:
http://resolver.tudelft.nl/uuid:fd3c74de-6b4a-46c6-a7c7-da1fff78ffb4
DOI
https://doi.org/10.1109/IPSN54338.2022.00026
Publisher
IEEE, Piscataway
Embargo date
2023-07-01
ISBN
978-1-6654-9625-4
Source
Proceedings of the 2022 21st ACM/IEEE International Conference on Information Processing in Sensor Networks (IPSN)
Event
2022 21st ACM/IEEE International Conference on Information Processing in Sensor Networks (IPSN), 2022-05-04 → 2022-05-06, Milano, Italy
Bibliographical note
Green Open Access added to TU Delft Institutional Repository 'You share, we take care!' - Taverne project https://www.openaccess.nl/en/you-share-we-take-care Otherwise as indicated in the copyright section: the publisher is the copyright holder of this work and the author uses the Dutch legislation to make this work public.
Part of collection
Institutional Repository
Document type
conference paper
Rights
© 2022 G. Lan, Tim Scargill, Maria Gorlatova