3ET: Efficient Event-based Eye Tracking using a Change-Based ConvLSTM Network

Conference paper (2023)

Authors

Qinyu Chen University of Zürich, ETH Zürich
Zuowen Wang ETH Zürich, University of Zürich
Shih Chii Liu University of Zürich, ETH Zürich
C. Gao Electronics - EEMCS
Research Group
Electronics (EEMCS) (TU Delft)
Copyright: © 2023 Qinyu Chen, Zuowen Wang, Shih Chii Liu, C. Gao
DOI
help_outline
https://doi.org/10.1109/BioCAS58349.2023.10389062
ConvLSTM Healthcare Sparsity Pupil tracking Event cameras AR/VR
More Info
expand_more

Published Date

2023

Language

English

Reuse Rights

Other than for strictly personal use, it is not permitted to download, forward or distribute the text or part of it, without the consent of the author(s) and/or copyright holder(s), unless the work is under an open content license such as Creative Commons.

Faculty

Electrical Engineering, Mathematics and Computer Science

Department

Microelectronics

Research Group

Electronics

Abstract

This paper presents a sparse Change-Based Convolutional Long Short-Term Memory (CB-ConvLSTM) model for event-based eye tracking, key for next-generation wearable healthcare technology such as AR/VR headsets. We leverage the benefits of retina-inspired event cameras, namely their low-latency response and sparse output event stream, over traditional frame-based cameras. Our CB-ConvLSTM architecture efficiently extracts spatio-temporal features for pupil tracking from the event stream, outperforming conventional CNN structures. Utilizing a delta-encoded recurrent path enhancing activation sparsity, CB-ConvLSTM reduces arithmetic operations by approximately 4.7× without losing accuracy when tested on a v2e-generated event dataset of labeled pupils. This increase in efficiency makes it ideal for real-time eye tracking in resource-constrained devices. The project code and dataset are openly available at https://github.com/qinche106/cb-convlstm-eyetracking.