JaneEye

Abstract

Eye tracking serves as a critical component across diverse areas of human-computer interaction (HCI) and healthcare applications, serving as a key enabler for intuitive and seamless user experiences. In HCI, eye tracking allows systems to detect users’ visual attention and gaze direction, facilitating natural interaction methods in augmented reality (AR), virtual reality (VR), and assistive technologies. In the healthcare domain, it is used for diagnosing and monitoring neurological disorders for patients. To be practical for deployment in wearable or embedded systems, eye tracking systems must deliver realtime performance while maintaining ultra-low power consumption. This requirement poses significant challenges in algorithm design and hardware implementation, especially when balancing accuracy, responsiveness, and energy efficiency. This thesis introduces JaneEye, an event-based eye tracking hardware accelerator optimized for ultralow latency and power consumption. The system is built around a lightweight, self-designed neural network architecture, in which a gated multilayer perceptron (MLP) and a novel convolutional Just Another Network (ConvJANET) layer play central roles in enabling efficient and accurate prediction. Despite having only 17.6K parameters, the proposed network maintains high prediction accuracy. Through software-hardware co-design strategies including quantization and nonlinear activation function approximation together with a new dataflow strategy that combines weight stationary and output stationary modes, the accelerator achieves remarkable performance. Based on post-layout simulation in GlobalFoundries 12LP-PLUS technolog, the JaneEye accelerator reaches an end-to-end latency of 0.5 ms (equivalent to 2000 FPS) while consuming just 18.9 μJ per frame operating at 400 MHz. To the best of our knowledge, this work demonstrates state-of-the-art trade-offs among accuracy, latency, and energy efficiency for event-based eye tracking, making it a strong candidate for integration into future AR/VR headsets and wearable healthcare devices.