Touchless Hand Gesture-Based Digit Recognition
using Light-Sensors, Convolutional Neural Networks and a Microcontroller
W. Smit (TU Delft - Electrical Engineering, Mathematics and Computer Science)
Q. Wang – Mentor (TU Delft - Embedded Systems)
M. Yang – Mentor (TU Delft - Embedded Systems)
R. Zhu – Mentor (TU Delft - Embedded Systems)
R. Venkatesha Venkatesha Prasad – Graduation committee member (TU Delft - Networked Systems)
More Info
expand_more
Other than for strictly personal use, it is not permitted to download, forward or distribute the text or part of it, without the consent of the author(s) and/or copyright holder(s), unless the work is under an open content license such as Creative Commons.
Abstract
Touchless interaction with computers has become more important in recent years, especially in the context of the COVID-19 pandemic.
Applications include situations where touch input is not possible or not desirable, e.g. for hygienic purposes in a public setting or a medical setting.
Practical examples for touchless interaction include elevators, vending machines, and other public devices that are used by many people.
However, most touchless interaction systems are expensive and require significant computational power.
This paper proposes a bare-bones low-power and low-cost system for recognizing air-written digits using a microcontroller and light sensitivity sensors.
A proof of concept has been created and tested in a fixed lighting scenario with a fixed set of gestures for the digits 0 to 9 to show the feasibility of such a system.
The system uses a convolutional neural network to recognize digits and achieves an average accuracy of 58,8% on a validation set of unseen participants.
It performs significantly better on new samples from users already seen during training, achieving an average accuracy of 93,5%.