Touchless Hand Gesture-Based Digit Recognition

using Light-Sensors, Convolutional Neural Networks and a Microcontroller

Bachelor thesis (2023)

Authors

W. Smit Electrical Engineering, Mathematics and Computer Science

Contributors

Q. Wang Embedded Systems - EEMCS (supervisor 1)
M. Yang Embedded Systems - EEMCS (supervisor 1)
R. Zhu Embedded Systems - EEMCS (supervisor 1)
Ranga Rao Venkatesha Prasad Networked Systems - EEMCS (supervisor 2)
Faculty
Electrical Engineering, Mathematics and Computer Science
Copyright: © 2023 Winstijn Smit
Arduino Nano 33 BLE OPT101 Convolutional Neural Networks (CNNs) Air-Writing Digit Recognition Photodiodes Touchless Hand-Gestures
More Info
expand_more

Published Date

03-07-2023

Language

English

Reuse Rights

Other than for strictly personal use, it is not permitted to download, forward or distribute the text or part of it, without the consent of the author(s) and/or copyright holder(s), unless the work is under an open content license such as Creative Commons.

Faculty

Electrical Engineering, Mathematics and Computer Science

Abstract

Touchless interaction with computers has become more important in recent years, especially in the context of the COVID-19 pandemic.
Applications include situations where touch input is not possible or not desirable, e.g. for hygienic purposes in a public setting or a medical setting.
Practical examples for touchless interaction include elevators, vending machines, and other public devices that are used by many people.
However, most touchless interaction systems are expensive and require significant computational power.
This paper proposes a bare-bones low-power and low-cost system for recognizing air-written digits using a microcontroller and light sensitivity sensors.
A proof of concept has been created and tested in a fixed lighting scenario with a fixed set of gestures for the digits 0 to 9 to show the feasibility of such a system.
The system uses a convolutional neural network to recognize digits and achieves an average accuracy of 58,8% on a validation set of unseen participants.
It performs significantly better on new samples from users already seen during training, achieving an average accuracy of 93,5%.