Hand Gesture Recognition on Arduino Using Recurrent Neural Networks and Ambient Light

Bachelor Thesis (2022)
Author(s)

M.S. Lipski (TU Delft - Electrical Engineering, Mathematics and Computer Science)

Contributor(s)

Mingkun Yang – Mentor (TU Delft - Embedded Systems)

R. Zhu – Mentor (TU Delft - Embedded Systems)

Q. Wang – Mentor (TU Delft - Embedded Systems)

Christoph Lofi – Graduation committee member (TU Delft - Web Information Systems)

Faculty
Electrical Engineering, Mathematics and Computer Science, Electrical Engineering, Mathematics and Computer Science
Copyright
© 2022 Matthew Lipski
More Info
expand_more
Publication Year
2022
Language
English
Copyright
© 2022 Matthew Lipski
Graduation Date
22-06-2022
Awarding Institution
Delft University of Technology
Project
CSE3000 Research Project
Programme
Computer Science and Engineering
Faculty
Electrical Engineering, Mathematics and Computer Science, Electrical Engineering, Mathematics and Computer Science
Reuse Rights

Other than for strictly personal use, it is not permitted to download, forward or distribute the text or part of it, without the consent of the author(s) and/or copyright holder(s), unless the work is under an open content license such as Creative Commons.

Abstract

Touching physical buttons to interact with public electronic devices has raised some concerns regrading disease transmission following the COVID-19 pandemic. The use of hand gestures as a touchless replacement sounds appealing, but comes with the challenge of recognizing which gesture is being performed by the user, with only the processing power of a microcontroller. This paper explores the use of recurrent neural networks (RNNs) and their derivatives to recognize hand gestures on an Arduino Nano 33 BLE. The neural networks receive input from 3 OPT101 photodiodes, which emit a voltage that increases with the intensity of light that hits them, meaning they can effectively track hand shadows cast by the user’s hand under ambient light. After testing various RNN-based neural network architectures, CNN-LSTMs produced the highest validation accuracy. However, due to issues with the testing setup, the highest validation accuracy measured for a CNN-LSTM was only 43%, indicating that further work is required.

Files

License info not available