WaveTune: Millimeter Wave Radar based Hand Gesture Recognition for Musical Applications

Master thesis (2023)

Authors

S.S. Juneja Electrical Engineering, Mathematics and Computer Science

Contributors

Marco Zuniga Networked Systems - EEMCS (supervisor 1)
Q. Wang Embedded Systems - EEMCS (supervisor 2)
G. Vaidya Networked Systems - EEMCS (supervisor 2)
Faculty
Electrical Engineering, Mathematics and Computer Science
Copyright: © 2023 Suchdeep Singh Juneja
More Info
expand_more

Published Date

30-08-2023

Language

English

Reuse Rights

Other than for strictly personal use, it is not permitted to download, forward or distribute the text or part of it, without the consent of the author(s) and/or copyright holder(s), unless the work is under an open content license such as Creative Commons.

Faculty

Electrical Engineering, Mathematics and Computer Science

Abstract

In the ever-evolving field of music technology, new solutions continue to emerge that enhance musical expression and creativity. This thesis introduces WaveTune, a novel lightweight hand gesture recognition system that enables real-time control of musical composition and performance through natural hand motions.

WaveTune utilizes Millimeter Wave radar technology to capture gesture data, combined with optimized deep learning techniques for real-time recognition. This provides an accessible and non-intrusive platform for gesture control that enhances privacy since no visual data is recorded. Users can dynamically select tracks and control musical parameters in real-time using expressive hand motions, integrating seamlessly with music software.

A key innovation of WaveTune is the development of an optimized gesture recognition model that achieves high accuracy for real-time music interaction while minimizing complexity. This is accomplished through novel optimizations to a state-of-the-art point cloud classification architecture, resulting in an efficient and tailored model for fluid musical control.

Furthermore, WaveTune promotes open-source collaboration by providing full access to code, configurations and datasets, inviting the community to build upon this system.