WaveTune: Millimeter Wave Radar based Hand Gesture Recognition for Musical Applications
S.S. Juneja (TU Delft - Electrical Engineering, Mathematics and Computer Science)
Marco Zúñiga Zuñiga Zamalloa – Mentor (TU Delft - Networked Systems)
Qing Wang – Graduation committee member (TU Delft - Embedded Systems)
G. Vaidya – Graduation committee member (TU Delft - Networked Systems)
More Info
expand_more
Other than for strictly personal use, it is not permitted to download, forward or distribute the text or part of it, without the consent of the author(s) and/or copyright holder(s), unless the work is under an open content license such as Creative Commons.
Abstract
In the ever-evolving field of music technology, new solutions continue to emerge that enhance musical expression and creativity. This thesis introduces WaveTune, a novel lightweight hand gesture recognition system that enables real-time control of musical composition and performance through natural hand motions.
WaveTune utilizes Millimeter Wave radar technology to capture gesture data, combined with optimized deep learning techniques for real-time recognition. This provides an accessible and non-intrusive platform for gesture control that enhances privacy since no visual data is recorded. Users can dynamically select tracks and control musical parameters in real-time using expressive hand motions, integrating seamlessly with music software.
A key innovation of WaveTune is the development of an optimized gesture recognition model that achieves high accuracy for real-time music interaction while minimizing complexity. This is accomplished through novel optimizations to a state-of-the-art point cloud classification architecture, resulting in an efficient and tailored model for fluid musical control.
Furthermore, WaveTune promotes open-source collaboration by providing full access to code, configurations and datasets, inviting the community to build upon this system.
Files
File under embargo until 01-09-2025