In the field of music technology, the ability to connect body movements with music generation has led to highly engaging applications. Many existing solutions use cameras or wearable devices to capture physical movements and translate them into commands for a music generation tool. While the camera-based systems are privacy-invasive and demand good lighting conditions, the wearable devices adversely impact the fluidity of movements. We propose WaveTune, a privacy-friendly and device-free interface for creating music beats through body gestures. Wavetune has the following components: (i) a millimeter-wave radar as the frontend sensor that captures body movements, (ii) a gesture recognition algorithm that performs gesture delimitation and identification in real-time, and (iii) a music generation component that maintains a seamless and pleasant experience. WaveTune also provides an option to continuously map random dynamic movements into musical parameters to encourage further interaction. We recruited 24 users to train and test WaveTune’s ability to map body gestures to musical commands. To the best of our knowledge, WaveTune is the first mmWave system that allows a layered composition of music beats.

Sensing people with mmWave radars is gaining significant attention. This growing interest is due to two factors: radar monitoring provides more privacy than camera-based alternatives, and radio waves are not as easily blocked as light waves. Most mmWave studies, however, have three common characteristics. They are done indoors, without protecting the sensor (no casing), and the evaluation is performed for short periods of time. To assess the suitability of mmWave sensing in realistic outdoor scenarios, we deploy two nodes to track the flow of pedestrians over a period of three months. This longterm deployment provides three main contributions. First, we follow a detailed process to design a casing that can protect the sensors from harsh environmental conditions. Second, we install our nodes close to a set of cameras that were already deployed in the area. To compare the performance of both types of sensors, we propose a framework that considers the different coverage patterns of cameras and radars. Third, the time frame of our evaluation considers various types of weather, from sunny days to rainy and windy. Our results indicate that mmWave sensors need to be explored further outside the comfort zone of indoor spaces. To the best of our knowledge, this is the first long-term study assessing the reliability of radar sensors in the 60 GHz ISM band.