Nowadays, the popularity of machine learning and artificial intelligence algorithms is very high. A new research direction has emerged where the machine learning algorithms are executed on resource-constrained embedded devices. With the development of the Internet of Things paradigm, these edge devices are deployed in a lot of places. Due to the limited resources of embedded devices, it is difficult to bring machine learning algorithms to them. This is where the on-device split inference comes in. It is possible to distribute the inference between multiple edge devices and the cloud so that the edge devices can execute the inference of complex machine learning models. This paper presents a systematized literature review of papers that focus on on-device split inference for edge devices. The papers are analyzed and compared based on pre-determined questions and displayed based on several categories. ... Read more

The explosive growth in data collection driven by the proliferation of interconnected devices necessitates novel approaches to data processing. Traditional centralised data processing methods are increasingly inadequate due to the sheer volume of data generated. Distributed Tiny Learning (DTL) offers a compelling solution by distributing machine learning tasks across multiple edge devices and processing data locally, thus minimising the need for data transmission to central servers. This approach is particularly beneficial in scenarios with limited network bandwidth and stringent privacy requirements, enhancing data security and compliance with privacy regulations. The advent of 6G networks, with their promise of unprecedented speed, capacity, and reliability, can further amplify the power of DTL. By providing higher bandwidth and lower latency, 6G enables more efficient data processing and communication among edge devices, thereby enhancing the overall performance and scalability of DTL systems. This integration supports real-time decision-making for applications such as autonomous vehicles, smart cities, and healthcare monitoring.

This paper conducts a comprehensive survey of the state-of-the-art in DTL, categorising scientific literature, mapping the ecosystem and tools, and addressing performance, efficiency, and scalability challenges in ultra-low-power devices within a 6G context. Additionally, it implements and benchmarks two DTL algorithms, providing practical insights into their effectiveness and operational viability. ... Read more

This paper presents a study focused on developing an efficient signal processing pipeline and identifying suitable machine learning models for real-time gesture recognition using a testbed consisting of an Arduino Nano 33 BLE and three OPT101 photodiodes. Our research aims to address the challenges of limited computational power whilst maintaining a high inference accuracy.

Experiments were conducted to optimise the signal processing and explore various machine learning model architectures, specifically revolving around convolutional neural networks. The data used for these experiments was gathered by creating a dataset of gestures from left- and right-handed participants. We took ethical considerations regarding participant recruitment and data security into account and we made sure to balance the dataset with both left- and right-handed participants as much as possible.

We obtained accurate gesture recognition results, surpassing the goal of a 75% success rate. Our machine learning models, trained on pre-processed 2D data, achieved near real-time inference times while running on the resource-constrained Arduino Nano 33 BLE.

The findings of this study contribute to the field of gesture recognition by providing insights into efficient signal processing techniques and identifying suitable machine learning models for resource-constrained devices. The developed system can be applied in various applications, ranging from games to healthcare. Furthermore, a dataset is contributed which can be used for further research. ... Read more

This paper describes the feasibility of digit detection using three photodiodes and an Arduino Nano 33 BLE. This is done using a controlled lighting condition, using a bright lamp. It dives into the process of data collection, preprocessing and model selection for a recurrent neural network to do the classification of gestures into digits. Using a ”ConvLSTM double conv. layer model” with 128 units we were able to achieve an average accuracy of 0.500 ± 0.091 on a 5-Fold cross-validation procedure based on data that was collected in a controlled lighting environment. While this provides a foundation for digit detection using time-series data in a controlled light environment, further suggestions are made for future improvements and expansions in this area. ... Read more

Visible light sensing is a field of research that creates new possibilities for human-computer interaction. This research shows the viability of designing a system for detecting hand gestures using a cost-effective detection circuit employing 3 light-sensitive photodiodes. The way this research shows viability is by developing a machine-learning model that works on 3D-structured sensor data that is able to distinguish 10 different gestures and deploying the model on a standalone Arduino Nano 33 BLE microcontroller controlling the system. Using a combination of Convolutional Neural Networks and Recurrent Neural Networks it is possible to deploy a model called ConvLSTM-128 that achieves an accuracy of 70% on a dataset of limited size. This research acknowledges that the achieved accuracy is not suitable for real-world use, but concludes by outlining steps that could help future research in increasing the accuracy. Furthermore, an analysis of the 10 gestures shows that in order to improve accuracy, the way some gestures are performed might need alteration. Finally, a model size of around 140Kb and an inference time of 660ms show that this model is compact and fast enough to be deployed in real-world applications. ... Read more

Touchless interaction with computers has become more important in recent years, especially in the context of the COVID-19 pandemic.
Applications include situations where touch input is not possible or not desirable, e.g. for hygienic purposes in a public setting or a medical setting.
Practical examples for touchless interaction include elevators, vending machines, and other public devices that are used by many people.
However, most touchless interaction systems are expensive and require significant computational power.
This paper proposes a bare-bones low-power and low-cost system for recognizing air-written digits using a microcontroller and light sensitivity sensors.
A proof of concept has been created and tested in a fixed lighting scenario with a fixed set of gestures for the digits 0 to 9 to show the feasibility of such a system.
The system uses a convolutional neural network to recognize digits and achieves an average accuracy of 58,8% on a validation set of unseen participants.
It performs significantly better on new samples from users already seen during training, achieving an average accuracy of 93,5%.
... Read more

This study introduces a novel system that leverages three photodiodes and ambient light to identify air-written characters on a resource-constrained device. Through experimentation, suitable methods of data preprocessing, machine learning and model compression were selected to recognize the first 10 characters of the Latin alphabet. The final system was able to recognize these characters with a between-participant accuracy of 50.80% and a within-participant accuracy of 67.82%. ... Read more

With the widespread application of artificial intelligence, centralized machine learning approaches, which require access to users' local data, have raised concerns about data privacy. In response, federated learning, an architecture that aggregates models trained locally with local data, has been proposed. This approach addresses the data privacy issues inherent in centralized machine learning while also alleviating the high communication costs and server resource demands. However, due to its architectural nature, federated learning, with its single global model, struggles to meet the diverse personalized needs of clients and suffers significant accuracy degradation when client data distributions are uneven or exhibit non-IID characteristics. Personalized federated learning has been introduced to address these issues of data heterogeneity and personalized needs. Its goal is not to train a single global model but to ensure that each client participating in the personalized federated learning framework has a local model that meets their individual needs. Yet, personalized federated learning also has its shortcomings: the global model in federated learning often becomes an intermediary product in this framework, lacking the advantage of learning a generalized model. This thesis proposes a new personalized federated learning scheme, Federated Learning with Rebalanced Dataset(FedReb), which is based on parameter decoupling. By introducing a rebalanced dataset generated according to the distribution of clients' local data, this framework achieves high accuracy for both the global model and average client models. Comparative experiments demonstrate its superior scalability and robustness over other federated learning and personalized federated learning algorithms, and the report suggests optimal configurations for achieving the best results with reasonable costs. Additionally, a testbed has been established, and the deployment of the algorithm on it has been realized, verifying the feasibility of the algorithm in real-world setups. ... Read more

This paper presents how a convolutional neural network can be constructed in order to recognise gestures using photodiodes and ambient light. A number of candidates are presented and evaluated, with the most performant being adopted for in-depth analysis. This network is then compressed in order to be ran on an Arduino Nano 33 BLE microcontroller to present its feasibility in embedded operation. The final utilised network was observed to have accuracies between 75.4% and 86.8% depending on the testing conditions. Further, all candidates were found to be sufficiently compact and low-latency for real-time operation. ... Read more

Touching physical buttons to interact with public electronic devices has raised some concerns regrading disease transmission following the COVID-19 pandemic. The use of hand gestures as a touchless replacement sounds appealing, but comes with the challenge of recognizing which gesture is being performed by the user, with only the processing power of a microcontroller. This paper explores the use of recurrent neural networks (RNNs) and their derivatives to recognize hand gestures on an Arduino Nano 33 BLE. The neural networks receive input from 3 OPT101 photodiodes, which emit a voltage that increases with the intensity of light that hits them, meaning they can effectively track hand shadows cast by the user’s hand under ambient light. After testing various RNN-based neural network architectures, CNN-LSTMs produced the highest validation accuracy. However, due to issues with the testing setup, the highest validation accuracy measured for a CNN-LSTM was only 43%, indicating that further work is required. ... Read more

There is a growing need for touch-free interaction with public utilities such as coffeemakers and vending machines that will help prevent the spread of diseases such as COVID-19. One solution is the integration of embedded gesture recognition systems relying on ambient light. However, existing work so far is found to be inefficient in terms of size, cost and recognisable gestures. This research is part of the development of a smaller and more economical machine learning-powered gesture recognition system using only 3 photodiodes and an Arduino microcontroller. The goal is to design the software for sensor reading, gesture detection and data preprocessing. The resulting receiver samples at 100 Hz, uses an adaptable threshold for identifying gesture endpoints and a mix of FFT,
maximum division and Linear Interpolation for signal processing. It is evaluated in two lighting conditions on two distinct gestures and is found to provide a
good trade-off between simplicity, real-time processing within milliseconds and robustness against environmental changes. This is achieved with a small RAM memory footprint of only 2 KB and independence of classification backend. The existing design
can be further improved in the future through software optimisation and extended environment dynamics support. ... Read more