Running deep learning models directly on microcontroller units (MCUs)—a field known as TinyML—enables artificial intelligence in energy-restricted applications that rely strictly on limited battery power or harvested local energy. Creating models for these constrained devices requires strict optimization via neural architecture search (NAS) frameworks such as µNAS. However, traditional proxies like multiply-accumulate (MAC) counts fail to serve as accurate energy predictors because they overlook complex hardware interactions and compiler-level runtime optimizations introduced by deployment engines such as TensorFlow Lite (TFLite) Micro.

In this paper, we explore accurate energy estimation within the µNAS framework. We build an automated hardware-in-the-loop (HIL) profiling pipeline to deploy diverse architectures on an MCU and record their physical power draw, generating a dataset of 671 unique models. Evaluating a baseline linear regression predictor applied to MAC counts achieves a high macro-level fit (R² = 0.985) but suffers from an unacceptable mean absolute percentage error (MAPE) of 85.4% due to structural oversights.

To address this limitation, we propose a novel energy estimator based on a directed acyclic graph neural network (DAGNN). By processing neural network topology directly, the DAGNN learns complex hardware interactions and runtime optimization behaviors. Our estimator substantially outperforms the baseline, reducing MAPE from 85.4% to 16.0%. ... Read more

Artificial Intelligence is increasingly being used in everyday devices. However, most AI systems are designed to run on powerful computers or cloud servers rather than on small, low-power devices such as microcontrollers. Running AI directly on these devices can reduce energy consumption and enable systems to operate without an internet connection. AIfES (Artificial Intelligence for Embedded Systems) is a machine learning framework that allows neural networks to be trained directly on microcontrollers. However, it currently lacks support for depthwise convolution, an important operation used in efficient neural network architectures such as MobileNet. As a result, many modern computer vision models cannot be trained within the framework.

This project extends AIfES with support for depthwise convolution and integrates the new operator into the existing training pipeline. The implementation was validated using a combination of manually verified test cases, comparisons with TensorFlow, and image classification experiments on embedded hardware. The results show that the new operator functions correctly during both inference and training. Models containing the implemented layer successfully learned classification tasks and achieved behavior similar to equivalent TensorFlow models. By adding support for depthwise convolution, this work expands the range of neural network architectures that can be trained directly on microcontrollers and contributes to making on-device AI more practical and flexible. ... Read more

Embedded sensing systems relying on energy harvesting — such as electromagnetic radiation, thermoelectric energy, and kinetic energy — generally are not able to harvest sufficient power to function under normal operations for most devices, and thus operate under severe power constraints. To ensure sustainable, battery free functionality, the microcontroller (MCU) must remain at a low power deep sleep state during idle periods. It is woken up by a sensor, sending an external hardware interrupt when an environmental event occurs. However, there is a trade off between a sensor’s power consumption, detection range, accuracy, and latency. This paper presents two primary contributions: 1) An empirical evaluation of various sensor wake up systems. 2) The design and implementation of a multi stage hierarchical event detection pipeline. This pipeline consists of an ultra low power coarse sensor that activates a high accuracy, but higher power sensor, minimizing the current draw while staying reliable. ... Read more

Microcontroller-based neural network inference faces significant RAM constraints, hindering performance and deployment. One of the main constraints is the peak memory usage, which is essential for conducting deep learning inferences with low latency. To address this, previous researchers have developed peak memory estimators to estimate the Peak Memory Usage, which could be used by inference-time optimization techniques like pruning to tackle the RAM constraint. But many of the peak memory estimators used by current state-of-the art frameworks like µNAS and TinyEngine produce underestimation or overestimation, reducing the reliability of model decisions made under RAM constraints. Underestimation often arises from failing to account for all components contributing to peak memory usage, while overestimation can occur when extra memory overheads irrelevant in MCU-specific inference scenarios are incorrectly included. In this paper, we propose our peak memory estimator, which estimates the peak memory usage of deep learning inference at the operator level and can accurately estimate the peak memory usage of a deep learning model during inference. The experiments show that our method achieves more accurate peak memory predictions across multiple MCU platforms and can be effectively integrated with pruning strategies to produce better model compression that satisfies both memory and accuracy constraints. Specifically, on a benchmark of 150 models, our method achieved an average estimation error margin of only 0.9%, significantly outperforming µNAS, which exhibited an average error margin of 61.7%. ... Read more

There has been a steady increase in technologies that leverage Deep Learning (DL) techniques on resource-constrained devices for real-time processing. While DL techniques are adept at recognition tasks, their performance depends on the training process. Training data is seldom fully representative of the deployment scenario, requiring retraining to preserve accuracy. Many works have proposed on-device learning techniques that enable training DL techniques like Convolutional Neural Networks (CNNs) on microcontroller units (MCUs), without requiring data to be sent to the cloud. These methods have yielded promising accuracy improvements while training with low memory footprints. However, limited research has been done on the energy implications of doing so.

This thesis presents methods for energy-aware on-device learning on MCUs. It leverages the principle of updating specific layers of a CNN, proposed in past works, to fit memory constraints. We propose an energy-accuracy trade-off objective based on computational costs (in MACs) and accuracy improvement to select which layers to train. Furthermore, we demonstrate how computationally light search algorithms can adequately maximize the newly defined objective for layer selection. Evaluations show that our approach saves up to 200mJ of energy on-device while yielding simulation accuracies similar to a recent study under the same conditions. ... Read more