Motivated by the desire to bring intelligent processing at the Edge, enabling online learning on resource- and latency-constrained embedded devices has become increasingly appealing, as it has the potential to tackle a wide range of challenges: on the one hand, it can deal with on-the-fly adaptation to fast sensor-generated streams of data under changing environments and on the other hand, it can address a variety of challenges associated with offline training in the cloud, such as incurred energy consumption of sensor data transfers and extra memory storage for the training samples, but also data privacy and security concerns. Concurrently, maintaining low-latency and power-efficient inference is paramount for edge AI computing systems, and thus learning/adapting online with minimal incurred overhead is crucial.

In this work, we propose EON-1, an Edge ONline Learning SCNN (Spiking Convolutional Neural Network) processor with 1-bit synaptic weights, 1-spike per neuron and 1-neuron updated per input, which we have benchmarked for both ASIC and FPGA platforms. Our key contribution is proposing a binary and stochastic SDTP rule which, benchmarked in an ASIC node, achieves less than 1% energy overhead for inference. To our knowledge, our solution incurs the least energy overhead for inference, compared to state-of-the-art solutions, showing a better efficiency by at least a factor of 10x. We also report 94% and 77.65% accuracy on the MNIST and Fashion-MNIST classification tasks, and we achieve 0.09pJ/SOP and 1.5pJ/SOP energy efficiency during inference and learning, respectively. We extend our solution to demonstrate a practical use-case of performing inference in real-time UHD videos while coping with streaming data and we showcase 60 FPS UHD video processing.

Ultra-low power Edge AI hardware is in increasing demand due to the battery-limited energy budget of typical Edge devices such as smartphones, wearables, and IoT sensor systems. For this purpose, this Thesis introduces an ultra-low power event-driven SRAM-based Compute In-Memory (CIM) accelerator optimized for inference of Binary Spiking Neural Networks (B-SNNs). In this Thesis, a custom-designed 3nm SRAM cell is developed, with up to four read ports to improve inference performance and one transposable read/write port for efficient on-chip learning functionality. The event-based nature of SNNs is exploited to minimize the computation and memory cost. The design benefits from technology scaling of fully digital design by synthesizing the accelerator in the imec 3nm FinFET technology node. The proposed accelerator's performance is evaluated by running MNIST inference at 97.6% accuracy, achieving an impressive throughput of 44M inferences/s at 607 pJ/inference (3.2 fJ per synaptic operation) while running at 29 mW. The results demonstrate that the proposed accelerator provides an energy-efficient and high-performance solution for inference of Binary SNNs, opening up new possibilities for Edge AI applications.

In the recent past, real-time video processing using state-of-the-art deep neural networks (DNN) has achieved human-like accuracy but at the cost of high energy consumption, making them infeasible for edge device deployment. The energy consumed by running DNNs on hardware accelerators is dominated by the number of memory read/writes and multiplyaccumulate (MAC) operations required. As a potential solution, this work explores the role of activation sparsity in efficient DNN inference. As the predominant operation in DNNs is matrix-vector multiplication of weights with activations, skipping operations and memory
fetches where (at least) one of them is zero can make inference more energy efficient. Although spatial sparsification of activations is researched extensively, introducing and exploiting temporal sparsity is much less explored in DNN literature. This work presents a new DNN layer (called temporal delta layer) whose primary objective is to induce temporal activation sparsity during training. The temporal delta layer promotes activation sparsity by performing delta operation facilitated by activation quantization and l1 norm based penalty to the cost function. During inference, the resulting model acts as a conventional quantized
DNN with high temporal activation sparsity. The new layer was incorporated as a part of the standard ResNet50 architecture to be trained and tested on the popular human action recognition dataset (UCF101). The method caused 2x improvement in activation sparsity, with 5% accuracy loss.