Machine learning can be effectively applied in control loops to robustly make optimal control decisions. There is increasing interest in using spiking neural networks (SNNs) as the apparatus for machine learning in control engineering, because SNNs can potentially offer high energy efficiency and new SNN-enabling neuromorphic hardwares are being rapidly developed. A defining character of control problems is that environmental reactions and delayed rewards must be considered. While reinforcement learning (RL) provides the fundamental mechanisms to address such problems, realizing these mechanisms in SNN learning has been underexplored. Previously, schemes of spike timing dependent plasticity (STDP) learning modulated by factors of temporal difference (TD-STDP) or reward (R-STDP) have been proposed for RL with SNN. Here we designed and implemented an SNN controller to explore and compare these two schemes by considering Cart-Pole balancing as a representative example. While the TD-based learning rules are very general ones, the resulted model exhibits rather slow convergence, producing noisy and imperfect results even after prolonged training. We show that by integrating the understanding of the dynamics of the environment into the reward function of R-STDP, a robust SNN-based controller can be learnt much more efficiently than by TD-STDP. The work of this master thesis project has also been published as a paper in Electronics, Vol. 12.

This thesis describes the design and implementation of a controller and GUI for an AI loudspeaker filter design program. This program uses a genetic algorithm to create a combination of analog passive filters, one for each driver in a loudspeaker system. In the controller, two cost functions are designed to evaluate these filter combinations, and when the genetic algorithm has created its final filters, unnecessary components are removed and all component values are optimized. A GUI is created to allow easy user interaction. For selecting a cost function, not enough data was obtained to make a deliberate decision based on performance. Therefore, this decision was based on theory and subordinate features. Nonetheless, the final program is able to create flat acoustic responses in a margin of 2 dB.

Scalable universal quantum computers require classical control hardware, physically close to the quantum devices at cryogenic temperatures. Such classical controllers need digital memory for various applications, ranging from high-speed queues to high-speed and low-speed lookup tables and working memory. The power consumption of the memories should be within the available cooling power at these temperatures. To obtain the best memory design with the lowest power consumption, cryogenic-CMOS characteristics need to be taken into account during design. This thesis aims to develop a model that can be used to find the optimal memory cell design for each application, while taking area, latency, error rate, and power constraints into account. A model is developed to estimate the error rate and power consumption of a memory core for four cell designs, namely three embedded dynamic cell designs and one static cell design, over a range of applications in terms of memory operation frequency and read/write operation ratio. The model is constructed using room temperature and 233 K simulation data of individual cells and peripherals from a TSMC 40 nm technology. To estimate the error rate and power consumption at 4.2 K, the model is extended with empirical cryogenic-CMOS characteristics, such as an increase in threshold voltage and a steeper subthreshold slope, since good device models at this temperature are not available. To verify and improve the memory model, a test chip in TSMC 40 nm technology is designed, which includes eight fully-custom memories with two threshold-variations of each of the four cell designs to mitigate the cryogenic-CMOS threshold voltage increase. These memories are connected to an on-chip programmable microcontroller through a bus which enables flexible and high-speed testing without the need for high-speed I/O. This chip design is taped out and will be measured to verify and refine the model. At room temperature, the static cell design outperforms the dynamic cells in all memory applications with less than 107 operations per second. However, at cryogenic temperatures, the embedded dynamic cell designs become feasible for applications with more than 300 operations per second, due to the significantly reduced refresh rate. The best embedded dynamic cell design depends on the read/write memory operation ratio required by the application. After verification and improvement of the memory model based on the measurement results from the test chip, this model can be used to find the best cell design for a given application based on its operation frequency and read/write operation ratio. Apart from comparing the performance of a single memory design at room temperature and 4.2 K, this work also allows for direct comparison between the different memory cells designs, designed with the same technology, architecture, peripherals, and by the same design rules. Since only a single architecture is used with a single design for each of the peripherals, further optimisation is required for a cryogenic-optimised full memory design.

Ultra-low power Edge AI hardware is in increasing demand due to the battery-limited energy budget of typical Edge devices such as smartphones, wearables, and IoT sensor systems. For this purpose, this Thesis introduces an ultra-low power event-driven SRAM-based Compute In-Memory (CIM) accelerator optimized for inference of Binary Spiking Neural Networks (B-SNNs). In this Thesis, a custom-designed 3nm SRAM cell is developed, with up to four read ports to improve inference performance and one transposable read/write port for efficient on-chip learning functionality. The event-based nature of SNNs is exploited to minimize the computation and memory cost. The design benefits from technology scaling of fully digital design by synthesizing the accelerator in the imec 3nm FinFET technology node. The proposed accelerator's performance is evaluated by running MNIST inference at 97.6% accuracy, achieving an impressive throughput of 44M inferences/s at 607 pJ/inference (3.2 fJ per synaptic operation) while running at 29 mW. The results demonstrate that the proposed accelerator provides an energy-efficient and high-performance solution for inference of Binary SNNs, opening up new possibilities for Edge AI applications.

Spiking Neural Networks(SNN) have been widely leveraged by neuromorphic systems due to their ability to closely mimic biological neural behavior, where information is exchanged and received between neurons in the form of sparse events(spikes). Such neuromorphic systems are highly energy-efficient because the use of a global clock can be avoided by asynchronous event-driven operations. Neurons, as the basic processing units of neuromorphic systems, are required to be low-power and high-speed for the implementation of complex networks. In this work, two fully event-driven digital Integrate-and-Fire(IF) neuron design is presented. Both design exploits the hierarchical structure, which allows the synaptic weights can be accumulated by local compute units in parallel. Instead of using handshake protocols, the proposed design generates on-demand event pulses to drive the weight accumulation, so we call it self-timed. Both neurons are designed by SystemVerilog and synthesized in TSMC 28nm technology. According to the synthesis results, both designs can finish the accumulation of 1024 6-bit weights within 100ns, with a power consumption of 0.055pJ per spike and 0.23pJ per spike respectively.

The advent of Artificial Intelligence (AI) and Internet-of-things (IoT) has led to a significant demand for edge computing and enabling Neural Network Implementation on edge devices. However, due to large MAC operations involved in the implementation of Neural networks, the traditional digital hardware based on the von-Neumann architecture is not well suited for an edge device. Computation-in-Memory (CIM) is an attractive alternative in mitigat- ing the challenges involved with traditional hardware by directly processing the data within the memory. It utilizes emerging memory devices such as Resistive Random Access Memory (RRAM) to perform in-place computations in the analogue domain, thereby, eliminating the bottlenecks associated with the constant movement of data in the von-Neumann architecture. However, the standard implementation of CIM comes with several challenges, with the pri- mary being high power consumption, which debate its implementation on an edge device in accelerating Neural network computation. Thus, this work proposes a novel CIM crossbar that has the potential to alleviate the challenges associated with the standard CIM cross- bar. Subsequently, an ultra-low power micro-architecture design is proposed, based on the novel CIM crossbar, that can accelerate Binary Neural Networks (BNN) and Spiking Neural Networks (SNN) with high power efficiency. The benchmark results obtained over the imple- mentation of custom-developed BNN and SNN, trained over the MNIST dataset, indicate a power reduction of 13x and 26x respectively for the proposed micro-architecture, compared to its standard CIM crossbar counterpart. In addition, the proposed micro-architecture exhibits energy savings of around 4-5x over both BNN and SNN, making it a promising alternative for accelerating neural network computation over edge devices.

The ever-increasing energy demands of traditional computing platforms (CPU, GPU) for large-scale deployment of Artificial Intelligence (AI) has spawned an exploration for better alternatives to existing von-Neumann compute architectures. Computation In-Memory (CIM) using emerging memory technologies such as Resistive Random Access Memory (RRAM) provide an energy-efficient and scalable alternative for Deep Neural Networks (DNN) applications. However, the benefits of CIM frameworks come at the cost of low DNN accuracy due to non-idealities in RRAM devices. In this thesis we address the conductance variation non-ideality in RRAM devices at an architectural level. We present two mapping schemes to improve the accuracy of CIM-based DNNs in the presence of RRAM conductance variation. Experimentation conducted with five datasets show that all proposed schemes provide up to 5.4x accuracy improvement over state-of-the-art implementations while inducing a 1.5% area cost and up to 10% energy overhead. Based on accuracy-energy trade-off, the thesis concludes the proposed Complementary Conductance Matrix (CCM) is the best candidate to improve inference accuracy of neural networks on CIM hardware using RRAM. It reports an accuracy improvement up to 5x with 1.52% area overhead and 9% energy overhead.

Computation-In-Memory (CIM) employing Resistive-RAM (RRAM)-based crossbar arrays is a promising solution to implement Neural Networks (NNs) on hardware, such that they are efficient with respect to consumption of energy, memory, computational resources, and computation time. In this respect, Binary NNs (BNNs), where the weights obtain single binary values, are inherently suitable for cost-effective CIM-based NN implementations. However, RRAM devices, due to variability and reliability issues, restrict the applicability of CIM-based NN. To address this issue and towards a low-cost NN hardware realization, in this thesis, we: a) thoroughly investigate the impact of RRAM faults on the inference accuracy of RRAM-based BNNs, and b) propose three complementary fault-tolerance techniques to mitigate the impact of RRAM faults on the BNN's accuracy. These techniques are namely: a) a fault-tolerant activation function, b) a redundancy and weight range adjustment scheme and c) a retraining technique. Evaluation results compiled on the MNIST, Fashion-MNIST, and CIFAR-10 datasets demonstrate that the proposed techniques can improve the inference accuracy in the presence of RRAM faults by up to 20%, 40%, and 80%, respectively. Moreover, comparisons with certain related state-of-the-art fault-tolerance frameworks indicate that the proposed techniques yield competitive results.

LoRaWAN is a public Wireless Sensor network with excellent properties like being long-range, low-energy radios and resulting in long battery life. Devices are connected to this network through gateways, and they will run in that deployment for years without replacement. Therefore, bugs and security issues in such devices' firmware are present for a long time, unless firmware updates are applied to fix them. Firmware-Updates Over-The-Air (FUOTA) is a firmware update application framework for LoRaWAN, but packet loss is an unsolved issue for such firmware updating frameworks. State-of-the-art packet dissemination uses the Low-Density Parity Checks (LDPC), but this has decoding overhead. Also, it is inflexible due to its fixed-rate nature. The code can not be dynamically adjusted to adapt to temporary changes in channel conditions. In this work, we provide critical analysis to justify replacing Low-Density Parity Checks code (LDPC) proposed in FUOTA with the famous code Random Linear Network Coding (RLNC). The benefit of RLNC shows when scaling to multicast networks using LoRaWAN FUOTA as fewer messages need to be exchanged to serve the complete network of firmware updates. An analysis is presented on how to configure the finite field size, generation size, and redundancy for generation-based RLNC. The parameters are optimized to cope with the worst-case packet-error rate. As such, RLNC can optimally counter every lost packet with redundancy with near-zero decoding overhead. Since devices sleep after decoding a set of fragments, or the generation, sending additional fragments does not decrease efficiency. Finally, the decoding probability is provided as an analytical tool to show that a specific amount of redundancy can deal with a specific worst-case packet-error rate. To evaluate and test the RLNC code for LoRa, embedded firmware and terminal software have been developed to do indoor and outdoor measurements. This testbed has been developed with a custom control plane replacing all FUOTA MAC layer modules (Fragmentation, Multicast, Synchronization, Device Management, etc.). Therefore, it can be shaped into custom network configurations meant for evaluating the firmware updating process without having to set up infrastructures like ChirpStack or TheThingsStack. Our work's testbed can isolate the decoding process and evaluate it for artificial packet drops. Results are presented, which show that systematic coding phase of LDPC will perform poorly compared to RLNC. This systematic phase is at least as large as the firmware update. Between 38% to 88% improvement in decoding success is found by using RLNC in case of varying network conditions due to burst loss.

Radar systems have been used for decades to detect targets on the ground and in the air. The radar signal is transformed into a range-doppler image that distinguishes each detected object by range and velocity for further processing. A target detection algorithm is used to filter noise and clutter. Each target can be in a region with a different noise level; a simple threshold would yield false positives or miss detections depending on this value. To solve this problem, a Constant False Alarm Rate or CFAR is desirable. A CFAR detector estimates the noise surrounding each target and uses a dynamic threshold based on this. Spiking Neural Networks are the third generation of Artificial Neural Networks where, instead of continuous signals, the input is encoded into trains of spikes over time. These networks have a potentially efficient hardware implementation instead of the older generation Artificial Neural Networks and could run directly at the sensor edge, lowering latency and power consumption. This thesis will explore a Spiking Cell Averaging CFAR implementation and attempt to use its desirable properties like a temporal average over multiple radar frames, mimicking the non-coherent integration sometimes done in radar processing. It is shown that some configurations will behave similarly in a simulated environment with additive white gaussian noise.

Diabetic Retinopathy (DR) is one of the leading causes of permanent vision loss. Its current prevalence is around 45 millions across the globe and is projected to 70 million by 2045. Most of the people with this disease condition belong to remote and low income settings. We can reduce this incidence, if quality medical care is accessible in remote areas. With the current advancements in imaging technologies, fundus examination can be carried out on a handheld device. We need to improve such devices to deliver high quality services through auto detecting DR based on convolutional neural networks(CNNs) in an offline setting. Addressing these challenges, we aim to develop an integrated solution which delivers high compute at ultra-low power consumption. Firstly, we have created 3 datasets of different sizes merging multiple public datasets to create a vigilant model training process. This is to make the CNN model robust to real-world noise. CNNs trained on smaller datasets have shown a 15% accuracy drop on evaluation datasets where as CNNs trained on large datasets showed consistent performance. Secondly, we have proposed a new binary labelling scheme using multi-class output to maximize the utility of its softmax probabilities. We have achieved 90.26% accuracy on evaluation dataset with the new scheme. These high performing models along with compression techniques are implemented on resistive random access memory (RRAM) based computational in memory (CIM) architecture. These implementations resulted in atleast 200x improvement in energy consumption for inferring on one image when compared to CPU, GPU and mTPU (google coral dev board). Similarly, latency improvements 28x,130x and 2000x compared GPU,CPU and mTPU are registered. Model quantization with 4-bit precision for weights have preserved the original accuracy and showed 4x improvements in energy consumption on RRAM implementation.

Epilepsy is a system-wide phenomenon which manifests physically across the body in various forms such as rapid muscle tone, sweating, elevated heart rate and synchronized neuron firing. Many of these aspects appear ahead of an epileptic incident. If combined together, these aspects are tell-tale signs of an imminent ictal event. Because of these observations, a wireless medical body area network (MBAN) has been proposed, which implements multimodal-data integration and closed-loop seizure suppression. The design and implementation of the MBAN introduces several challenges, such as data collection, seizure prediction and suppression, secure pairing and communication and the design of a user friendly interface. This thesis will focus on the design and implementation of a secure bus architecture that connects multiple processing cores, a sensor and an actuator within an MBAN node. The interconnect will provide communication between the masters and slaves via an AMBA AHB-Lite protocol. Furthermore, a memory protection unit (MPU) will deny accesses from unauthorized peripherals. Additionally, the prototype will provide a secure communication protocol for updating the MPU. Finally, the prototype will be able to communicate wirelessly with a smartphone. The deliverable will be a proof-of-concept implementation on an FGPA, demonstrating the previously described functionalities. Nevertheless, the design choices will be made with the application in mind.