Power analysis can be used to retrieve key information as secure systems leak data-dependent information over side channels. A proposed solution to break the correlation between side channel information and secret information was to replace a vulnerable part of the cryptography implementation with a neural network. This uses the inherent properties of a neural network to disrupt the correlation by breaking the linear power characteristics assumed by leakage models. To test this neural network without physically creating a hardware implementation a simulation must be performed that provides both the data and the power information. Currently neural network simulators do not generate a power trace and analog circuit simulators generate more information traces than required increasing the simulation time. This thesis describes the creation of a complete SystemC spiking neural network model that generates both data and power information. The information generated by this model was compared and verified with results acquired by the Cadence Spectre  analog circuit simulation platform. The results indicate that the created SystemC SNN model works and generates comparable data and power traces as the Spectre simulator.

... Read more

Neuromorphic computing can be used to efficiently implement spiking neural networks.
Such spiking neural networks can be used in edge AI applications, where low power consumption is paramount.
The use of analog components allows for extremely low power implementations.
This thesis contributes the designs of an analog spike generator, synaptic elements and an accumulating neuron in 28 nm CMOS technology.
The elements are assembled in a neural network and laid out in an SoC.
Energy consumption numbers of less than 1 pJ/synaptic operation are achieved in the analog neuromorphic components. ... Read more

Spiking Neural Networks(SNN) have been widely leveraged by neuromorphic systems due to their ability to closely mimic biological neural behavior, where information is exchanged and received between neurons in the form of sparse events(spikes). Such neuromorphic systems are highly energy-efficient because the use of a global clock can be avoided by asynchronous event-driven operations. Neurons, as the basic processing units of neuromorphic systems, are required to be low-power and high-speed for the implementation of complex networks. In this work, two fully event-driven digital Integrate-and-Fire(IF) neuron design is presented. Both design exploits the hierarchical structure, which allows the synaptic weights can be accumulated by local compute units in parallel. Instead of using handshake protocols, the proposed design generates on-demand event pulses to drive the weight accumulation, so we call it self-timed. Both neurons are designed by SystemVerilog and synthesized in TSMC 28nm technology. According to the synthesis results, both designs can finish the accumulation of 1024 6-bit weights within 100ns, with a power consumption of 0.055pJ per spike and 0.23pJ per spike respectively. ... Read more

A Spiking neural network (SNN) is a type of artificial neural network which encodes information using spike timing, network structure, and synaptic weights to emulate the information processing function of the human brain. Within an SNN, it is always required to support the spike transmission that travels between neurons(array). This thesis aims to design a customized high-speed interconnect system which supports multi-point communication in a neuromorphic computing system. The burst-mode two-wire protocol in point-to-point communication is applied in this interconnect system, which is designed in high-level modelling with SystemC. In order to improve the utilization of hardware resources, a virtual channel system is involved. Furthermore, this system could be extended to a variable number of neuron arrays to support different types of spiking neural networks. Also, optimization methods are adopted to increase the transmission rate of the system and save unnecessary energy consumption. The interconnect system could achieve a throughput of 3.802 Gbits/s with the given MNIST use case, based on the evaluation of simulation results. ... Read more

The prosperity of the Internet-of-Things (IoT) imposes increasing demand on endpoint microcontroller-based devices' performance and energy efficiency. The MCUs are demanded to process the raw data acquired from the sensors with the integer-based workload, such as digital signal processing (DSP) algorithms and quantized neural network (QNN) inference. Snitch is a tiny RV32I control core based on RISC-V open-source instruction set architecture. Currently, the Snitch system built around the Snitch core aims to achieve high performance in floating-point applications. Novel hardware extensions have been implemented in its floating-point subsystem to achieve high floating-point unit (FPU) utilization, such as stream semantic registers (SSRs) and floating-point repetition (FREP) hardware loop. However, it only has RV32IM instruction set support for integer computation, which does not satisfy the increasing demand from the integer workload we mentioned. In this work, we present a unified Snitch architecture with integer extensions targeting integer workload acceleration. Some existing custom extensions to address performance bottlenecks in DSP and QNN applications were proposed, which are Xpulpimg ISA and sub-byte single-instruction-multiple-data (SIMD) ISA, respectively. Both extensions are built on the outdated version of Snitch in another many-core system Mempool. In our work, we first integrated the DSP-oriented ISA extension Xpulpimg and the sub-byte SIMD ISA extension into the mainline Snitch. Then we extended the existing floating-point SSR to have integer support. To evaluate the proposed extensions, we benchmarked the Snitch core complex (CC) with integer matrix multiplication algorithms and compared the performance between the baseline RV32IM and our extensions. A speedup of 5.9$\times$, 22.6$\times$, and 77.4$\times$ in terms of MACs/cycle with respect to the baseline was measured for 32-bit, 8-bit and 4-bit data sizes, respectively. Post-synthesis figures have been obtained from GlobalFoundries 22 nm technology for area and timing evaluations. Our integer extensions only introduced 12\% area overhead compared with the original FP-capable Snitch CC, and they led to no measurable impact in terms of the maximum effective frequency with FP extensions enabled. ... Read more

Over the last decade, the recognition of the potential value of augmented reality (AR) and other human-machine interfaces has been growing. These applications are all based on depth sensing technologies. Among various depth sensing technologies, the Time-of-Flight (ToF) approach is emerging as a widely applicable method because it has the potential of reaching much longer distances at higher speed and accuracy, is natively suitable for mobile phones or AR.
One obvious problem is that the dToF system mainly targets the automotive industry or 3D imaging in close range, which means different power, accuracy, and fewer area requirements than mobile phone or AR applications. The application of the dToF system in the mobile phone or AR industry needs to be tested. The data volume is another problem of dToF. Usually, in the read out part, the dToF time-to digital converter (TDC)’s timestamps will all be saved into a histogram. The peak value of the histogram is the detected result. However, in a large pixels scenario, the area cost will be too much for mobile phone or AR applications if the whole histogram for all pixels is saved. Hence, an algorithm that can save histogram partially or find out the peak value without saving histogram is needed.
This thesis proposed two novel algorithms for the dToF system’s read out part and tested three other algorithms’ functions in mobile phone or AR applications. With the Lambertian model and probabilistic theory, a module of the dToF system is built using MATLAB to generate a testing dataset. Besides, introductions to depth-sensing technologies, single-photon avalanche diode (SPAD) sensors, and dToF systems will be given before the core chapters of the thesis.

... Read more

Spiking neural networks (SNN), as the third-generation artificial neural network, has a similar potential pulse triggering mechanism to the biological neuron. This mechanism enables the spiking neural network to increase computing power compared to the traditional artificial neural network to process complex information. However, a large number of interconnection resources is required. This requirement is highly consistent with the characteristics of the network on chip (NoC). This thesis is aimed at developing a scalable cycle-accurate simulator based on Noxim, which provides a configurable NoC that can simulate neuron-to-neuron communication for delivering spiking traffic. This simulator achieves several configurable metrics including topology and routing schemes, network size, the number of channels, and neuron mapping methods. This thesis then evaluates the effects of these metrics on performance for two kinds of traffic patterns. To take power consumption and area into account, this thesis also provides an approximate estimate of area and power consumption for trade-offs in the early-design stage. ... Read more

Renewed interest in memory technologies such as memristors and ferroelectric devices can provide opportunities for traditional and non-traditional computing systems alike. To make versatile, reprogrammable AI hardware possible, neuromorphic systems are in need of a low-power, non-volatile and analog memory solution to store the weights of the spiking neural network (SNN). In addition to being used for memory, memristive memory can be read out passively and thus also replaces digital-to-analog circuitry.
In this thesis, two solutions are proposed: one is based on a generalized memristor, the other is based on ferroelectric memory. Both solutions are implemented and simulated in SystemC AMS and tested with a SNN. As a final test, both memory solutions are integrated into a full-sized SNN and simulated against the MNIST dataset. The simulation results validate the capabilities of memristive and ferroelectric memory when it comes to providing a sensible weight storage solution for neuromorphic systems. ... Read more

Spiking Neural Networks use Address Event Representation to communicate among different Neuron Arrays. To mimic the behavior of the human neural system and meets the requirement for large Neuron Array communication, the AER interconnect should be area-saving, have low power, and operates at high speed.
This thesis aims to build self-timed interconnects for point-to-point and multi-array communication. The whole system is designed at the RTL level using SystemVerilog. For point-to-point communication, two transmitters are implemented and compared according to their synthesis results. In the multi-array communication structure, we develop a generalized segmented-bus topology and the element - Fence to control its segments. Different timing problems in the design are analyzed, and corresponding solutions are proposed. The whole system can operate at around 1Gbps in a self-timed manner without
any timing problems. ... Read more

Convolutional Neural Networks (CNN) have become a popular solution for computer vision problems. However, due to the high data volumes and intensive computation involved in CNNs, deploying CNNs on low-power hardware systems is still challenging.
The power consumption of CNNs can be prohibitive in the most common implementation platforms: CPUs and GPUs. Therefore, hardware accelerators that can exploit CNN parallelism and methods to reduce the computation burden or memory requirements are still hot research topics. Quantization is one of these methods. One suitable quantization strategy for low-power deployments is logarithmic quantization.

Logarithmic quantization for Convolutional Neural Networks (CNN): a) fits well typical weights and activation distributions, and b) allows the replacement of the multiplication operation by a shift operation that can be implemented with fewer hardware resources.
In this thesis, a new quantization method named Jumping Log Quantization (JLQ) is proposed. The key idea of JLQ is to extend the quantization range, by adding a coefficient parameter "s" in the power of two exponents ($2^{sx+i}$).

This quantization strategy skips some values from the standard logarithmic quantization. In addition, a small hardware-friendly optimization called weight de-zeroing is proposed in this work. Zero-valued weights that cannot be performed by a single shift operation are all replaced with logarithmic weights to reduce hardware resources with little accuracy loss.

To implement the Multiply-And-Accumulate (MAC) operation (needed to compute convolutions) when the weights are JLQ-ed and de-zeroed, a new Processing Element (PE) have been developed. This new PE uses a modified barrel shifter that can efficiently avoid the skipped values.
Resource utilization, area, and power consumption of the new PE standing alone are reported. Resource utilization and power consumption in a systolic-array-based accelerator are also reported.
The results show that JLQ performs better than other state-of-the-art logarithmic quantization methods when the bit width of the operands becomes very small. ... Read more

As the new generation of neural networks, Spiking Neural Network architectures
executes on specialized Neuromorphic devices. The mapping of Spiking Neural Network architectures affects the power consumption and performance of the system. The target platform of the thesis is a hardware platform with Neuromorphic Arrays with columns for neural signal processing.
The explorations for the mapping methods are based on VPR, an open-source academic CAD tool for FPGA architecture exploration. The packing of VPR is used for mapping neurons to Neuromorphic Arrays. VPR includes two levels of mapping: pins and neurons.
An evaluation of the mapping methods is established. Based on the evaluation, the optimized mapping solution is generated. Modifications are made in VPR to adapt to SNN architectures. An Activity-Criticality input file is added to the VPR flow for the optimized mapping solution. ... Read more

To support the spike propagates between neurons, neuromorphic computing systems always require a high-speed communication link.
Meanwhile, spiking neural networks are event-driven so that the communication links normally exclude the clock signal and related blocks. This thesis aims to develop a self-timed off-chip interconnect system with ring topology that supports multi-point communication in neuromorphic computing systems. This interconnect system is implemented in high-level modeling with SystemC and involves the burst-mode two-wire protocol in point-to-point communication. In order to ensure the flexibility of the system, the distributed control system is involved. Further, the system can be configured with different numbers of chiplets to fulfill various spiking neural network structures. We also explore optimization methods, which is a bi-directional ring topology achieving the growth of throughput. Based on evaluation and simulation results, the interconnect system can achieve 4.302Gbps with the specific application scenario. ... Read more

The recently introduced posit number system was designed as a replacement for IEEE 754 floating point, to alleviate some of its shortcomings. As the number distribution of posits is similar to the data distributions in deep neural networks (DNNs), posits offer a good alternative to fixed point numbers in DNNs: using posits can result in high inference accuracy while using low precision numbers. The number accuracy is most important for the first and last network layers to achieve good performance. For this reason, these are often computed using larger precision fixed point numbers compared to the hidden network layers. Instead, these can be computed using low precision posit, to reduce the memory access energy consumption and the required memory bandwidth. The hidden layer computation can still be performed using cheaper fixed point numbers.
An inference accuracy analysis is performed to quantify what the effect of this approach is on the VGG16 network for the ImageNet image classification task. Using 8 bit posit for the first and last network layer instead of 16 bit fixed point is shown to result in a top-5 accuracy degradation of only 0.24%. The hidden layers are computed using 8 bit fixed point in both cases.
The design of a parameterized systolic array accelerator performing exact accumulation is proposed that can be used in a scale-out system along with fixed point systolic array tiles. To increase hardware utilization, a hybrid posit decoder is designed to enable fixed point computation on the posit hardware. Using this hardware, the entire network can be computed using 8 bit data, instead of using 16 bits for some layers. This reduces energy consumption and the complexity of the memory hierarchy. ... Read more

This dissertation describes an approach to building a self-timed asynchronous pulse-mode serial link circuit. Unlike asynchronous handshake circuits or synchronous circuits, this design style does not require any feedback control blocks, which can increase latency, or any clock recovery circuits, which can increase energy consumption and overhead. In addition, the serial link design strategy can reduce resource consumption and the complexity of wire and pins to facilitate placement and routing. In this dissertation, a self-timed asynchronous circuit is used, which necessitates the usage of the pulse logic family. The link also includes the two-wire burst mode protocol. The time notion is included in the zero-line and one-line of the link, and the timing information is included in the transmitted pulse signal without the use of a clock.

The overall design process follows a hierarchical structure. Firstly, to achieve the logic function, the link is designed at the behavior level with VHDL code. The serializer and deserializer functions should therefore be performed at the transistor level. They are based on the pulse-logic library, which should be built initially. The transistors utilized in the design are all HVT transistors, which reduce current leakage and power consumption. The final step is to do timing characterization and optimization to ensure that the design works properly under various conditions. The entire point-to-point link can be built by using 28nm technology and can function at 5Gb/s with a power consumption of 0.5mW. ... Read more

In the recent past, real-time video processing using state-of-the-art deep neural networks (DNN) has achieved human-like accuracy but at the cost of high energy consumption, making them infeasible for edge device deployment. The energy consumed by running DNNs on hardware accelerators is dominated by the number of memory read/writes and multiplyaccumulate (MAC) operations required. As a potential solution, this work explores the role of activation sparsity in efficient DNN inference. As the predominant operation in DNNs is matrix-vector multiplication of weights with activations, skipping operations and memory
fetches where (at least) one of them is zero can make inference more energy efficient. Although spatial sparsification of activations is researched extensively, introducing and exploiting temporal sparsity is much less explored in DNN literature. This work presents a new DNN layer (called temporal delta layer) whose primary objective is to induce temporal activation sparsity during training. The temporal delta layer promotes activation sparsity by performing delta operation facilitated by activation quantization and l1 norm based penalty to the cost function. During inference, the resulting model acts as a conventional quantized
DNN with high temporal activation sparsity. The new layer was incorporated as a part of the standard ResNet50 architecture to be trained and tested on the popular human action recognition dataset (UCF101). The method caused 2x improvement in activation sparsity, with 5% accuracy loss. ... Read more

Mobile devices are getting increasingly powerful, becoming compatible
for an ever increasing set of functionality. Applications based around
neural networks however still have to offload parts of their computations
to the cloud since current Artificial Neural Networks (ANNs) are
still too computationally expensive for any practical standalone use in
energy constrained mobile devices. Developments in the next generation
of ANN: Spiking Neural Network (SNN), are expected to bring
neural networks directly to the edge. Even though SNNs are becoming
a reality, they can not (yet) effectively operate on raw sensory input
data. For this, a preprocessing algorithm can be used to extract low-level
features in an efficient way to boost the neural network efficiency.
A parallel can be found in biology with the cochlea that, for audio,
provides preprocessing for the brain. Recent research has shown that
an SNN is capable of reaching high classification accuracy when combined
with an biologically plausible audio preprocessing stage. To be
of interest for edge-computing it however also needs to be area and
energy efficient. This thesis will provide the first steps in researching
the optimal configuration of a specific audio preprocessing algorithm
by mapping its current software simulation to embedded hardware.
For this purpose the software simulation is analyzed and an efficient
hardware implementation is designed. For evaluation a prototype,
and its hardware constrained simulation, is developed and optimized. ... Read more

As we move towards edge computing, not only low power but concurrently, critical timing is demanded from the underlying hardware platform. Spiking neural networks ensure high performance and low power when run on specialized architectures like neuromorphic hardware. However, the techniques in use to configure these neural networks on massively parallel neuromorphic crossbar arrays remain sparsely explored. This motivates the research on how neural network topologies encompassing spiking architectures can be configured on a neuromorphic hardware. In this thesis, a unique placement algorithm is devised to map diverse and complex neural network architectures on a connectivity-constrained array with thousands of processing elements(PEs) within seconds. Wide spectra of SNNs with varying complexity are investigated to evaluate the feasibility of mapping on the target neuromorphic architecture involving unique connectivity constraints. The performance of the proposed ALAPIN mapper is validated through time-to-solution for the surveyed SNN schemes with varying network sizes and diverse complexity measures. Experiments show that simple networks converge within 10 milliseconds. With limited resources and as the network architectural complexity increases, hardware constraints become overwhelming to achieve placement solution within a decent time frame. Further experiments are carried out to estimate the resource utilization of each candidate SNN for varying network sizes on target hardware. Liquid state machines use a greater number of synapses for a same number of neurons than the rest of the candidates, with approximately 100% neurons, 30% input resources, and 20% synapses on target hardware. ... Read more

A big catalyst of the AI revolution has been Artificial Neural Networks (ANN), abstract computation models based on the biological neural networks in the brain. However, they require an immense amount of computational resources and power to configure and when deployed often are dependent on cloud resources to function. This makes ANNs less suitable for edge computing devices where all these resources are scares. Spiking Neural Networks (SNN) are a new generation of neural networks which process information via sparse discrete time events, called "spikes". When mapped to neuromorphic hardware, SNNs promise high energy efficiency and low computational latency. This work proposes a SNN classification architecture, using it to classify radar based hand gesture signatures. Literature on this topic is limited, leading us to explore certain ANN topologies to test our assumptions. By considering additional design limitations, we aim to find a neuromorphic hardware compatible design. While the proposed architecture is still limited in terms of classification accuracy. Our experiments have exposed interesting relationships between network sizes, accuracy and dimensionality reduction in SNNs. ... Read more

Neuromorphic electronic systems have used asynchronous logic combined with continuous-time analog circuits to emulate neurons, synapses, and learning algorithms. It is attractive because of its low power consumption and feasible implementation. Typically, the neuron firing rates are lower than the modern digital systems. Thus, the endpoints of neuromorphic electronic systems are clusters of neurons instead of individual neurons. Address event representation (AER) was proposed in 1991 to multiplex communication for a cluster of neurons into an individual communication channel. AER circuits provide multiplexing/demultiplexing functionality for spikes that are asynchronously generated by/delivered to an array of individual neurons. Asynchronous techniques are not only used in neuromorphic electronic systems, but also widely used in globally asynchronous and locally synchronous (GALS) SoCs, or SoCs with full-asynchronous solutions. However, commercial tools on the market do not support designing asynchronous circuits, making the circuits cannot be adopted easily by most products. This thesis aims at addressing the challenge by providing an asynchronous library establishment strategy. The strategy uses SR-latches as standard asynchronous cells together with logic gates to build an AER communication circuit. With the strategy, the performance of using a modified traditional arbiter in the AER transmitter can be compared favourably with using state-of-the-art arbiters. A back-end flow and a verification flow are developed to evaluate the performance of the design as well as to check the feasibility of the strategy. The proposed 32-bit AER transmitter under TSMC 28nm CMOS technology sacrifices area and power to achieve better timing performance, where the modified arbiter inside has an 11.54% better response time than the arbiter who used to be the best in an old comparison. ... Read more

Conversion from digital information to spike trains is needed for Spiking Neural Networks. Moreover, it is one of the most important steps for Spiking Neural Networks. This conversion could lead to much information loss depending on which encoding algorithm is used. Another major problem that can occur in a specific use-case is the limited bandwidth for the spikes that get generated through the encoding algorithm. In this thesis, we propose population Step Forward Encoding algorithm. This algorithm takes the signal encoding accuracy of Step Forward encoding algorithm and makes it into a population, generating multiple spike trains. This allows a higher threshold to encode a large part of the signal, increasing the efficiency. We show that population Step Forward Encoding algorithm doesn't just work good for the signal encoding accuracy, but also for the classification accuracy. Moreover, population Step Forward Encoding algorithm does not only have a high efficiency with a low spike count, it can also achieve higher efficiency with higher spike count. Thus, population Step Forward can make most use of a limited bandwidth of spikes. ... Read more