The growing data demands of industry applications have led to Von Neumann computing systems increasingly facing a performance bottleneck in the compute-memory data path because of the memory wall problem. Computation-in-Memory (CiM) is a novel type of architecture that tries to address this by combining memory storage and computing in one CiM macro device to achieve a more efficient dataflow.
However, CiM-centric systems still require theoretical analysis through the use of analytical modeling due to a broad design space and practical hardware limitations. One unexplored modeling scope is multi-macro accelerators with a distributed-memory architecture, where inter-macro data messages are necessary to combine and redistribute results. In this thesis, we present 3DCLM - a latency model
for end-to-end execution of 3D convolutional layers on multi-macro distributed-memory accelerators. In the absence of a good real-life reference for model validation, we design CIMsim-distributed, an event-based simulator of the target CiM-centric accelerator system, which accounts for local and inter-
macro data movement as well as the effects of traffic congestion. We compare predicted and simulated latencies to assess 3DCLM’s ability to capture how key parameters influence overall execution and the relative importance of compute and data movement stages. The results show that 3DCLM can predict the overall impact of convolution input channels and CiM macro columns, but often fails to represent
the importance of memory movement costs, both absolute and proportional to total execution latency. Consequently, it can be practically useful for constrained Design Space Exploration of CiM-centric systems, and future iterations should extend its model of critical data movement operations. ... Read more

For a very long period of time, computing could meet the increasing demands of different applications due to the continued downscaling of transistors, which allowed data to be processed at a higher frequency. In the early 2000s, predictions about the physical limits and rising costs of continued down scaling prompted researchers to adopt alternative techniques to sustain performance improvements beyond frequency scaling. Among these, the most prevalent technique was the extraction and utilization of parallelism, which successfully extended performance scaling for more than a decade but has since begun to stagnate. Today, experts agree that specialized complementary hardware is crucial for further advancements. Computing-in-memory (CIM) accelerators are gaining traction as an innovative solution to the problems conventional computing is facing. While most CIM research is directed towards device, circuit, and architectural level challenges, it is also important to consider the challenges at the programming level. In this chapter, we first discuss the motivation behind CIM accelerators and why developing a programming model for them is essential. Next, we provide an overview of the challenges associated with developing a dedicated programming model for this emerging technology. Finally, we will outline the research direction of this thesis. ... Read more

Model compression techniques are crucial for reducing the deployment cost of large neural networks. Among these, depth pruning (removing layers/blocks) and width pruning (removing sections within layers) are essential for reducing memory footprint and inference latency. While various depth pruning methods exist, there is currently no unified approach that works consistently across different architectures. This work proposes a novel methodology that enables effective depth pruning across diverse networks, spanning from Large Language Models (LLMs) to Convolutional Neural Networks (CNNs). To achieve this unified compression, we introduce a hybrid technique that combines a primary depth pruning step with an auxiliary width pruning step. This combined approach specifically targets the elementary linear and convolutional layers, which constitute the foundational building blocks of diverse architectures. Our method demonstrates strong performance across different model types: First, on Llama2 LLM, the method demonstrates competitive results against state-of-the-art methods, offering a unique set of favorable trade-offs at varying compression levels. Second, on YOLOv5 CNNs the method enables removing up to 46% of parameters while preserving 80% of COCO dataset accuracy. Furthermore, our experiments reveal previously undiscovered pruning dependencies between network layers, providing new insights into the model compression landscape. ... Read more