A Survey on Accelerating Sparse CNN Inference on GPUs

Bachelor thesis (2022)

Authors

Q. Chen Electrical Engineering, Mathematics and Computer Science

Contributors

Hasan Mohamed University of Zürich (supervisor 1)
Shih Chii Liu University of Zürich (supervisor 1)
N. Tömen Pattern Recognition and Bioinformatics - EEMCS (supervisor 1)
Marco Zuniga Embedded Systems - EEMCS (supervisor 2)
Faculty
Electrical Engineering, Mathematics and Computer Science
Copyright: © 2022 Qilin Chen
Convolutional Neural Networks (CNNs) Sparsity Accelerators Inference
More Info
expand_more

Published Date

24-06-2022

Language

English

Reuse Rights

Other than for strictly personal use, it is not permitted to download, forward or distribute the text or part of it, without the consent of the author(s) and/or copyright holder(s), unless the work is under an open content license such as Creative Commons.

Faculty

Electrical Engineering, Mathematics and Computer Science

Abstract

Convolutional neural networks (CNNs) are often pruned to achieve faster training and inference speed while also requiring less memory. Nevertheless, during computation, most modern GPUs cannot take advantage of the sparsity automatically, especially on networks with unstructured sparsity. Therefore, many libraries that exploit sparsity, have been proposed for accelerating CNN inference on GPUs. However, there is little research on systematically comparing them. In this paper, some state-of-the-art libraries for accelerating sparse CNN inference on GPUs are reviewed and benchmarked. Most of the libraries speedup the convolution and/or pooling operations by skipping zero calculations, therefore, they are able to perform sparse matrix calculations faster. However, many of them have hardware and software restrictions and are hard to integrate into a new model to perform end-to-end inference.