A Survey on Accelerating Sparse CNN Inference on GPUs

Bachelor Thesis (2022)
Author(s)

Q. Chen (TU Delft - Electrical Engineering, Mathematics and Computer Science)

Contributor(s)

Hasan Mohamed – Mentor (Universitat Zurich)

Shih Chii Liu – Mentor (Universitat Zurich)

N. Tömen – Mentor (TU Delft - Pattern Recognition and Bioinformatics)

M Zuñiga Zamalloa – Graduation committee member (TU Delft - Embedded Systems)

Faculty
Electrical Engineering, Mathematics and Computer Science
Copyright
© 2022 Qilin Chen
More Info
expand_more
Publication Year
2022
Language
English
Copyright
© 2022 Qilin Chen
Graduation Date
24-06-2022
Awarding Institution
Delft University of Technology
Project
['CSE3000 Research Project']
Programme
['Computer Science and Engineering']
Faculty
Electrical Engineering, Mathematics and Computer Science
Reuse Rights

Other than for strictly personal use, it is not permitted to download, forward or distribute the text or part of it, without the consent of the author(s) and/or copyright holder(s), unless the work is under an open content license such as Creative Commons.

Abstract

Convolutional neural networks (CNNs) are often pruned to achieve faster training and inference speed while also requiring less memory. Nevertheless, during computation, most modern GPUs cannot take advantage of the sparsity automatically, especially on networks with unstructured sparsity. Therefore, many libraries that exploit sparsity, have been proposed for accelerating CNN inference on GPUs. However, there is little research on systematically comparing them. In this paper, some state-of-the-art libraries for accelerating sparse CNN inference on GPUs are reviewed and benchmarked. Most of the libraries speedup the convolution and/or pooling operations by skipping zero calculations, therefore, they are able to perform sparse matrix calculations faster. However, many of them have hardware and software restrictions and are hard to integrate into a new model to perform end-to-end inference.

Files

License info not available