Differentiable Transportation Pruning

Differentiable Transportation Pruning

Li, Yunqiang (Axelera AI)
van Gemert, J.C. (TU Delft Pattern Recognition and Bioinformatics)
Hoefler, Torsten (ETH Zürich)
Moons, Bert (Axelera AI)
Eleftheriou, Evangelos (Axelera AI)
Verhoef, Bram-Ernst (Axelera AI)

Ceballos, Cristina (editor)

2023

Deep learning algorithms are increasingly employed at the edge. However, edge devices are resource constrained and thus require efficient deployment of deep neural networks. Pruning methods are a key tool for edge deployment as they can improve storage, compute, memory bandwidth, and energy usage. In this paper we propose a novel accurate pruning technique that allows precise control over the output network size. Our method uses an efficient optimal transportation scheme which we make end-to-end differentiable and which automatically tunes the exploration-exploitation behavior of the algorithm to find accurate sparse sub-networks. We show that our method achieves state-of-the-art performance compared to previous pruning methods on 3 different datasets, using 5 different models, across a wide range of pruning ratios, and with two types of sparsity budgets and pruning granularities.

http://resolver.tudelft.nl/uuid:223db2c3-485e-4eff-8a38-d97ab41fc3ff

https://doi.org/10.1109/ICCV51070.2023.01555

IEEE, Piscataway

2024-07-15

979-8-3503-0719-1

Proceedings of the 2023 IEEE/CVF International Conference on Computer Vision (ICCV)

2023 IEEE/CVF International Conference on Computer Vision (ICCV), 2023-10-01 → 2023-10-06, Paris, France

Proceedings of the IEEE International Conference on Computer Vision, 1550-5499

Green Open Access added to TU Delft Institutional Repository 'You share, we take care!' - Taverne project https://www.openaccess.nl/en/you-share-we-take-care Otherwise as indicated in the copyright section: the publisher is the copyright holder of this work and the author uses the Dutch legislation to make this work public.

Institutional Repository

conference paper

© 2023 Yunqiang Li, J.C. van Gemert, Torsten Hoefler, Bert Moons, Evangelos Eleftheriou, Bram-Ernst Verhoef