Print Email Facebook Twitter Towards lossless binary convolutional neural networks using piecewise approximation Title Towards lossless binary convolutional neural networks using piecewise approximation Author Zhu, B. (TU Delft Computer Engineering) Al-Ars, Z. (TU Delft Computer Engineering) Pan, W. (TU Delft Robot Dynamics) Contributor De Giacomo, Giuseppe (editor) Catala, Alejandro (editor) Dilkina, Bistra (editor) Milano, Michela (editor) Barro, Senen (editor) Bugarin, Alberto (editor) Lang, Jerome (editor) Date 2020 Abstract Binary Convolutional Neural Networks (CNNs) can significantly reduce the number of arithmetic operations and the size of memory storage, which makes the deployment of CNNs on mobile or embedded systems more promising. However, the accuracy degradation of single and multiple binary CNNs is unacceptable for modern architectures and large scale datasets like ImageNet. In this paper, we proposed a Piecewise Approximation (PA) scheme for multiple binary CNNs which lessens accuracy loss by approximating full precision weights and activations efficiently, and maintains parallelism of bitwise operations to guarantee efficiency. Unlike previous approaches, the proposed PA scheme segments piece-wisely the full precision weights and activations, and approximates each piece with a scaling coefficient. Our implementation on ResNet with different depths on ImageNet can reduce both Top-1 and Top-5 classification accuracy gap compared with full precision to approximately 1.0%. Benefited from the binarization of the downsampling layer, our proposed PA-ResNet50 requires less memory usage and two times Flops than single binary CNNs with 4 weights and 5 activations bases. The PA scheme can also generalize to other architectures like DenseNet and MobileNet with similar approximation power as ResNet which is promising for other tasks using binary convolutions. The code and pretrained models will be publicly available. To reference this document use: http://resolver.tudelft.nl/uuid:b760b9cc-c61b-4954-9033-15a426f5b550 DOI https://doi.org/10.3233/FAIA200286 Publisher IOS Press, Amsterdam ISBN 978-1-64368-100-9 Source ECAI 2020: 24TH EUROPEAN CONFERENCE ON ARTIFICIAL INTELLIGENCE: 24th European Conference on Artificial Intelligence, including 10th Conference on Prestigious Applications of Artificial Intelligence, PAIS 2020 - Proceedings, 325 Event 24th European Conference on Artificial Intelligence, ECAI 2020, including 10th Conference on Prestigious Applications of Artificial Intelligence, PAIS 2020, 2020-08-29 → 2020-09-08, Santiago de Compostela, Online, Spain Series Frontiers in Artificial Intelligence and Applications, 0922-6389, 325 Part of collection Institutional Repository Document type book chapter Rights © 2020 B. Zhu, Z. Al-Ars, W. Pan Files PDF FAIA_325_FAIA200286.pdf 468.29 KB Close viewer /islandora/object/uuid:b760b9cc-c61b-4954-9033-15a426f5b550/datastream/OBJ/view