Comparative Evaluation of Machine Learning Inference Machines on Edge-class Devices

Comparative Evaluation of Machine Learning Inference Machines on Edge-class Devices

Amanatidis, Petros (International Hellenic University)
Iosifidis, G. (TU Delft Embedded Systems)
Karampatzakis, Dimitris (International Hellenic University)

Vassilakopoulos, Michael Gr. (editor)
Karanikolas, Nikitas N. (editor)

2021

Computer science and engineering have evolved rapidly over the last decade offering innovative Machine Learning frameworks and high-performance hardware devices. Executing data analytics at the edge promises to transform the mobile computing paradigm by bringing intelligence next to the end user. However, it remains an open question to explore if, and to what extent, today's Edge-class devices can support ML frameworks and which is the best configuration for efficient task execution. This paper provides a comparative evaluation of Machine Learning inference machines on Edge-class compute engines. The testbed consists of two hardware compute engines (i.e., CPU-based Raspberry Pi 4 and Google Edge TPU accelerator) and two inference machines (i.e., TensorFlow-Lite and Arm NN). Through an extensive set of experiments in our bespoke testbed, we compared three setups using TensorFlow-Lite ML framework, in terms of accuracy, execution time, and energy efficiency. Based on the results, an optimized configuration of the workload parameters can increase accuracy by 10%, and in addition, the class of the Edge compute engine in combination with the inference machine affects execution time by 86% and power consumption by almost 145%.

Edge computing
Inference machine
Machine learning

http://resolver.tudelft.nl/uuid:a94878a8-b633-48d7-8fc6-a438ea0365ed

https://doi.org/10.1145/3503823.3503843

Association for Computing Machinery (ACM), New York

2022-05-26

978-1-4503-9555-7

Proceedings - 25th Pan-Hellenic Conference on Informatics, PCI 2021

25th Pan-Hellenic Conference on Informatics, PCI 2021, 2021-11-26 → 2021-11-28, Virtual, Online, Greece

Green Open Access added to TU Delft Institutional Repository 'You share, we take care!' - Taverne project https://www.openaccess.nl/en/you-share-we-take-care Otherwise as indicated in the copyright section: the publisher is the copyright holder of this work and the author uses the Dutch legislation to make this work public.

Institutional Repository

conference paper

© 2021 Petros Amanatidis, G. Iosifidis, Dimitris Karampatzakis