EDGETUNE

EDGETUNE: Inference-Aware Multi-Parameter Tuning

Rocha, Isabelly (University of Neuchâtel)
Felber, Pascal (University of Neuchâtel)
Schiavoni, Valerio (University of Neuchâtel)
Chen, Lydia Y. (TU Delft Dataintensive Systems)

2022

Deep Neural Networks (DNNs) have demonstrated impressive performance on many machine-learning tasks such as image recognition and language modeling, and are becoming prevalent even on mobile platforms. Despite so, designing neural architectures still remains a manual, time-consuming process that requires profound domain knowledge. Recently, Parameter Tuning Servers have gathered the attention o industry and academia. Those systems allow users from all domains to automatically achieve the desired model accuracy for their applications. However, although the entire process of tuning and training models is performed solely to be deployed for inference, state-of-the-art approaches typically ignore system-oriented and inference-related objectives such as runtime, memory usage, and power consumption. This is a challenging problem: besides adding one more dimension to an already complex problem, the information about edge devices available to the user is rarely known or complete. To accommodate all these objectives together, it is crucial for tuning system to take a holistic approach to parameter tuning and consider all levels of parameters simultaneously into account. We present EdgeTune, a novel inference-aware parameter tuning server. It considers the tuning of parameters in all levels backed by an optimization function capturing multiple objectives. Our approach relies on inference estimated metrics collected from our emulation server running asynchronously from the main tuning process. The latter can then leverage the inference performance while still tuning the model. We propose a novel one-fold tuning algorithm that employs the principle of multi-fidelity and simultaneously explores multiple tuning budgets, which the prior art can only handle as suboptimal case of single type of budget. EdgeTune outputs inference recommendations to the user while improving tuning time and energy by at least 18\% and 53\% when compared to the baseline.

deep neural networks
inference
training
tuning

http://resolver.tudelft.nl/uuid:a91b9abf-1e64-4bf4-859f-e8d236985374

https://doi.org/10.1145/3528535.3533273

Association for Computing Machinery (ACM)

2023-07-01

9781450393409

Middleware 2022 - Proceedings of the 23rd ACM/IFIP International Middleware Conference

23rd ACM/IFIP International Middleware Conference, Middleware 2022, 2022-11-07 → 2022-11-11, Quebec, Canada

Middleware 2022 - Proceedings of the 23rd ACM/IFIP International Middleware Conference

Green Open Access added to TU Delft Institutional Repository 'You share, we take care!' - Taverne project https://www.openaccess.nl/en/you-share-we-take-care Otherwise as indicated in the copyright section: the publisher is the copyright holder of this work and the author uses the Dutch legislation to make this work public.

Institutional Repository

conference paper

© 2022 Isabelly Rocha, Pascal Felber, Valerio Schiavoni, Lydia Y. Chen