Sustainability of Edge AI at Scale

Sustainability of Edge AI at Scale: An empirical study on the sustainability of Edge AI in terms of energy consumption

van der Noort, Rover (TU Delft Electrical Engineering, Mathematics and Computer Science)

Cruz, Luis (mentor)
Martínez-Fernández, Silverio (mentor)
van Deursen, A. (graduation committee)

Delft University of Technology

Computer Science

2024-05-08

Edge AI is an architectural deployment tactic that brings AI models closer to the user and data, relieving internet bandwidth usage and providing low latency and privacy. It remains unclear how this tactic performs at scale, since the distribution overhead could impact the total energy consumption. We identify four architectural scalability factors that could impact the energy consumption of AI: environment, optimisation, throughput, and overhead. The latter consists of downloading, verification, and updating the model over time. This work performs an empirical study on the sustainability of Edge AI compared to Cloud AI at scale in terms of energy consumption. For the environment variable, energy consumption measurement experiments are run on a cloud device and multiple edge devices, various quantized models for optimisation, and various throughput levels per hour. We simulate the distribution overhead and combine the results with the measurements to find the holistic energy efficiency of each architectural strategy. We find that all four variables impact energy consumption, but the main contributors are environment, throughput, and overhead. We observe that Edge AI is most energy-efficient in low-distribution, low-demand scenarios, whereas in high-distribution, high-demand scenarios Cloud AI is better optimised and outperforms Edge AI in energy efficiency. This means that developers depending on their use case and the project’s scalability need to consider these quality attributes for the most sustainable architectural solution.

Sustainable software engineering
Green AI
Quantization
Scalability
Edge-cloud architecture
Energy-aware software

http://resolver.tudelft.nl/uuid:78ac2503-426c-406b-a88f-8c0ed1abe0c0

2024-05-08

https://zenodo.org/records/11065939 Reproducability package https://github.com/rvandernoort/local-vs-cloud Repository of code used for this study

Student theses

master thesis

© 2024 Rover van der Noort