Sustainability of Edge AI at Scale

Abstract

Edge AI is an architectural deployment tactic that brings AI models closer to the user and data, relieving internet bandwidth usage and providing low latency and privacy. It remains unclear how this tactic performs at scale, since the distribution overhead could impact the total energy consumption. We identify four architectural scalability factors that could impact the energy consumption of AI: environment, optimisation, throughput, and overhead. The latter consists of downloading, verification, and updating the model over time. This work performs an empirical study on the sustainability of Edge AI compared to Cloud AI at scale in terms of energy consumption. For the environment variable, energy consumption measurement experiments are run on a cloud device and multiple edge devices, various quantized models for optimisation, and various throughput levels per hour. We simulate the distribution overhead and combine the results with the measurements to find the holistic energy efficiency of each architectural strategy. We find that all four variables impact energy consumption, but the main contributors are environment, throughput, and overhead. We observe that Edge AI is most energy-efficient in low-distribution, low-demand scenarios, whereas in high-distribution, high-demand scenarios Cloud AI is better optimised and outperforms Edge AI in energy efficiency. This means that developers depending on their use case and the project’s scalability need to consider these quality attributes for the most sustainable architectural solution.