The Energy Impact of Batch Testing in Continuous Integration
An Empirical Study of Static and Dynamic Batching Strategies
Máté Oszkó (Student TU Delft)
Xutong Liu (TU Delft - Electrical Engineering, Mathematics and Computer Science)
Andy Zaidman (TU Delft - Electrical Engineering, Mathematics and Computer Science)
Carolin Brandt (TU Delft - Electrical Engineering, Mathematics and Computer Science)
More Info
expand_more
Other than for strictly personal use, it is not permitted to download, forward or distribute the text or part of it, without the consent of the author(s) and/or copyright holder(s), unless the work is under an open content license such as Creative Commons.
Abstract
Continuous integration pipelines execute automated tests on every commit, consuming substantial energy. Batch testing, which groups multiple commits into a single test run, has been shown to reduce test executions in simulation studies, but no prior work has measured whether these reductions translate into actual energy savings. This study measures CPU package energy consumption of CI builds under a baseline run-all approach and two batching strategies (BatchStop4 and linear-4 lwd) across eight open-source Java projects. BatchStop4 achieves energy savings between 57% and 88%, while linear-4 lwd achieves savings between 59% and 94%. Energy savings correlate almost perfectly with time savings (r >0.99), indicating that batching reduces energy primarily by shortening total execution time. Baseline failure rate strongly predicts achievable savings, while CPU utilisation shows no significant relationship. These findings provide empirical evidence that batch testing effectively reduces the environmental footprint of continuous integration, particularly for projects with low failure rates.