Scalable Incremental Building with Dynamic Task Dependencies

Scalable Incremental Building with Dynamic Task Dependencies

Konat, G.D.P. (TU Delft Programming Languages)
Erdweg, S.T. (TU Delft Programming Languages)
Visser, Eelco (TU Delft Programming Languages)

2018

Incremental build systems are essential for fast, reproducible software builds. Incremental build systems enable short feedback cycles when they capture dependencies precisely and selectively execute build tasks efficiently. A much overlooked feature of build systems is the expressiveness of the scripting language, which directly influences the maintainability of build scripts. In this paper, we present a new incremental build algorithm that allows build engineers to use a full-fledged programming language with explicit task invocation, value and file inspection facilities, and conditional and iterative language constructs. In contrast to prior work on incrementality for such programmable builds, our algorithm scales with the number of tasks affected by a change and is independent of the size of the software project being built. Specifically, our algorithm accepts a set of changed files, transitively detects and re-executes affected build tasks, but also accounts for new task dependencies discovered during building. We have evaluated the performance of our algorithm in a real-world case study and confirm its scalability.

scalable
incremental
build
dynamic task dependency

http://resolver.tudelft.nl/uuid:e167152d-540a-4afc-a6b4-636478c2eebe

https://doi.org/10.1145/3238147.3238196

Association for Computing Machinery (ACM), New York, NY

978-1-4503-5937-5

ASE 2018: Proceedings of the 33rd ACM/IEEE International Conference on Automated Software Engineering

ASE 2018, 2018-07-03 → 2018-07-07, Montpellier, France

Institutional Repository

conference paper

© 2018 G.D.P. Konat, S.T. Erdweg, Eelco Visser