Analyzing the Impact of Documentation on Performance Metrics in Different Continuous Integration Open-Source Projects
Is documentation important?
D. Rachev (TU Delft - Electrical Engineering, Mathematics and Computer Science)
S. Huang – Mentor (TU Delft - Software Engineering)
Sebastian Proksch – Mentor (TU Delft - Software Engineering)
Marco A. Zuñiga Zamalloa – Graduation committee member (TU Delft - Networked Systems)
More Info
expand_more
Other than for strictly personal use, it is not permitted to download, forward or distribute the text or part of it, without the consent of the author(s) and/or copyright holder(s), unless the work is under an open content license such as Creative Commons.
Abstract
Continuous Integration (CI) has become a standard practice for speeding up software development. However, the effect of comparatively slower artifacts, like documentation, on its performance is still unclear. Although documentation is often regarded as important, there is little data that connects documentation practices to key DevOps metrics. This study examines this relationship by looking at 670 open-source projects that use CI. We measured how documentation completeness, update frequency, and release notes affect delivery frequency, defect counts, and mean time to recovery. Our results show a "tipping point" where high documentation completeness greatly increases delivery frequency. We also found a "sweet spot" for update ratios between 20% and 55%, which relates to the lowest defect counts. On the other hand, we found no proof that long release notes improve recovery time. We conclude that the effectiveness of documentation depends more on quality and rhythm than on volume. This provides developers with practical, data-driven strategies to improve project performance.