Fixing Continuous Integration Tests From Within the IDE With Contextual Information

Conference paper (2022)

Authors

Casper Boone Student
C.E. Brandt Software Engineering - EEMCS
A.E. Zaidman Software Engineering - EEMCS
Research Group
Software Engineering (EEMCS) (TU Delft)
Copyright: © 2022 Casper Boone, C.E. Brandt, A.E. Zaidman
DOI
help_outline
https://doi.org/10.1145/3524610.3527908
Software Testing Continuous Integration Developer Assistance IDE Plugin User Experiment
More Info
expand_more

Published Date

2022

Language

English

Reuse Rights

Other than for strictly personal use, it is not permitted to download, forward or distribute the text or part of it, without the consent of the author(s) and/or copyright holder(s), unless the work is under an open content license such as Creative Commons.

Faculty

Electrical Engineering, Mathematics and Computer Science

Department

Software Technology

Research Group

Software Engineering

Abstract

The most common reason for Continuous Integration (CI) builds to break is failing tests. When a build breaks, a developer often has to scroll through hundreds to thousands of log lines to find which test is failing and why. Finding the issue is a tedious process that relies on a developer's experience and increases the cost of software testing. We investigate how presenting different kinds of contextual information about CI builds in the Integrated Development Environment (IDE) impacts the time developers take to fix a broken build. Our IntelliJ plugin TESTAXIS surfaces additional information such as a unique view of the code under test that was changed leading up to the build failure. We conduct a user experiment and show that TESTAXIS helps developers fix failing tests 13.4% to 48.6% faster. The participants found the features of TESTAXIS useful and would incorporate it in their development workflow to save time. With TESTAXIS we set an important step towards removing the need to manually inspect build logs and bringing CI build results to the IDE, ultimately saving developers time.