When, how, and why developers (do not) test in their IDEs

Conference paper (2015)

Authors

M.M. Beller Software Engineering - EEMCS
G. Gousios Radboud Universiteit Nijmegen, Software Engineering - EEMCS
A. Panichella Software Engineering - EEMCS
A.E. Zaidman Software Engineering - EEMCS
Research Group
Software Engineering (EEMCS) (TU Delft)
Copyright: © 2015 M.M. Beller, G. Gousios, A. Panichella, A.E. Zaidman
DOI
help_outline
https://doi.org/10.1145/2786805.2786843
Developer testing Field study Test- driven development (TDD) Testing effort Unit tests
More Info
expand_more

Published Date

30-08-2015

Language

English

Reuse Rights

Other than for strictly personal use, it is not permitted to download, forward or distribute the text or part of it, without the consent of the author(s) and/or copyright holder(s), unless the work is under an open content license such as Creative Commons.

Faculty

Electrical Engineering, Mathematics and Computer Science

Department

Software Technology

Research Group

Software Engineering

Abstract

The research community in Software Engineering and Software Testing in particular builds many of its contributions on a set of mutually shared expectations. Despite the fact that they form the basis of many publications as well as open-source and commercial testing applications, these common expectations and beliefs are rarely ever questioned. For example, Frederic Brooks' statement that testing takes half of the development time seems to have manifested itself within the community since he first made it in the "Mythical Man Month" in 1975. With this paper, we report on the surprising results of a large-scale field study with 416 software engineers whose development activity we closely monitored over the course of five months, resulting in over 13 years of recorded work time in their integrated development environments (IDEs). Our findings question several commonly shared assumptions and beliefs about testing and might be contributing factors to the observed bug proneness of software in practice: The majority of developers in our study does not test; developers rarely run their tests in the IDE; Test-Driven Development (TDD) is not widely practiced; and, last but not least, software developers only spend a quarter of their work time engineering tests, whereas they think they test half of their time.