Analyzing the State of Static Analysis

A Large-Scale Evaluation in Open Source Software

Conference Paper (2016)
Author(s)

M.M. Beller (TU Delft - Software Engineering)

Radjino Bholanath (Student TU Delft)

Shane McIntosh (McGill University)

A.E. Zaidman (TU Delft - Software Engineering)

Research Group
Software Engineering
Copyright
© 2016 M.M. Beller, R.M.R. Bholanath, Shane McIntosh, A.E. Zaidman
DOI related publication
https://doi.org/10.1109/SANER.2016.105
More Info
expand_more
Publication Year
2016
Language
English
Copyright
© 2016 M.M. Beller, R.M.R. Bholanath, Shane McIntosh, A.E. Zaidman
Research Group
Software Engineering
Pages (from-to)
470-481
ISBN (electronic)
978-1-5090-1855-0
Reuse Rights

Other than for strictly personal use, it is not permitted to download, forward or distribute the text or part of it, without the consent of the author(s) and/or copyright holder(s), unless the work is under an open content license such as Creative Commons.

Abstract

The use of automatic static analysis has been a software engineering best practice for decades. However, we still do not know a lot about its use in real-world software projects: How prevalent is the use of Automated Static Analysis Tools (ASATs) such as FindBugs and JSHint? How do developers use these tools, and how does their use evolve over time? We research these questions in two studies on nine different ASATs for Java, JavaScript, Ruby, and Python with a population of 122 and 168,214 open-source projects. To compare warnings across the ASATs, we introduce the General Defect Classification (GDC) and provide a grounded-theory-derived mapping of 1,825 ASAT-specific warnings to 16 top-level GDC classes. Our results show that ASAT use is widespread, but not ubiquitous, and that projects typically do not enforce a strict policy on ASAT use. Most ASAT configurations deviate slightly from the default, but hardly any introduce new custom analyses. Only a very small set of default ASAT analyses is widely changed. Finally, most ASAT configurations, once introduced, never change. If they do, the changes are small and have a tendency to occur within one day of the configuration's initial introduction.

Files

License info not available