A controlled experiment for program comprehension through trace visualization

Report (2009)
Department
Software Computer Technology (EEMCS) (TU Delft)
Copyright: IEEE
More Info
expand_more

Published Date

31-12-2009

Reuse Rights

Other than for strictly personal use, it is not permitted to download, forward or distribute the text or part of it, without the consent of the author(s) and/or copyright holder(s), unless the work is under an open content license such as Creative Commons.

Source:

Technical Report Series TUD-SERG-2009-001

Faculty

Electrical Engineering, Mathematics and Computer Science

Department

Software Computer Technology

Abstract

Accepted version for publication in IEEE Transactions on Software Engineering. Software maintenance activities require a sufficient level of understanding of the software at hand that unfortunately is not always readily available. Execution trace visualization is a common approach in gaining this understanding, and among our own efforts in this context is EXTRAVIS, a tool for the visualization of large traces. While many such tools have been evaluated through case studies, there have been no quantitative evaluations to the present day. This paper reports on the first controlled experiment to quantitatively measure the added value of trace visualization for program comprehension. We designed eight typical tasks aimed at gaining an understanding of a representative subject system, and measured how a control group (using the Eclipse IDE) and an experimental group (using both Eclipse and EXTRAVIS) performed in terms of correctness and time spent. The results are statistically significant in both regards, showing a 22% decrease in time needed for the given tasks, and a 43% increase in correctness of the results for the group using trace visualization.