Using an eye tracker to measure developer focus while writing unit test from within the IDE

Abstract

Background: Reading and understanding source
code and writing test cases are indispensable parts of routine for
modern software developers. Unit testing helps many developers
to detect and prevent bugs at an early moment and is a cost-
effective way to improve the effectiveness of a program. With the
availability of portable eye trackers, it has become possible to
measure what developer focus on while comprehending code.
Goal: This paper identifies the effort spent fixating between
production code, test code and feedback on test results from
the IDE.
Method: Participant developers are given production code and
are tasked with testing the code like they would in their routine
development situation. To enable a routine-like development
situation, developers have access to a popular Java IDE. The
gaze of the developer is tracked using an eye tracker. Fixations
within marked areas of interest (AOI) are calculated from the
gaze measurements using the i-Vt algorithm. Fixations are then
mapped to areas of interest which have been created from the
screen recording. From the fixation-to-AOI mapped results we
can learn what developers look at when developing unit tests.
Results: Developers spend the majority of their fixation time,
focusing on test cases (tc), followed by the production code (ic),
which encompasses the method under testing, the auxiliary code
and the code comments. Little time is spent fixating on the test
results, but this could be explained by the test results being only
the indication of feedback to a developer. When delving deeper in
understanding on why written test cases might fail, a developer
will look into the ic and tc areas again, as they provide the source
to find that understanding. Conclusions: We investigated where
developers spent their time while unit testing. From the results
we can conclude that on average developers spent slightly more
time fixating on the test code than on the production code. This
could imply that writing test cases requires active understanding
of the test case being written. The method used did not allow for
reliably identification of AOI sequences.