Test Behaviors, Not Methods! Detecting Tests Obsessed by Methods

Conference Paper (2026)
Author(s)

Andre Hora (Universidade Federal de Minas Gerais)

Andy Zaidman (TU Delft - Electrical Engineering, Mathematics and Computer Science)

Department
Software Technology
DOI related publication
https://doi.org/10.1145/3794763.3794791 Final published version
More Info
expand_more
Publication Year
2026
Language
English
Department
Software Technology
Pages (from-to)
438-442
Publisher
ACM
ISBN (electronic)
9798400724824
Event
34th IEEE/ACM International Conference on Program Comprehension, ICPC 2026 (2026-04-12 - 2026-04-13), Rio de Janeiro, Brazil
Page Views
44
Reuse Rights

Other than for strictly personal use, it is not permitted to download, forward or distribute the text or part of it, without the consent of the author(s) and/or copyright holder(s), unless the work is under an open content license such as Creative Commons.

Abstract

Best testing practices state that tests should verify a single functionality or behavior of the system. Tests that verify multiple behaviors are harder to understand, lack focus, and are more coupled to the production code. An attempt to identify this issue is the test smell Eager Test, which aims to capture tests that verify too much functionality based on the number of production method calls. Unfortunately, prior research suggests that counting production method calls is an inaccurate measure, as these calls do not reliably serve as a proxy for functionality. We envision a complementary solution based on runtime analysis: we hypothesize that some tests that verify multiple behaviors will likely cover multiple paths of the same production methods. Thus, we propose a novel test smell named Test Obsessed by Method, a test method that covers multiple paths of a single production method. We provide an initial empirical study to explore the presence of this smell in 2,054 tests provided by 12 test suites of the Python Standard Library. (1) We detect 44 Tests Obsessed by Methods in 11 of the 12 test suites. (2) Each smelly test verifies a median of two behaviors of the production method. (3) The 44 smelly tests could be split into 118 novel tests. (4) 23% of the smelly tests have code comments recognizing that distinct behaviors are being tested. We conclude by discussing benefits, limitations, and further research.