Aligning Object Detector Bounding Boxes with Human Preference

Conference Paper (2025)
Author(s)

Ombretta Strafforello (TNO, TU Delft - Electrical Engineering, Mathematics and Computer Science)

Osman Semih Kayhan (TU Delft - Electrical Engineering, Mathematics and Computer Science, Haga Ziekenhuis)

Oana Inel (Universitat Zurich)

Klamer Schutte (TNO)

Jan van Gemert (TU Delft - Electrical Engineering, Mathematics and Computer Science)

Research Group
Pattern Recognition and Bioinformatics
DOI related publication
https://doi.org/10.1007/978-3-031-92591-7_10 Final published version
More Info
expand_more
Publication Year
2025
Language
English
Research Group
Pattern Recognition and Bioinformatics
Pages (from-to)
151-168
Publisher
Springer
ISBN (print)
9783031925900
Event
European Conference on Computer Vision – ECCV 2024 (2024-09-29 - 2024-10-04), MiCo Milano, Milan, Italy
Page Views
76
Reuse Rights

Other than for strictly personal use, it is not permitted to download, forward or distribute the text or part of it, without the consent of the author(s) and/or copyright holder(s), unless the work is under an open content license such as Creative Commons.

Abstract

Previous work shows that humans tend to prefer large bounding boxes over small bounding boxes with the same IoU. However, we show here that commonly used object detectors predict large and small boxes equally often. In this work, we investigate how to align automatically detected object boxes with human preference and study whether this improves human quality perception. We evaluate the performance of three commonly used object detectors through a user study (N = 123). We find that humans prefer object detections that are upscaled with factors of 1.5 or 2, even if the corresponding AP is close to 0. Motivated by this result, we propose an asymmetric bounding box regression loss that encourages large over small predicted bounding boxes. Our evaluation study shows that object detectors fine-tuned with the asymmetric loss are better aligned with human preference and are preferred over fixed scaling factors. A qualitative evaluation shows that human preference might be influenced by some object characteristics, like object shape.

Files

978-3-031-92591-7_10.pdf
(pdf | 3.81 Mb)
- Embargo expired in 12-11-2025
– Personal use only – Dutch Copyright Act (Article 25fa)