Humans Disagree With the IoU for Measuring Object Detector Localization Error

Conference paper (2022)

Authors

O. Strafforello TNO, Pattern Recognition and Bioinformatics - EEMCS
O.S. Kayhan Pattern Recognition and Bioinformatics - EEMCS
O. Inel Web Information Systems - EEMCS , University of Zürich
J.C. van Gemert Pattern Recognition and Bioinformatics - EEMCS
Research Group
Pattern Recognition and Bioinformatics (EEMCS) (TU Delft)
Copyright: © 2022 O. Strafforello, Vanathi Rajasekart, O.S. Kayhan, O. Inel, J.C. van Gemert
DOI: https://doi.org/10.1109/ICIP46576.2022.9898043
More Info
expand_more

Published Date

2022

Language

English

Reuse Rights

Other than for strictly personal use, it is not permitted to download, forward or distribute the text or part of it, without the consent of the author(s) and/or copyright holder(s), unless the work is under an open content license such as Creative Commons.

Faculty

Electrical Engineering, Mathematics and Computer Science

Department

Intelligent Systems

Research Group

Pattern Recognition and Bioinformatics

Abstract

The localization quality of automatic object detectors is typically evaluated by the Intersection over Union (IoU) score. In this work, we show that humans have a different view on localization quality. To evaluate this, we conduct a survey with more than 70 participants. Results show that for localization errors with the exact same IoU score, humans might not consider that these errors are equal, and express a preference. Our work is the first to evaluate IoU with humans and makes it clear that relying on IoU scores alone to evaluate localization errors might not be sufficient.

Files

Humans_Disagree_With_the_IoU_f... (.pdf)
(.pdf | 5.66 Mb)
- Embargo expired in 01-07-2023