Humans Disagree With the IoU for Measuring Object Detector Localization Error

Humans Disagree With the IoU for Measuring Object Detector Localization Error

Strafforello, O. (TU Delft Pattern Recognition and Bioinformatics; TNO)
Rajasekart, Vanathi (Student TU Delft)
Kayhan, O.S. (TU Delft Pattern Recognition and Bioinformatics)
Inel, O. (TU Delft Web Information Systems; University of Zürich)
van Gemert, J.C. (TU Delft Pattern Recognition and Bioinformatics)

2022

The localization quality of automatic object detectors is typically evaluated by the Intersection over Union (IoU) score. In this work, we show that humans have a different view on localization quality. To evaluate this, we conduct a survey with more than 70 participants. Results show that for localization errors with the exact same IoU score, humans might not consider that these errors are equal, and express a preference. Our work is the first to evaluate IoU with humans and makes it clear that relying on IoU scores alone to evaluate localization errors might not be sufficient.

object detection
IoU
human preference

http://resolver.tudelft.nl/uuid:7bb5011f-3024-4fba-ad4f-84b1064568cd

https://doi.org/10.1109/ICIP46576.2022.9898043

IEEE, Piscataway

2023-07-01

978-1-6654-9621-6

Proceedings of the 2022 IEEE International Conference on Image Processing (ICIP)

2022 IEEE International Conference on Image Processing (ICIP), 2022-10-16 → 2022-10-19, Bordeaux, France

Green Open Access added to TU Delft Institutional Repository 'You share, we take care!' - Taverne project https://www.openaccess.nl/en/you-share-we-take-care Otherwise as indicated in the copyright section: the publisher is the copyright holder of this work and the author uses the Dutch legislation to make this work public.

Institutional Repository

conference paper

© 2022 O. Strafforello, Vanathi Rajasekart, O.S. Kayhan, O. Inel, J.C. van Gemert