Hallucination In Object Detection

A Study In Visual Part VERIFICATION

Conference paper (2021)

Authors

O.S. Kayhan Pattern Recognition and Bioinformatics - EEMCS
Bart Vredebregt Aiir Innovations
J.C. van Gemert Aiir Innovations, Pattern Recognition and Bioinformatics - EEMCS
Research Group
Pattern Recognition and Bioinformatics (EEMCS) (TU Delft)
Object detection Visual part verification
More Info
expand_more

Published Date

2021

Language

English

Reuse Rights

Other than for strictly personal use, it is not permitted to download, forward or distribute the text or part of it, without the consent of the author(s) and/or copyright holder(s), unless the work is under an open content license such as Creative Commons.

Faculty

Electrical Engineering, Mathematics and Computer Science

Department

Intelligent Systems

Research Group

Pattern Recognition and Bioinformatics

Abstract

We show that object detectors can hallucinate and detect missing objects; potentially even accurately localized at their expected, but non-existing, position. This is particularly problematic for applications that rely on visual part verification: detecting if an object part is present or absent. We show how popular object detectors hallucinate objects in a visual part verification task and introduce the first visual part verification dataset: DelftBikes
1, which has 10,000 bike photographs, with 22 densely annotated parts per image, where some parts may be missing. We explicitly annotated an extra object state label for each part to reflect if a part is missing or intact. We propose to evaluate visual part verification by relying on recall and compare popular object detectors on DelftBikes.