Detecting darting out pedestrians with occlusion aware sensor fusion of radar and stereo camera

Journal article (2023)

Authors

A. Palffy Intelligent Vehicles - Mechanical, Maritime and Materials Engineering
J.F.P. Kooij Intelligent Vehicles - Mechanical, Maritime and Materials Engineering
D. Gavrila Intelligent Vehicles - Mechanical, Maritime and Materials Engineering
Research Group
Intelligent Vehicles (Mechanical, Maritime and Materials Engineering) (TU Delft)
Copyright: © 2023 A. Palffy, J.F.P. Kooij, D. Gavrila
DOI
help_outline
https://doi.org/10.1109/TIV.2022.3220435
Radar Intelligent vehicles Cameras Radar tracking Automobiles Radar detection Laser radar
More Info
expand_more

Published Date

2023

Language

English

Reuse Rights

Other than for strictly personal use, it is not permitted to download, forward or distribute the text or part of it, without the consent of the author(s) and/or copyright holder(s), unless the work is under an open content license such as Creative Commons.

Faculty

Mechanical, Maritime and Materials Engineering

Department

Cognitive Robotics

Research Group

Intelligent Vehicles

Abstract

Early and accurate detection of crossing pedestrians is crucial in automated driving in order to perform timely emergency manoeuvres. However, this is a difficult task in urban scenarios where pedestrians are often occluded (not visible) behind objects, e.g., other parked vehicles. We propose an occlusion aware fusion of stereo camera and radar sensors to address scenarios with crossing pedestrians behind such parked vehicles. Our proposed method adapts both the expected rate and properties of detections in different areas according to the visibility of the sensors. In our experiments on a real-world dataset, we show that the proposed occlusion aware fusion of radar and stereo camera detects the crossing pedestrians on average 0.26 seconds earlier than using the camera alone, and 0.15 seconds earlier than fusing the sensors without occlusion information. Our dataset containing 501 relevant recordings of pedestrians behind vehicles will be publicly available on our website for non-commercial, scientific use.