Design of a real-time computer-vision and gaze-based pedestrian warning system

Master thesis (2022)

Authors

L. Thirunavukkarasu Kumaaravelu Mechanical Engineering

Contributors

J.C.F. de Winter Human-Robot Interaction - Mechanical, Maritime and Materials Engineering (mentor)
D. Dodou Medical Instruments & Bio-Inspired Technology - Mechanical, Maritime and Materials Engineering (mentor)
V. Onkhar Human-Robot Interaction - Mechanical, Maritime and Materials Engineering (mentor)
P. Bazilinskyy Human-Robot Interaction - Mechanical, Maritime and Materials Engineering (mentor)
W. Tabone Human-Robot Interaction - Mechanical, Maritime and Materials Engineering (mentor)
Y.B. Eisma Human-Robot Interaction - Mechanical, Maritime and Materials Engineering (graduation committee member)
Faculty
Mechanical Engineering, Mechanical Engineering
Copyright: © 2022 Lokkeshver Thirunavukkarasu Kumaaravelu
More Info
expand_more

Published Date

31-05-2022

Language

English

Reuse Rights

Other than for strictly personal use, it is not permitted to download, forward or distribute the text or part of it, without the consent of the author(s) and/or copyright holder(s), unless the work is under an open content license such as Creative Commons.

Faculty

Mechanical Engineering

Abstract

Vulnerable road users account for more than 50% of traffic fatalities, and among these, pedestrians are the most susceptible to fatalities due to their distraction and misperception of other road users. To mitigate their plight, systems that warn drivers and pedestrians in case of a possible collision have been developed. Among systems that focus on the pedestrian's perspective, existing concepts are capable of predicting collisions but lack elements that monitor the visual attention of pedestrians. We address this gap by developing a gaze-based pedestrian warning system based on the Tobii Pro Glasses 2, a head-mounted eye-tracker. The system consists of: (1) a custom trained fast neural network (YOLO v4) on the KITTI object detection dataset that processes the video feed of the eye-tracker to detect approaching vehicles and (2) a module that uses the pedestrian's gaze to identify whether their attention falls on the closest moving vehicle that is approaching the pedestrian, both in real-time. If the pedestrian does not look at the approaching vehicle, they are given an auditory alert that warns them of a possible collision. In a pilot study conducted on a busy road in an urban environment, the system was evaluated under different pedestrian walking speeds and gaze behaviours to test the algorithm's robustness. The pilot study revealed that our system alerted the inattentive pedestrian with an accuracy of 67%. The mean vehicle detection accuracy and a mean moving vehicle identification accuracy from the pilot were 93% and 60%, respectively, a promising result given the use of only a mono camera. Despite the use of computer vision techniques, the system worked at an inference speed of 50 FPS due to the multi-processing capabilities of our algorithm. Our efforts are a first step in developing pedestrian warning systems based on eye-tracking technology to improve road safety in the future. The algorithm (Python-based) code used for this work has been made publicly available.