Vulnerable road users account for more than 50% of traffic fatalities, and among these, pedestrians are the most susceptible to fatalities due to their distraction and misperception of other road users. To mitigate their plight, systems that warn drivers and pedestrians in case of a possible collision have been developed. Among systems that focus on the pedestrian's perspective, existing concepts are capable of predicting collisions but lack elements that monitor the visual attention of pedestrians. We address this gap by developing a gaze-based pedestrian warning system based on the Tobii Pro Glasses 2, a head-mounted eye-tracker. The system consists of: (1) a custom trained fast neural network (YOLO v4) on the KITTI object detection dataset that processes the video feed of the eye-tracker to detect approaching vehicles and (2) a module that uses the pedestrian's gaze to identify whether their attention falls on the closest moving vehicle that is approaching the pedestrian, both in real-time. If the pedestrian does not look at the approaching vehicle, they are given an auditory alert that warns them of a possible collision. In a pilot study conducted on a busy road in an urban environment, the system was evaluated under different pedestrian walking speeds and gaze behaviours to test the algorithm's robustness. The pilot study revealed that our system alerted the inattentive pedestrian with an accuracy of 67%. The mean vehicle detection accuracy and a mean moving vehicle identification accuracy from the pilot were 93% and 60%, respectively, a promising result given the use of only a mono camera. Despite the use of computer vision techniques, the system worked at an inference speed of 50 FPS due to the multi-processing capabilities of our algorithm. Our efforts are a first step in developing pedestrian warning systems based on eye-tracking technology to improve road safety in the future. The algorithm (Python-based) code used for this work has been made publicly available. ... Read more

Naturalistic driving research with a focus on trucks has been gaining momentum in the past decade. With the advancement in sensor technology and access to big data, it becomes possible to understand driver behaviour at a more fundamental level. This can assist in mitigating the impact trucks have on the environment while enhancing safety. Several studies have worked towards examining the predictability of driving behaviour through driver profiling (i.e. scoring a driver's behaviour or classifying drivers by assigning them different categories such as aggressive/non-aggressive). However, little research still focuses on the importance and impact of individual features used to develop these models. In the current study, an analysis of driving data from 1,727 trucks recorded over one year as part of a Dutch Field Operational Test (FOT) has been performed. This FOT, to date, has not been investigated in the published academic literature. Recent studies have analysed historical location data to assess risk associated with specific routes and environments. This is being used to provide notifications to drivers around work zones to mitigate the impact of accidents.

The current thesis extends the geo-specific analysis of (truck) driving data by analysing stability in truck driving behaviour with a focus on time and location (urban areas and motorways). Here correlation analysis has been used to explore stability. Correlational analysis elucidates that metrics such as the number of headway warnings, braking events and lane departure warnings are stable over space and time. A discussion reflects on the role of vehicle characteristics (i.e. mass and engine power) towards stability.

It is concluded that in the case of spatial stability: Mean point speed has higher stability on motorways than in urban areas. It has been determined that trucks with higher mass and lower engine power tend to have lower mean speed than the norm. Contrary to mean point speed, headway warnings show higher stability in urban areas than on motorways. Braking events and lane departure warnings exhibit high stability. Secondly, a strong correlation between (t and t+1) hours over the entire day is observed for temporal stability.

This research is a precursor to building generalised models for profiling drivers and assessing various driving patterns. An in-depth understanding of different driving patterns can help driver coaching companies better understand metrics when time and location are factored in before providing targeted feedback. Apart from this can also facilitate fleet management. The code for analysing this dataset is accessible online and may stimulate future researchers to explore this dataset further. ... Read more

The level of automation in vehicles is growing. But until all vehicles are completely automated, there will be a transition period where automated vehicles and human drivers coexist. Because these road users will coexist, it is necessary that automated vehicles understand human drivers and vice versa. This study aims to create a model that predicts human risk perception in different driving scenarios, to provide an understanding of the fundamental features of human threat perception while driving. The model created is a multi-criteria decision-making process that uses KITTI Vision Benchmark data as an input. This model is tested against the data gathered by an online survey, where 1918 participants answered the question: "How high is the risk on a scale from 0-10?" for 100 situations, chosen from the KITTI Vision Benchmark data. The survey response data is then compared to the model. Analysis of the survey data revealed that risk perception of driving situations is non-linear in the extremities of risk, showing that the input image are perceived as normally distributed instead of uniformly distributed. The comparison further shows that a model with features and weights solely based on literature is only slightly capable of predicting the risk of situations with a Pearson correlation coefficient with the survey responses of 0.28, whereas a model with feature weights optimised is moderately capable of predicting the risk of situations with a correlation coefficient of 0.57. However, multivariate regression is better capable of predicting risk with a correlation coefficient of 0.70, and shows that features and weights based on literature were not enough to establish an accurate model. The features that have the most impact on the result are the information about other road users' location and heading, the ego vehicle velocity, and the road type. Further research should focus on determining if speed is the cause of the perceived risk, or if it follows from other unknown predictors. Furthermore, an extension of the questions asked of the participants and the usage of videos instead of images can clarify the discrepancy between the literature-based model and the perceived risk of the participants. ... Read more

Problem statement. The introduction of automated vehicles (AVs) changes the role of the driver and may cause a lack of social interaction with pedestrians. This study proposes a concept where the AV is manoeuvre-based controlled via eye gaze, and the AV driver’s gaze is visualised for the driver and pedestrians. However, it was unknown if gaze-based AV control is a viable concept and how the AV’s yielding behaviour should depend on the eye driver’s gaze. Method. A two-agent virtual-reality-based experiment was conducted using two Varjo VR2-PRO head-mounted displays (HMDs). Seventeen pairs of participants (a pedestrian and a driver) each interacted in a road crossing scenario. The pedestrians’ task was to hold a button when they felt safe to cross the road, and the drivers’ task was to direct their gaze according to the instructions. Each session consisted of three blocks of 16 trials: the baseline block, in which the AV driver did not communicate with the pedestrian, and two other blocks in which the driver’s gaze was visualised, namely “gaze at the pedestrian to yield” (GTY) and “look away to yield” (LATY). The effectiveness of the interaction was examined using the pedestrians’ button presses. Acceptance and preference were measured using questionnaires. Results. Pedestrians showed the highest crossing performance and acceptance in the GTY mapping, followed by the LATY mapping and the baseline. The eye gaze visualisation caused pedestrians to spend more time looking at the AV; this effect was particularly dominant when the driver looked at the pedestrian. Conclusion. Gaze visualisation in combination with GTY mapping has the potential to be used as a communication tool for AVs at intersections until full automation of driving (SAE level 5) is technically feasible. ... Read more

Lane change decision-making is an important challenge for automated vehicles, urging the need for high performance algorithms that are able to handle complex traffic situations. Deep reinforcement learning (DRL), a machine learning method based on artificial neural networks, has recently become a popular choice for modelling the lane change decision-making process, outperforming various traditional rule-based models. So far, performance has often been expressed in terms of achieved average speed, absence of collisions or merging success rate. However, no studies have investigated how humans will react to the resulting behavior as potential occupants. This study addresses this research gap by validating a self-developed DRL-based lane changing model (trained using proximal policy optimization) from a technology acceptance perspective through an online crowdsourcing experiment. Participants (N=1085) viewed a random subset of 32 out of 120 videos of an automated vehicle driving on a three-lane highway with varying traffic densities featuring our proposed model or a baseline policy (i.e. a state-of-the-art rule-based model, MOBIL). They were tasked to press a response key if the decision-making was deemed undesirable and subsequently rated the vehicle's behavior along four acceptance constructs (performance expectancy, safety, human-likeness and reliability) on a scale of 1 to 5. Results showed that the proposed model caused a significantly lower amount of disagreements and was rated significantly higher on all four acceptance constructs compared to the baseline policy. Moreover, considerable differences between individual disagreement rates were observed for both models. Our findings offer prospects for the practical application of DRL-based lane change models in a use-case scenario, depending on the user. Further research is necessary to examine whether these observations hold in other (more complex) traffic situations. Additionally, we recommend combining DRL with other modelling techniques that allow for personalization of behavioral parameters, such as imitation learning. ... Read more

Automatic lane change identification can be used in warning systems that alert the driver when drifting off-lane, or in lane-specific navigation systems. Furthermore, they may help road authorities plan better road network design and infrastructure. Though lane-departure warning systems are already commercially available, these systems rely on on-board vehicle sensors. The availability of such systems can be increased if it became available to owners of smartphones or navigation devices. In this thesis, I propose and test several methods that rely solely on GPS data originating from portable devices (two smartphones, a GPS-equipped GoPro Max, and a USB GPS receiver), recorded during test rides on a Dutch highway (42.8 km) with a total of 64 lane changes. The methods rely on observing changes in the lateral offset between the GPS trajectories and the road geometry. The resulting identification accuracy of the best performing algorithm was achieved using the GoPro Max with an overall F1 score (harmonic mean of precision and recall) of 0.9 on the validation data using the signal of filtered projected lateral distance. It is concluded that GPS-equipped portable devices could be a suitable choice for identifying lane changes, provided there is improvement in the quality of GPS receiver chips with higher data collection frequency such as the ones used in GoPro Max. ... Read more

The presence of a passenger can affect driver safety positively or negatively. Therefore gaining a better understanding of the nature of this interaction with a passenger is crucial. Additionally, drivers and passengers acquire nearly identical visual information from the driving scene. As a result, synchronised behaviour may occur. The driver-passenger synchrony may provide additional information on driver-passenger interactions. This led to the aim of the current study: The identification of synchrony in head orientation and movement between drivers and passengers for various road types and cornering events. This study was based on real-world driving data. The head angle (orientation) and angular velocity (movement) of all three principal axes (pitch, roll, and yaw) were analysed for synchrony.

The following methods were applied for the identification of driver-passenger synchrony. The head angles of drivers and passengers were detected using OpenFace 2.0, a video-based pose estimation method. Second, the windowed cross-lagged correlation (WCLC), a linear approach for synchrony identification, was applied. Next, two distinct facets of synchrony were measured. The first facet was determining the frequency of synchrony derived by the peak-picking algorithm developed by Altmann (2013). The frequency of synchrony was summarised as the percentage of synchrony that occurred over the measured road segment. The second facet of synchrony studied was the strength of synchrony with the help of the peak-picking algorithm developed by Boker et al. (2002). This peak-picking algorithm searches for the maximum peak correlation at every window of the WCLC. Then all maximum peak correlations were averaged into a single mean peak correlation per road segment or cornering event. Since behavioural synchrony could appear by chance, the results of the peak-picking algorithms were compared to pseudosynchrony. The first hypothesis states that it is possible to distinguish detected synchrony from pseudosynchrony of driver’s and passenger’s head angle and angular velocity. Secondly, it is hypothesised that more synchrony is detected in the urban road type than outside built-up areas- and highway road types.

The results showed that differentiation between the detected synchrony and pseudosynchrony could be made for the analysis on the total route for almost all head angles and angular velocity for the frequency- and strength of synchrony. Furthermore, the analysis on the different road types, revealed that the identified synchrony could be differentiated from pseudosynchrony for almost all urban road segments. The findings on the other road types were mixed. It can be concluded from these research results that synchronisation has been identified for the urban road type. The road types where the detected synchrony was not significantly different from pseudosynchrony could indicate that there was no synchrony present or that the applied methods were unable to capture the driver-passenger synchrony. Synchrony identification in cornering events revealed no observable patterns. As a result, no conclusions on the cornering events could be formed. The second hypothesis could only be tested for one condition, since the identified synchrony could not be differentiated from pseudosynchrony for all the required road types for the second hypothesis. The result showed no significant difference between the urban road type compared to the built-up areas and highway road types for the pitch–angular velocity.

According to this study, drivers and passengers exhibited synchronisation in their head orientations and movements along particular road segments. However, more research is needed to truly comprehend the synchronised behaviour of drivers and passengers. A good place to start is for a study that looks into the relation between synchronous behaviour and the impact on driver safety as a result of the passenger's presence. ... Read more

Previous research showed that perceived risk is an important psychological determinant of road user behaviour and accident prevalence. However, little knowledge exists about how objective in-scene features affect a driver’s perceived risk in interactions with pedestrians. This crowdsourcing study tries to fill this research gap. A total of 1082 participants watched 35 out of a total of 86 dashcam videos featuring interactions with pedestrians extracted from the Pedestrian Intention Estimation (PIE) dataset. The videos contained annotations of pedestrian eye contact, crossing behaviour, GPS location, vehicle speed, and yielding rules. The distance between vehicle and pedestrian was manually added, and object counts (detected number of pedestrians, cyclists and vehicles) and respective sizes were added as an index of visual clutter. In each video, participants were asked to press a key on their keyboard and hold it as long as they felt a situation could become risky, and after each video rate perceived risk using a slider and answer whether the pedestrian had made eye contact. Videos in which the participant observed eye contact, increased perceived risk, suggesting that eye contact increases drivers’ vigilance. Videos with more visual clutter, and with higher vehicle speed were also associated with increased perceived risk. However, the causality of the correlation with vehicle speed can be questioned and may be mediated by the environment and whether crossing occurred. Videos in which yielding rules were absent, compared to videos in which they were present, did not affect perceived risk. This study is the first to investigate how pedestrians’ eye contact affects drivers’ perceived risk. The presented results could be useful in safe road design or be used as input for eHMI activation to enhance safety. ... Read more

Highly automated vehicles may lead to vehicle occupants getting distracted from driving-related tasks, so it may be necessary to introduce at new modalities to achieve effective pedestrian-vehicle communication. This research proposes using the lateral deviation of the automated vehicle within its lane as a method to communicate if it is going to yield to the pedestrian. In a crowdsourced experiment, videos containing an approaching automated vehicle were shown to participants. The effect of 1) levels of deviation (no deviation, deviation of 0.4 m, 0.8 m and 1.2 m), 2) direction of deviation (towards pedestrian, away from pedestrian), 3) vehicle behaviour (yielding, not yielding), 4) onset of deviation (onset at a distance of 50 m and 30 m from the pedestrian), and 5) intended vehicle path (vehicle going straight, vehicle taking a turn) were studied. A between-subjects design was used to assign participants (total N = 945; and after filtering, N = 638) randomly to one of 4 groups based on 1) the deviation direction-behaviour mapping (deviation towards pedestrian was ‘yielding’ and away was ‘non-yielding’, deviation towards pedestrian was ‘non-yielding’, away was ‘yielding’) and 2) instructions at the experiment start (not informed of the vehicle deviation, informed of the deviation). Each participant viewed 28 videos, and the task was to press and hold a key as long as it felt safe to cross. The results showed that 1) vehicle deviation to indicate yielding led to a statistically significant improvement in willingness to cross, for all four groups 2) the deviation level was significant when comparing the two extreme values (0.4 m and 1.2 m) for the yielding trials in two of four groups, 3) for one group there was a statistically significant difference in willingness to cross when the vehicle indicates that it intends to take a turn versus when it intends to go straight ahead. It is concluded that lateral deviation of the vehicle, when used to communicate yielding intent. affects the pedestrians’ willingness to cross the road. ... Read more

Various visual external human-machine interfaces (eHMIs) have been proposed that communicate the intent of automated vehicles (AVs) to vulnerable road users. However, there is no consensus on which eHMI concept is most suitable for intent communication. In nature, animals have evolved the ability to communicate intent via visual signals. Inspired by intent communication in nature, this paper investigated three novel and potentially intuitive eHMI designs that rely on posture, gesture, or colouration, respectively. In an online crowdsourcing study, 1523 participants viewed videos featuring a yielding or non-yielding AV with one of the three bio-inspired eHMIs, as well as a green/red lightbar eHMI, a walk/don’t walk text-based eHMI, and a baseline condition (i.e., no eHMI). Participants were asked to press and hold a key when they felt safe to cross and to answer rating questions. Together, these measures were used to determine the intuitiveness of the tested eHMIs. Results showed that the lightbar eHMI and text-based eHMI were more intuitive than the three bio-inspired eHMIs, which, in turn, were more intuitive than the baseline condition. An exception was the bio-inspired colouration eHMI, which attained a higher performance score than the other eHMIs when communicating ‘non-yielding’ before driving away from a standstill. Further research is necessary to examine whether these observations hold in more complex traffic situations and other eHMI designs. Additionally, we recommend combining features from different eHMIs, such as the full-body communication of the bio-inspired colouration eHMI with the colours of the lightbar eHMI. ... Read more