The present study surveyed actual extensive users of SAE Level 2 partially automated cars to investigate how driver’s characteristics (i.e., socio-demographics, driving experience, personality), system performance, perceived safety, and trust in partial automation influence use of partial automation. 81% of respondents stated that they use their automated car with speed (ACC) and steering assist (LKA) at least 1–2 times a week, and 84 and 92% activate LKA and ACC at least occasionally. Respondents positively rated the performance of Adaptive Cruise Control (ACC) and Lane Keeping Assistance (LKA). ACC was rated higher than LKA and detection of lead vehicles and lane markings was rated higher than smooth control for ACC and LKA, respectively. Respondents reported to primarily disengage (i.e., turn off) partial automation due to a lack of trust in the system and when driving is fun. They rarely disengaged the system when they noticed they become bored or sleepy. Structural equation modelling revealed that trust had a positive effect on driver’s propensity for secondary task engagement during partially automated driving, while the effect of perceived safety was not significant. Regarding driver’s characteristics, we did not find a significant effect of age on perceived safety and trust in partial automation. Neuroticism negatively correlated with perceived safety and trust, while extraversion did not impact perceived safety and trust. The remaining three personality dimensions ‘openness’, ‘conscientiousness’, and ‘agreeableness’ did not form valid and reliable scales in the confirmatory factor analysis, and could thus not be subjected to the structural equation modelling analysis. Future research should re-assess the suitability of the short 10-item scale as measure of the Big-Five personality traits, and investigate the impact on perceived safety, trust, use and use of automation.@en

We present a novel method for vehicle-pedestrian path prediction that takes into account the awareness of the driver and the pedestrian towards each other. The method jointly models the paths of vehicle and pedestrian within a single Dynamic Bayesian Network (DBN). In this DBN, sub-graphs model the environment and entity-specific context cues of the vehicle and pedestrian (incl. awareness), which affect their future motion and allow to increase the prediction horizon. These sub-graphs share a latent state which models whether vehicle and pedestrian are on collision course; this accounts for a certain degree of motion coupling. The method was validated with real-world data obtained by onboard vehicle sensing (stereo vision, GNSS and proprioceptive). Data consist of 93 vehicle and pedestrian encounters, spanning various awareness conditions and dynamic characteristics of the participants. In ablation studies, we quantify the benefits of various components of our proposed DBN model for path prediction and collision risk estimation. Results show that at a prediction horizon of 1.5 s, context aware models outperform context-agnostic models in path prediction for scenarios with a dynamics change, while performing similarly otherwise. Results further indicate that driver attention aware models improve collision risk estimation compared to driver-agnostic models.

@en

To encourage appropriate use of driving automation, we need to understand and monitor driver's trust and risk perception. We examined (1) how trust and perceived risk are affected by automation, driving conditions and experience and (2) how well perceived risk can be inferred from behaviour and physiology at three levels: over traffic conditions, aggregated risk events, and individual risk events. 30 users with and without automation experience drove a Toyota Corolla with driving support. Safety attitude, subjective ratings, behaviour and physiology were examined. Driving support encouraged a positive safety attitude and active driver involvement. It reduced latent hazards while maintaining saliently perceived risks. Drivers frequently overruled lane centring (3.1 times/minute) and kept their feet on or above the pedals using ACC (65.8% of time). They comfortably used support on curvy motorways and monotonic and congested highways but less in unstable traffic and on roundabouts. They trusted the automation 65.4%, perceived 36.0% risk, acknowledged the need to monitor and would not engage in more secondary tasks than during manual driving. Trust-in situation reduced 2.0% when using automation. It was 8.2% higher than trust-in-automation, presumably due to driver self-confidence. Driving conditions or conflicts between driver and automation did not affect trust-in-automation. At the traffic condition level, physiology showed weak and partially counter-intuitive effects. For aggregated risk events, skin conductance had the clearest response but was discernible from baseline in < 50%. Pupil dilation and heart rate only increased with strong braking and active lane departure assist. For individual risk events, a CNN classifier could not identify risk events from physiology. We conclude that GSR, heart rate and pupil dilation respond to perceived risk, but lack specificity to monitor it on individual events.

@en

This study reports usage of supervised automation and driver attention from longitudinal naturalistic driving observations. Automation inexperienced drivers were provided with instrumented vehicles with adaptive cruise control (ACC) and lane keeping (LK) features (SAE level 2). Data was collected comparing one month of driving without support to two months where drivers were instructed to use automation as desired. On highways, level 2 automation was used respectively 63% and 57% of the time by Tesla and BMW users, with peak usage during slow stop-and-go traffic (0–30 km/h) and higher speeds (>80 km/h). On roads with speed limits below 70 km/h, automation was used less than 8%, and use on urban roads was incidental rather than habitual. Automation usage increased with time in trip, but no clear time of day effects were found. Head pose data could not classify driver attention, and we recommend gaze tracking in future studies. Head pose deviation was selected as alternative indicator for monitoring activity. Comparing among forms of automation usage on the highway, head heading deviation was smallest during ACC use, but did not differ between automation and baseline manual driving. Head heading deviation during manual driving was smaller in the baseline than the experimental phase, which suggests that motives for manual highway driving may be attention related. Automation usage did not change much over the first 12 weeks of the experimental condition, and there were no longitudinal changes in head pose deviation.

@en

Perceived risk and trust are crucial for user acceptance of driving automation. In this study, we identify important predictors of perceived risk and trust in a driving simulator experiment and develop models through stepwise regression to predict event-based changes in perceived risk and trust. 25 participants were tasked to monitor SAE Level 2 driving automation (ACC + LC) while experiencing merging and hard braking events with varying criticality on a motorway. Perceived risk and trust were rated verbally after each event, and continuous perceived risk, pupil diameter and ECG signals were explored as possible indictors for perceived risk and trust. The regression models show that relative motion with neighbouring road users accounts for most perceived risk and trust variations, and no difference was found between hard braking with merging and hard braking without merging. Drivers trust the automation more in the second exposure to events. Our models show modest effects of personal characteristics: experienced drivers are less sensitive to risk and trust the automation more, while female participants perceive more risk than males. Perceived risk and trust highly correlate and have similar determinants. Continuous perceived risk accurately reflects participants’ verbal post-event rating of perceived risk; the use of brakes is an effective indicator of high perceived risk and low trust, and pupil diameter correlates to perceived risk in the most critical events. The events increased heart rate, but we found no correlation with event criticality. The prediction models and the findings on physiological measures shed light on the event-based dynamics of perceived risk and trust and can guide human-centred automation design to reduce perceived risk and enhance trust.

@en

The present online study surveyed drivers of SAE Level 2 partially automated cars on automation use and attitudes towards automation. Respondents reported high levels of trust in their partially automated cars to maintain speed and distance to the car ahead (M = 4.41), and to feel safe most of the time (M = 4.22) on a scale from 1 to 5. Respondents indicated to always know when the car is in partially automated driving mode (M = 4.42), and to monitor the performance of their car most of the time (M = 4.34). A low rating was obtained for engaging in other activities while driving the partially automated car (M = 2.27). Partial automation did, however, increase reported engagement in secondary tasks that are already performed during manual driving (i.e., the proportion of respondents reporting to observe the landscape, use the phone for texting, navigation, music selection and calls, and eat during partially automated driving was higher in comparison to manual driving). Unsafe behaviour was rare with 1% of respondents indicating to rarely monitor the road, and another 1% to sleep during partially automated driving. Structural equation modeling revealed a strong, positive relationship between perceived safety and trust (β = 0.69, p = 0.001). Performance expectancy had the strongest effects on automation use, followed by driver engagement, trust, and non-driving related task engagement. Perceived safety interacted with automation use through trust. We recommend future research to evaluate the development of perceived safety and trust in time, and revisit the influence of driver engagement and non-driving related task engagement, which emerged as new constructs related to trust in partial automation.

@en

Problem Definition According to the World Health Organization, traffic injuries have become the eighth cause of death and the leading cause among children and young adults. Human error, and in particular perceptual error, is among the most frequently reported causes of road fatalities. The desire to reduce traffic fatalities has led to the development of automated driving, which promises revolutionary advances in driver safety, traffic capacity and driver convenience. Since true autonomy in mixed traffic has not yet been achieved, today's automated vehicles require the driver to continuously supervise the automation and to capably intervene when necessary. However, simulator studies and experiences from disciplines such as aviation and factories have demonstrated that humans are generally ill-equipped to monitor automation for longer periods. This raises the concern that partial automation may harm rather than help traffic safety if not designed to adequately support the drivers in their supervisory tasks. Research objectives To address this concern, further insights are needed in how drivers monitor automation in complex real-world traffic, and how their behaviour and performance change with long-term automated driving experience. This dissertation sets out to investigate how real-world automation changes the availability of attentional resources, to establish where and how drivers use automation in naturalistic conditions, and evaluate how these change with experience. While these objectives investigate periods of automated driving, vehicles with automated driving functionalities will often be driven manually, when outside the operational design domain or at the driver’s preference. In these conditions, the available automation may still outperform the driver on particular tasks, such as detecting and tracking surrounding road users without bias or distraction. This dissertation therefore also contributes to the search for ways in which automation can provide meaningful support to the traffic monitoring task in manual and supervised driving. To evaluate if and when supervised automated driving negatively affects the driver’s ability to monitor, mental workload is evaluated in a Tesla model S on public roads (Chapter 2). Voluntary automation use and attention are examined in a naturalistic driving study on public roads (Chapter 3). To evaluate the effect of experience with automated driving, Chapter 2 compares drivers with and without prior automation use, whereas Chapter 3 examines how behaviour changes over a two-month period, compared to one month of manual driving. Two studies are performed to examine how driving automation can support the driver with the monitoring task, for which an instrumented vehicle was extended with cameras which track the driver’s gaze and associate it to surrounding road users as detected by the vehicle perception. The first study (Chapter 4) investigates how well gaze behaviour can indicate driver awareness toward individual road users, and proposes a recognition task to obtain a ground truth for awareness of multiple other road-users. The second study (Chapter 5) evaluates if driver gaze and head pose can provide earlier predictions for emergency alerting and intervention systems. A crossing pedestrian collision risk prediction system is used as a case study where gaze and contextual cues are evaluated in their contribution to path and risk prediction using a dynamic Bayesian network. Findings & recommendations Chapter 2 found that workload differed between roads with high and low traffic complexity, both for manual and automated driving, which indicates that drivers remain sensitive to changes in task demand while supervising automated driving. Drivers with prior experience in automated driving perceived a lower workload while supervising automation compared to manual driving. No workload difference was perceived for first-time users. In contrast, attentional demand as measured by a detection-response task was higher during automation use compared to manual driving regardless of experience. This indicates that monitoring automation (SAE2) requires more mental capacity compared to manual driving, which suggests that in contrast to a wide range of studies, SAE2 can increase workload. Supervising automation may therefore be beneficial for driver attention, but perception of workload during supervision may be too low for this to occur naturally. Future work should consider calibrating workload perception and system limitation understanding rather than actual task demand to encourage attentive supervision. Chapter 3 shows that automation is mostly used on road types generally considered suitable for automated driving with only incidental use on urban roads. This suggests that users are adhering to the operational design domain of these vehicles. On highways, automation is used at all speeds, but less during short periods of slow driving. No time-in-drive, time-of-day or experience effects were found for automation use. On the highway, head pose deviation was smaller during automation use compared to manual driving but tended to increase over the first six weeks of use, which may indicate a change in monitoring strategy. Further research is needed to assess if this difference indicates better or worse monitoring behaviour. Chapter 4 found that drivers performed better on the recognition task when road users were relevant for the driven manoeuvre and when drivers had directed their gaze within 10 degrees of these road users. However, at least 18% of road users were recognised while only observed peripherally, suggesting that peripheral vision should not be neglected in attention monitoring. Recognition performance was not predicted by gaze metrics and requires further development to reduce forget rates. Further analysis is needed to compare the recognition task to established situation awareness measures after these improvements are obtained. Chapter 5 demonstrates that driver and pedestrian attention monitoring can provide a benefit to pedestrian crossing collision risk prediction when predicting further than 0.75 seconds ahead. The higher workload during supervised automation and the general adherence to the operational design domain in naturalistic driving indicate that supervising driving automation can be beneficial to driver attention and traffic safety, but literature and recent accidents demonstrate that challenges remain in encouraging such attentive behaviour. Strategies to encourage attentive supervision should therefore be further developed, as well as ways to maintain these strategies while automation technology improves in pursuit of the opposite objective to reduce engagement in the driving task. The joint analysis of driver gaze and road scene may improve driver support during manual driving and supervised automation, and benefit the development of automated driving. But care should be taken that systems which use driver attention or rely on other contextual cues do not become susceptible to the same mistakes as drivers tend to make. While careful design approaches can reduce the risk of mimicking human error, validation will ultimately require a reliable way to distinguish between awareness and inattentional blindness. The instrumentation and conducted studies with on-road automation demonstrate that on-road research is becoming more practical and accessible than ever before, thanks to recent developments in automation. The observation that during on-road automation, inexperienced drivers perceive higher workload compared to in simulators testifies for the importance of on-road driving research. Challenges encountered during the naturalistic study and attention study demonstrate that the instrumentation and processing have to be designed and tested carefully for on-road research to be effective. @en

Non-verbal communication, such as eye contact between drivers and pedestrians, has been regarded as one way to reduce accident risk. So far, studies have assumed rather than objectively measured the occurrence of eye contact. We address this research gap by developing an eye contact detection method and testing it in an indoor experiment with scripted driver–pedestrian interactions at a pedestrian crossing. Thirty participants acted as a pedestrian either standing on an imaginary curb or crossing an imaginary one-lane road in front of a stationary vehicle with an experimenter in the driver's seat. In half of the trials, pedestrians were instructed to make eye contact with the driver; in the other half, they were prohibited from doing so. Both parties’ gaze was recorded using eye trackers. An in-vehicle stereo camera recorded the car's point of view, a head-mounted camera recorded the pedestrian's point of view, and the location of the driver's and pedestrian's eyes was estimated using image recognition. We demonstrate that eye contact can be detected by measuring the angles between the vector joining the estimated location of the driver's and pedestrian's eyes, and the pedestrian's and driver's instantaneous gaze directions, respectively, and identifying whether these angles fall below a threshold of 4°. We achieved 100% correct classification of the trials involving eye contact and those without eye contact, based on measured eye contact duration. The proposed eye contact detection method may be useful for future research into eye contact.

@en

A necessary condition for the effective integration of automated vehicles in our daily lives is their acceptance by passengers inside and pedestrians and cyclists outside the automated vehicle. 119 respondents experienced an automated shuttle ride with a ‘hidden steward on board’ in a mixed traffic environment in Berlin-Schöneberg. A mixed-method approach was applied gathering qualitative interview data during the ride and quantitative questionnaire data after the ride. Responses were classified into three main categories: (1) Perceived safety, (2) interactions with automated shuttles in crossing situations, and (3) communication with automated shuttles. Respondents associated their perceptions of safety with the low speed, dynamic object and event identification, longitudinal and lateral control, pressing the emergency button inside the shuttle, their general trust in technology, sharing the shuttle with fellow travellers, the operation of the shuttle in a controlled environment, and the behaviour of other road users outside the shuttle. Respondents pressed the emergency button inside the automated shuttle on 28 out of 62 test rides in order to test its behavior. They further expected to be more cautious in crossing the road before an automated shuttle due to the lack of eye contact with the human driver and a lack of trust in the behavior of the automated shuttle, and expected road users testing the automated shuttle due to the conservative driving behavior of automated shuttles. We recommend future research into the hypothesis that the acceptance of automated shuttles will be associated with the perceived safety of and their effective and intuitive interaction and communication with both passengers and other road users.

@en

Objective: To investigate how well gaze behavior can indicate driver awareness of individual road users when related to the vehicle’s road scene perception. Background: An appropriate method is required to identify how driver gaze reveals awareness of other road users. Method: We developed a recognition-based method for labeling of driver situation awareness (SA) in a vehicle with road-scene perception and eye tracking. Thirteen drivers performed 91 left turns on complex urban intersections and identified images of encountered road users among distractor images. Results: Drivers fixated within 2° for 72.8% of relevant and 27.8% of irrelevant road users and were able to recognize 36.1% of the relevant and 19.4% of irrelevant road users one min after leaving the intersection. Gaze behavior could predict road user relevance but not the outcome of the recognition task. Unexpectedly, 18% of road users observed beyond 10° were recognized. Conclusions: Despite suboptimal psychometric properties leading to low recognition rates, our recognition task could identify awareness of individual road users during left turn maneuvers. Perception occurred at gaze angles well beyond 2°, which means that fixation locations are insufficient for awareness monitoring. Application: Findings can be used in driver attention and awareness modelling, and design of gaze-based driver support systems.

@en

Objective: We investigated a driver monitoring system (DMS) designed to adaptively back up distracted drivers with automated driving. Background: Humans are likely inadequate for supervising today’s on-road driving automation. Conversely, backup concepts can use eye-tracker DMS to retain the human as the primary driver and use computerized control only if needed. A distraction DMS where perceived false alarms are minimized and the status of the backup is unannounced might reduce problems of distrust and overreliance, respectively. Experimental research is needed to assess the viability of such designs. Methods: In a driving simulator, 91 participants either supervised driving automation (auto-hand-on-wheel vs. auto-hands-off-wheel), drove with different forms of DMS-induced backup control (eyes-only-backup vs. eyes-plus-context-backup; visible-backup vs. invisible-backup), or drove without any automation. All participants performed a visual N-back task throughout. Results: Supervised driving automation increased visual distraction and hazard non-responses compared to backup and conventional driving. Auto-hand-on-wheel improved response generation compared to auto-hands-off-wheel. Across entire driving trials, the backup improved lateral performance compared to conventional driving. Without negatively impacting safety, the eyes-plus-context-backup DMS reduced unnecessary automated control compared to the eyes-only-backup DMS conditions. Eyes-only-backup produced low satisfaction ratings, whereas eyes-plus-context-backup satisfaction was on par with automated driving. There were no appreciable negative consequences attributable to the invisible-backup driving automation. Conclusions: We have demonstrated preliminary feasibility of DMS designs that incorporate driving context information for distraction assessment and suppress their status indication. Application: An appropriately designed DMS can enable benefits for automated driving as a backup.@en

Driver mental workload is an important factor in the operational safety of automated driving. In this study, workload was evaluated subjectively (NASA R-TLX) and objectively (auditory detection-response task) on Dutch public highways (∼150 km) comparing manual and supervised automated driving in a Tesla Model S with moderators automation experience and traffic complexity. Participants (N = 16) were either automation-inexperienced drivers or automation-experienced Tesla owners. Complexity ranged from an engaging environment with a road geometry stimulating continuous traffic interaction, and a monotonic environment with lower traffic density and a simple road geometry. Perceived and objective workload increased with traffic complexity. When using the automation, automation-experienced drivers perceived a lower workload, while automation-inexperienced drivers perceived their workload to be similar to manual driving. However, the detection-response task indicated an increase in cognitive load with automation, in particular in complex traffic. This indicates that drivers under-estimate the actual task load of attentive monitoring. The findings also highlight the relevance of using system-experienced participants and the importance of incorporating both objective and subjective measures when examining workload.@en

Auditory feedback produced by driver assistance systems can benefit safety. However, auditory feedback is often regarded as annoying, which may result in disuse of the system. An auditory headway feedback system was designed with the aim to improve user acceptance and driving safety. The algorithm used a graded approach, which means that it delivered a more urgent warning if the time headway was smaller. In an on-road test, we compared this design with a conventional binary headway warning system. Participants drove a test vehicle on the highway, once with our graded feedback and once with conventional feedback. User acceptance was assessed through a questionnaire and interview. An inspection of the time headway distributions suggested that participants responded to the auditory feedback for both systems. There were substantial individual differences in time headway, and extremely short headways were rare. These findings suggest that long-term naturalistic trials are needed to assess the safety-effectiveness of graded auditory feedback.@en

Driver mental underload is an important concern in the operational safety of automated driving. In this study, workload was evaluated subjectively (NASA RTLX) and objectively (auditory detection-response task) on Dutch public highways (~150km) in a Tesla Model S comparing manual and supervised automated driving with moderators automation experience and traffic complexity. Participants (N=16) were either automationinexperienced drivers or automation-experienced Tesla owners. Complexity ranged from an engaging environment with a road geometry stimulating continuous traffic interaction, and a monotonic environment with lower traffic density and a simple road geometry. Perceived and objective workload increased with traffic complexity. Automation use reduced perceived workload in both environments for automation-experienced drivers, but not for inexperienced drivers. However, the DRT did not reveal a reduced attentional demand with automation. This suggests that attentive monitoring requires a similar attentional demand as manual driving. The findings highlight the relevance of using system-experienced participants and the relevance of on-road testing for behavioral validity. @en