Objective: We investigated a driver monitoring system (DMS) designed to adaptively back up distracted drivers with automated driving. Background: Humans are likely inadequate for supervising today’s on-road driving automation. Conversely, backup concepts can use eye-tracker DMS to retain the human as the primary driver and use computerized control only if needed. A distraction DMS where perceived false alarms are minimized and the status of the backup is unannounced might reduce problems of distrust and overreliance, respectively. Experimental research is needed to assess the viability of such designs. Methods: In a driving simulator, 91 participants either supervised driving automation (auto-hand-on-wheel vs. auto-hands-off-wheel), drove with different forms of DMS-induced backup control (eyes-only-backup vs. eyes-plus-context-backup; visible-backup vs. invisible-backup), or drove without any automation. All participants performed a visual N-back task throughout. Results: Supervised driving automation increased visual distraction and hazard non-responses compared to backup and conventional driving. Auto-hand-on-wheel improved response generation compared to auto-hands-off-wheel. Across entire driving trials, the backup improved lateral performance compared to conventional driving. Without negatively impacting safety, the eyes-plus-context-backup DMS reduced unnecessary automated control compared to the eyes-only-backup DMS conditions. Eyes-only-backup produced low satisfaction ratings, whereas eyes-plus-context-backup satisfaction was on par with automated driving. There were no appreciable negative consequences attributable to the invisible-backup driving automation. Conclusions: We have demonstrated preliminary feasibility of DMS designs that incorporate driving context information for distraction assessment and suppress their status indication. Application: An appropriately designed DMS can enable benefits for automated driving as a backup.

This work aimed to organise recommendations for keeping people engaged during human supervision of driving automation, encouraging a safe and acceptable introduction of automated driving systems. First, heuristic knowledge of human factors, ergonomics, and psychological theory was used to propose solution areas to human supervisory control problems of sustained attention. Driving and non-driving research examples were drawn to substantiate the solution areas. Automotive manufacturers might (1) avoid this supervisory role altogether, (2) reduce it in objective ways or (3) alter its subjective experiences, (4) utilize conditioning learning principles such as with gamification and/or selection/training techniques, (5) support internal driver cognitive processes and mental models and/or (6) leverage externally situated information regarding relations between the driver, the driving task, and the driving environment. Second, a cross-domain literature survey of influential human-automation interaction research was conducted for how to keep engagement/attention in supervisory control. The solution areas (via numeric theme codes) were found to be reliably applied from independent rater categorisations of research recommendations. Areas (5) and (6) were addressed by around 70% or more of the studies, areas (2) and (4) in around 50% of the studies, and areas (3) and (1) in less than around 20% and 5%, respectively. The present contribution offers a guiding organisational framework towards improving human attention while supervising driving automation.

The topic of situation awareness has received continuing interest over the last decades. Freeze-probe methods, such as the Situation Awareness Global Assessment Technique (SAGAT), are commonly employed for measuring situation awareness. The aim of this paper was to review validity issues of the SAGAT and examine whether eye movements are a promising alternative for measuring situation awareness. First, we outlined six problems of freeze-probe methods, such as the fact that freeze-probe methods rely on what the operator has been able to remember and then explicitly recall. We propose an operationalization of situation awareness based on the eye movements of the person in relation to their task environment to circumvent shortfalls of memory mediation and task interruption. Next, we analyzed experimental data in which participants (N = 86) were tasked to observe a display of six dials for about 10 min, and press the space bar if a dial pointer crossed a threshold value. Every 90 s, the screen was blanked and participants had to report the state of the dials on a paper sheet. We assessed correlations of participants’ task performance (% of threshold crossing detected) with visual sampling scores (% of dials glanced at during threshold crossings) and freeze-probe scores. Results showed that the visual-sampling score correlated with task performance at the threshold-crossing level (r = 0.31) and at the individual level (r = 0.78). Freeze-probe scores were low and showed weak associations with task performance. We conclude that the outlined limitations of the SAGAT impede measurement of situation awareness, which can be computed more effectively from eye movement measurements in relation to the state of the task environment. The present findings have practical value, as advances in eye-tracking cameras and ubiquitous computing lessen the need for interruptive tests such as SAGAT. Eye-based situation awareness is a predictor of performance, with the advantage that it is applicable through real-time feedback technologies.

In pioneering work, Senders (1983) tasked five participants to watch a bank of six dials, and found that glance rates and times glanced at dials increase linearly as a function of the frequency bandwidth of the dial's pointer. Senders did not record the angle of the pointers synchronously with eye movements, and so could not assess participants’ visual sampling behavior in regard to the pointer state. Because the study of Senders has been influential but never repeated, we replicated and extended it by assessing the relationship between visual sampling and pointer state, using modern eye-tracking equipment. Eye tracking was performed with 86 participants who watched seven 90-second videos, each video showing six dials with moving pointers. Participants had to press the spacebar when any of the six pointers crossed a threshold. Our results showed a close resemblance to Senders’ original results. Additionally, we found that participants did not behave in accordance with a periodic sampling model, but rather were conditional samplers, in that the probability of looking at a dial was contingent on pointer angle and velocity. Finally, we found that participants sampled more in agreement with Nyquist sampling when the high bandwidth dials were placed in the middle of the bank rather than at its outer edges. We observed results consistent with the saliency, effort, expectancy, and value model and conclude that human sampling of multidegree of freedom systems should not only be modeled in terms of bandwidth but also in terms of saliency and effort.

A common challenge with processing naturalistic driving data is that humans may need to categorize great volumes of recorded visual information. By means of the online platform CrowdFlower, we investigated the potential of crowdsourcing to categorize driving scene features (i.e., presence of other road users, straight road segments, etc.) at greater scale than a single person or a small team of researchers would be capable of. In total, 200 workers from 46 different countries participated in 1.5. days. Validity and reliability were examined, both with and without embedding researcher generated control questions via the CrowdFlower mechanism known as Gold Test Questions (GTQs).By employing GTQs, we found significantly more valid (accurate) and reliable (consistent) identification of driving scene items from external workers. Specifically, at a small scale CrowdFlower Job of 48 three-second video segments, an accuracy (i.e., relative to the ratings of a confederate researcher) of 91% on items was found with GTQs compared to 78% without. A difference in bias was found, where without GTQs, external workers returned more false positives than with GTQs. At a larger scale CrowdFlower Job making exclusive use of GTQs, 12,862 three-second video segments were released for annotation. Infeasible (and self-defeating) to check the accuracy of each at this scale, a random subset of 1012 categorizations was validated and returned similar levels of accuracy (95%).In the small scale Job, where full video segments were repeated in triplicate, the percentage of unanimous agreement on the items was found significantly more consistent when using GTQs (90%) than without them (65%). Additionally, in the larger scale Job (where a single second of a video segment was overlapped by ratings of three sequentially neighboring segments), a mean unanimity of 94% was obtained with validated-as-correct ratings and 91% with non-validated ratings. Because the video segments overlapped in full for the small scale Job, and in part for the larger scale Job, it should be noted that such reliability reported here may not be directly comparable. Nonetheless, such results are both indicative of high levels of obtained rating reliability.Overall, our results provide compelling evidence for CrowdFlower, via use of GTQs, being able to yield more accurate and consistent crowdsourced categorizations of naturalistic driving scene contents than when used without such a control mechanism. Such annotations in such short periods of time present a potentially powerful resource in driving research and driving automation development.

Across the automotive industry, manufacturers have recently released various Partial Automation systems (SAE Level 2) which allow simultaneous/combined execution of both lateral and longitudinal vehicle control at the same time, yet still require active human supervision/engagement. Current reactive trends will be reviewed across major automotive players regarding differences in terminology, HMI input/outputs, and escalation intervals. Scholarly research is also reviewed pertaining to proactive strategies for driver engagement. Additionally, human factors research and findings will be presented regarding recommendations for situation awareness, human machine interfaces, TOR, as well as shared control concepts. The tutorial will conclude with discussion and brainstorming around outlook toward tele-operated remote driving services (Tele-Driving); what they have to offer beyond assisted/automated driving, autonomous vehicles, and ride-hailing/car-sharing paradigms; as well as the design/conduct of human factors research regarding Tele-Driving.

Automated driving vehicles of the future will most likely include multiple modes and levels of operation and thus include various transitions of control (ToC) between human and machine. Traditional activation devices (e.g., knobs, switches, buttons, and touchscreens) may be confused by operators among other system setting manipulators and also susceptible to inappropriate usage. Non-intrusive eye-tracking measures may assess driver states (i.e., distraction, drowsiness, and cognitive overload) automatically to trigger manual-to-automation ToC and serve as a driver readiness verification during automation-to-manual ToC. Our integrated driver state monitor is overviewed here within the scope of this brief system description/demonstration paper. It combines gaze position, gaze variability, eyelid opening, as well as external environmental complexity from the driving scene to facilitate ToC in automated driving. As both driver facing and forward facing cameras become increasingly commonplace and even legally mandated within various automated driving vehicles, our integrated system helps inform relevant future research and development towards improved human-computer interaction and driving safety.