J.C.F. de Winter
Please Note
267 records found
1
Now or never
Eye tracking and response times reveal the dynamics of highway merging decisions
Merging onto a highway is a safety-critical task resulting in a large number of traffic accidents; fundamental research into merging behavior of human drivers can help reduce this toll. Two cognitive processes critical to merging, attention allocation and decision making, have been extensively studied in real-world and simulated driving scenarios. However, how these processes interact during highway merging remains poorly understood. While the relationship between attention and decision making has been widely examined in cognitive science, this work has largely relied on simple decision-making paradigms involving choices between static items on a computer screen, which limits the understanding of more dynamic and naturalistic decisions such as in driving. To address this gap, we investigated the relationship between attention and decision making in a simplified highway merging task. In a video-based experiment, participants (N=24) repeatedly made merging gap acceptance decisions based on the dynamic information about the distance and time-to-arrival to the end of the merging lane and the gap to the target-lane vehicle (available in the front view and the side mirror, respectively). Participants’ decisions, response times, and eye movements were recorded. We found that decisions to accept a gap were considerably faster than decisions to reject a gap. Decision outcomes and timing depended on the distance to and time-to-arrival of the target-lane vehicle, but also on the time pressure due to approaching the end of the merging lane. Most importantly, under high time pressure, a greater proportion of time spent looking at the side mirror was associated with a lower probability of accepting the gap. This finding indicates that differences in visual information sampling can be closely linked to decision outcomes when time budgets are constrained. Our results provide initial empirical insights relevant for future cognitive modeling of the interplay between decision making and attention during highway merging. This work can inform early-stage exploration of driver monitoring and support systems for partially automated driving.
This study aims to contribute to guidelines for driver licensing organizations on assessing driver competence in using Level 3 Automated Lane Keeping Systems (ALKS), based on an on-road experiment with eight professional driving assessors (i.e., expert driving examiners who train examiner candidates; 6 males, 2 females, all driving more than 20,000 km per year) in a Wizard-of-Oz vehicle. Using a think-aloud protocol, we captured cognitive processes during system supervision and take-over requests (TORs) in real-world traffic jams. A large language model (LLM)-based thematic analysis of transcripts revealed five themes: (1) Requirement for immediate environmental assessment, (2) Requirement for causal understanding, (3) Requirement for proactive intervention to maintain traffic flow, (4) Requirement for continuous “supervisor” engagement, and (5) Physical ergonomics and mode awareness. These findings indicate that, at least during short-duration usage, drivers do not simply rely on the system to disengage from driving; instead, they maintain active monitoring, physical readiness, and anticipatory skills. These observations blur the distinction between Level 2 and Level 3 automation, as the expert participants in this study generally remained attentive rather than adopting the ‘mind-off’ state that Level 3 theoretically allows. In conclusion, assessing ALKS usage involves not only evaluating a driver’s reaction to a TOR but also judging their performance as a systems manager responsible for anticipating conflicts and smoothly executing control transitions.
Shopping in immersive virtual reality
Effects of visual, auditory, and cognitive demands on mental workload
Introduction Immersive virtual reality applications are increasingly popular in entertainment, education, and professional training. While many aim for maximal realism, simplifying the virtual environment may offer benefits such as reducing mental workload and improving focus on core tasks. However, the impact of different types of demand on users’ mental workload remains unclear. Objective This study explored the impact of visual, auditory, and cognitive demands on users’ mental workload during a daily living activity in immersive virtual reality. Methods Twenty-four participants used a head-mounted display for a virtual shopping task, i.e., picking ten listed products from a shelf, under different conditions: visual demands (moving characters), auditory demands (background noise), cognitive demands (simultaneous arithmetic task), and a combination of all three. Mental workload measures included heart rate, pupil diameter, and self-reported mental demand & effort. Results The cognitively demanding secondary task induced the largest mental workload, significantly exceeding that of auditory and visual demands. For example, on a scale of 1 (low) to 10 (high), self-reported mental demand & effort was 4.40 for the moving characters, 5.00 for the background noise, 6.67 for the arithmetic task, and 7.17 for the combined condition. Biosignal differences were consistent within participants but were masked by high inter-individual variability. Conclusions In virtual shopping tasks, reducing enforced cognitive demands may be more effective for decreasing mental workload than reducing non-task-relevant visual or auditory demands.
To obtain a driver’s licence, one must successfully complete a practical driving test and a theory test. Although the theory test is widely regarded as an important element of driving competence, little is known about the predictors of theory test performance, and in particular the extent to which the acquired knowledge is retained over the years. All individuals who passed a car theory test in the Netherlands between November 2019 and October 2023 were invited to complete a questionnaire, which included a retention test (i.e., a representative retake test) consisting of 20 items not used before. The results based on 50,857 respondents revealed that those with a lower level of education exhibited lower performance on the retention test. Moreover, respondents who took a course with an instructor, an approach mostly used by those with a lower level of education, had a relatively high likelihood of passing the official car theory test on the first attempt. It was also found that the extent to which knowledge increased or decreased over the years was item-dependent, a pattern possibly explained by whether the test item measures functionally relevant driving experiences or if it primarily assesses isolated rules. The results of this study are relevant for training institutes and policymakers.
As automated vehicles (AVs) become increasingly popular, the question arises as to how cyclists will interact with such vehicles. This study investigated (1) whether cyclists spontaneously notice if a vehicle is driverless, (2) how well they perform a driver-detection task when explicitly instructed, and (3) how they carry out these tasks. Using a Wizard-of-Oz method, 37 participants cycled a designated route and encountered an AV multiple times in two experimental sessions. In Session 1, participants cycled the route uninstructed, while in Session 2, they were instructed to verbally report whether they detected the presence or absence of a driver. Additionally, we recorded participants’ gaze behaviour with eye-tracking and their responses in post-session interviews. The interviews revealed that 30% of the cyclists spontaneously mentioned the absence of a driver (Session 1), and when instructed (Session 2), they detected the absence and presence of the driver with 93% accuracy. The eye-tracking data showed that cyclists looked more frequently and for longer at the vehicle in Session 2 compared to Session 1. Additionally, participants exhibited intermittent sampling of the vehicle, and they looked at the area in front of the vehicle when it was far away and towards the windshield region when it was closer. The post-session interviews also indicated that participants were curious, but felt safe, and reported a need to receive information about the AV's driving state. In conclusion, cyclists can detect the absence of a driver in the AV, and this detection may influence their perception of safety. Further research is needed to explore these findings in real-world traffic conditions.
Loneliness, personality, and attention to AI-generated images depicting social threat
An eye-tracking study
Attention bias towards social threat has been linked to loneliness and anxiety, though findings are mixed and concerns about measurement reliability persist. This study examined whether state and trait loneliness, along with personality, self-esteem, social anxiety, and life satisfaction, are associated with attention bias towards social threat images (indicating rejection or exclusion) in young adults (N = 241). AI-generated images were used to enhance control over stimulus content and category distinctions. Participants completed an eye-tracking free-viewing task comprising 40 image matrices (four images per matrix, displayed for 6000 ms). We then computed attention bias (dwell time percentage, total fixation duration percentage, and fixation count percentage) and initial orientation of attention (first fixation percentage). The attention bias measures showed adequate-to-good internal consistency (α = 0.61–0.86). No significant associations emerged between loneliness and attention to socially threatening stimuli, suggesting that heightened vigilance to social threat may not be a feature of loneliness in non-clinical young adults. However, it was found that females exhibited greater attention to social positive images, and baseline pupil diameter was associated with social anxiety. Future research should assess whether loneliness-specific attention bias is a replicable phenomenon, ideally by using an extreme-sampling approach with very lonely individuals.
“Why were you speeding?”
A self-confrontation study on awareness and reasons for speed behaviour
Despite extensive prevention, speeding remains a major contributor to traffic casualties. Understanding drivers’ perceived awareness and the subjective reasons for their speed behaviour could improve intervention strategies, and specifically inform the potential of speed feedback. A self-confrontation study was conducted in which 25 regular drivers recorded one of their drives using GoPro cameras, capturing both the road view and their speed, and selected video excerpts were later discussed with these participants. The study explored participants’ awareness and reasons for their speed behaviour, as well as general attitudes towards speeding, perceptions of its problematic nature, the acceptability of exceeding speed limits, and decision-making in speed choice. This study design aimed to provide an objective basis for the interviews and reduce recall biases. The results revealed that drivers show a latent awareness of their speeding behaviour, which they most often justified as usual, normal and safe. This general tolerance towards speeding suggests the normalisation of speed violations. As a result, individual safety interventions, such as feedback on driving behaviour, may not be effective. Prevention efforts should focus on changing norms, common beliefs and systemic factors regarding speeding.
Cross-national differences in drivers’ eye contact and traffic violations
An online survey across 20 countries
The advent of self-driving cars has sparked discussions about eye contact in traffic, particularly due to challenges that automated vehicles face in non-verbal communication with human road users. In his 1992 book, Turn Signals Are The Facial Expressions Of Automobiles, Don Norman describes how drivers in Mexico City deliberately avoid eye contact when entering a roundabout to create uncertainty in the minds of other drivers, leading the latter to yield right of way. Norman argued that such manipulative or aggressive behavior would not be tolerated in the United States. In the present study, we tested these claims through an online survey involving 3,857 respondents from 20 countries. The results confirmed that Mexican drivers reported a higher frequency of non-speeding ‘aggressive’ violations compared to those from most other countries. Regarding eye contact in the roundabout scenario presented in the survey, national differences were found not so much in the frequency of eye contact but in the reasons behind its use. Mexican drivers tended to avoid eye contact to reduce tension or avoid conflict with other drivers. However, they also frequently reported making eye contact to assert or subtly enforce their right of way. In higher-income countries like the United States, driver-driver eye contact is often deemed unnecessary. In conclusion, our findings partially correspond with Norman's anecdote based on his experiences in 1950s Mexico City. These results may have implications for understanding the stability of traffic cultures and the challenges related to eye contact and non-verbal communication faced by developers of automated vehicles.
Detecting Midjourney-Generated Images
An Eye-Tracking Study
This study investigated human performance in identifying AI-generated images. In a speeded forced-choice task, 255 participants viewed paired images (one real, one AI-generated by Midjourney) of standard or futuristic cars and buildings and had to identify the AI-generated one, while eye movements were recorded using an eye-tracker. Results revealed a powerful “futurism-as-artificiality” heuristic. Specifically, participants performed poorly (55% correct) when an AI-generated standard image was paired with a real futuristic image. Conversely, accuracy was high (91% correct) when the AI-generated futuristic image was paired with a real standard image. Participants’ gaze landed first on the AI-generated image more often when it depicted a futuristic design than when it depicted a standard one. The demonstrated heuristic presents a double-edged sword for information veracity: it may lead to the uncritical acceptance of AI-generated misinformation that appears conventional, while simultaneously causing real forward-thinking designs to be dismissed as fake.
The use of situation awareness (SA) measures to assess relative safety in driving is common, with higher levels of SA being interpreted as safer. These relative interpretations do not allow researchers to determine whether the level of SA could be considered “safe” or “unsafe”. In contrast to such interpretations based on relative performance, the current position paper explores the potential for a normative interpretation of situation awareness with regard to safety assessment in driving. A series of expert interviews yielded viewpoints on the current relation between SA and safe driving, theoretical underpinnings for a normative approach, and potential actions towards an SA criterion for safe or unsafe driving. Methodological challenges regarding a normative approach are discussed together with considerations towards a weighted criterion-based approach to SA. The selection of SA requirements relevant for safety and the differentiation and weighting of these requirements on high and lower importance is presented. A method towards objective determination of relevance and weight of SA requirements may increase the usefulness of SA measures for assessment of safety in a driving context.
Neuroscience evidence suggests that personalized, task-specific, high-intensity training is essential for maximizing recovery after acquired brain injury. Robotic devices combined with immersive virtual reality (VR) games, visualized through head-mounted displays (HMDs), can support such intensive training within naturalistic virtual environments with audio-visual stimuli tailored to individual needs. However, the impact of these auditory and visual demands on cognitive load remains an open question. To address this, we conducted an experiment with 22 healthy participants to explore how varying levels of visual, auditory, and cognitive demands affect users’ cognitive load and performance during a shopping task in immersive VR. We found that mental demand had the most significant impact on increasing cognitive load and hampering task performance. Visual demands, although affecting gaze behavior, did not significantly affect cognitive load or performance. Auditory demands showed small effects on cognitive load.
BACKGROUND: Head-mounted displays can be used to offer personalized immersive virtual reality (IVR) training for patients who have suffered an Acquired Brain Injury (ABI) by tailoring the complexity of visual and auditory stimuli to the patient's cognitive capabilities. However, it is still an open question how these virtual environments should be designed. METHODS: We used a human-centered design approach to help define the characteristics of suitable virtual training environments for ABI patients. We conducted (i) observations, (ii) interviews with eleven neurorehabilitation experts, and (iii) an online questionnaire with 24 neurorehabilitation experts to examine how therapists modify current training environments to promote patients' recovery in conventional sensorimotor neurorehabilitation settings. Finally, (iv) we involved eight neurorehabilitation experts in a participatory design workshop to co-create examples of IVR training environments. RESULTS: Five phases of the recovery process (Screening, Planning, Training, Reflecting, and Discharging) and six key themes describing the characteristics of suitable (physical) training environments (Specific, Meaningful, Versatile, Educational, Safe, and Supportive) were identified. The experts agreed that modulating the number of elements (e.g., objects, people) or distractions (e.g., background noise) in the physical training environment enables therapists to provide their patients with suitable conditions to execute functional tasks. Additionally, the experts highlighted the importance of developing IVR training environments that are meaningful and realistic. CONCLUSIONS: Through consultations with neurorehabilitation experts, we gained insights into how therapists adjust physical training environments to promote the execution of functional sensorimotor tasks in patients with diverse cognitive capabilities. Their recommendations on how to modulate and make IVR environments meaningful may contribute to increased motivation and skill transfer. Future studies on IVR-based neurorehabilitation should involve patients themselves.
ChatGPT and academic work
New psychological phenomena
This study describes the impact of ChatGPT use on the nature of work from the perspective of academics and educators. We elucidate six phenomena: (1) the cognitive workload associated with conducting Turing tests to determine if ChatGPT has been involved in work productions; (2) the ethical void and alienation that result from recondite ChatGPT use; (3) insights into the motives of individuals who fail to disclose their ChatGPT use, while, at the same time, the recipient does not reveal their awareness of that use; (4) the sense of ennui as the meanings of texts dissipate and no longer reveal the sender’s state of understanding; (5) a redefinition of utility, wherein certain texts show redundancy with patterns already embedded in the base model, while physical measurements and personal observations are considered as unique and novel; (6) a power dynamic between sender and recipient, inadvertently leaving non-participants as disadvantaged third parties. This paper makes clear that the introduction of AI tools into society has far-reaching effects, initially most prominent in text-related fields, such as academia. Whether these implementations represent beneficial innovations for human prosperity, or a rather different line of social evolution, represents the pith of our present discussion.
Information, assessment, or decision
A driving simulator study on the effect of real-time feedback based on information-processing stages
This driving simulator study, which focused on supporting drivers through feedback rather than automating the driving task, examined the effect of real-time feedback based on different stages of information processing on driving behaviour. The stages investigated included providing information alone, assessment of that information, and a decision based on that assessment, following Parasuraman, Sheridan, and Wickens’s (2000) model of information-processing automation. The acceptability and effectiveness of the different stages of feedback were assessed on two key driving behaviours: speed and distance from the vehicle ahead. The results indicated that feedback had a limited effect on driving behaviour. However, the stage of information processing in the feedback did affect a number of outcomes, with decision-oriented feedback leading to improved behaviours but less favourable attitudinal results. Future safety interventions should consider altering risk perception and beliefs, or providing external motivation for behavioural change.
As automated vehicles require human drivers to resume control in critical situations, predicting driver takeover behaviour could be beneficial for safe transitions of control. While previous research has explored predicting takeover behaviour in relation to driver state and traits, little work has examined the predictive value of manual driving style. We hypothesised that drivers’ behaviour during manual driving is predictive of their takeover behaviour when resuming control from an automated vehicle. We assessed 38 drivers with varying experience in a high-fidelity driving simulator. After completing manual driving sessions to assess their driving style, participants performed an automated driving task, typically on a subsequent date. Measures of driving style from manual driving sessions, including headway and lane change speed, were found to be predictive of takeover behaviour. The level of driving experience was associated with the behavioural measures, but correlations between measures of manual driving style and takeover behaviour remained after controlling for driver experience. Our findings demonstrate that how drivers reclaim control from their automated vehicle is not an isolated phenomenon but is associated with manual driving behaviour and driving experience. Strategies to improve takeover safety and comfort could be based on driving style measures, for example by the automated vehicle adapting its behaviour to match a driver's driving style.
Vision-language models are of interest in various domains, including automated driving, where computer vision techniques can accurately detect road users, but where the vehicle sometimes fails to understand context. This study examined the effectiveness of GPT-4V in predicting the level of 'risk' in traffic images as assessed by humans. We used 210 static images taken from a moving vehicle, each previously rated by approximately 650 people. Based on psychometric construct theory and using insights from the self-consistency prompting method, we formulated three hypotheses: (i) repeating the prompt under effectively identical conditions increases validity, (ii) varying the prompt text and extracting a total score increases validity compared to using a single prompt, and (iii) in a multiple regression analysis, the incorporation of object detection features, alongside the GPT-4V-based risk rating, significantly contributes to improving the model's validity. Validity was quantified by the correlation coefficient with human risk scores, across the 210 images. The results confirmed the three hypotheses. The eventual validity coefficient was r = 0.83, indicating that population-level human risk can be predicted using AI with a high degree of accuracy. The findings suggest that GPT-4V must be prompted in a way equivalent to how humans fill out a multi-item questionnaire.