Circular Image

J.C.F. de Winter

info

Please Note

267 records found

Eye tracking and response times reveal the dynamics of highway merging decisions

Merging onto a highway is a safety-critical task resulting in a large number of traffic accidents; fundamental research into merging behavior of human drivers can help reduce this toll. Two cognitive processes critical to merging, attention allocation and decision making, have been extensively studied in real-world and simulated driving scenarios. However, how these processes interact during highway merging remains poorly understood. While the relationship between attention and decision making has been widely examined in cognitive science, this work has largely relied on simple decision-making paradigms involving choices between static items on a computer screen, which limits the understanding of more dynamic and naturalistic decisions such as in driving. To address this gap, we investigated the relationship between attention and decision making in a simplified highway merging task. In a video-based experiment, participants (N=24) repeatedly made merging gap acceptance decisions based on the dynamic information about the distance and time-to-arrival to the end of the merging lane and the gap to the target-lane vehicle (available in the front view and the side mirror, respectively). Participants’ decisions, response times, and eye movements were recorded. We found that decisions to accept a gap were considerably faster than decisions to reject a gap. Decision outcomes and timing depended on the distance to and time-to-arrival of the target-lane vehicle, but also on the time pressure due to approaching the end of the merging lane. Most importantly, under high time pressure, a greater proportion of time spent looking at the side mirror was associated with a lower probability of accepting the gap. This finding indicates that differences in visual information sampling can be closely linked to decision outcomes when time budgets are constrained. Our results provide initial empirical insights relevant for future cognitive modeling of the interplay between decision making and attention during highway merging. This work can inform early-stage exploration of driver monitoring and support systems for partially automated driving. ...
Journal article (2026) - Pavlo Bazilinskyy, Daniël D. Heikoop, Rutger Verstegen, Marieke H. Martens, Joost C.F. de Winter
This study aims to contribute to guidelines for driver licensing organizations on assessing driver competence in using Level 3 Automated Lane Keeping Systems (ALKS), based on an on-road experiment with eight professional driving assessors (i.e., expert driving examiners who train examiner candidates; 6 males, 2 females, all driving more than 20,000 km per year) in a Wizard-of-Oz vehicle. Using a think-aloud protocol, we captured cognitive processes during system supervision and take-over requests (TORs) in real-world traffic jams. A large language model (LLM)-based thematic analysis of transcripts revealed five themes: (1) Requirement for immediate environmental assessment, (2) Requirement for causal understanding, (3) Requirement for proactive intervention to maintain traffic flow, (4) Requirement for continuous “supervisor” engagement, and (5) Physical ergonomics and mode awareness. These findings indicate that, at least during short-duration usage, drivers do not simply rely on the system to disengage from driving; instead, they maintain active monitoring, physical readiness, and anticipatory skills. These observations blur the distinction between Level 2 and Level 3 automation, as the expert participants in this study generally remained attentive rather than adopting the ‘mind-off’ state that Level 3 theoretically allows. In conclusion, assessing ALKS usage involves not only evaluating a driver’s reaction to a TOR but also judging their performance as a systems manager responsible for anticipating conflicts and smoothly executing control transitions. ...

Effects of visual, auditory, and cognitive demands on mental workload

Introduction Immersive virtual reality applications are increasingly popular in entertainment, education, and professional training. While many aim for maximal realism, simplifying the virtual environment may offer benefits such as reducing mental workload and improving focus on core tasks. However, the impact of different types of demand on users’ mental workload remains unclear. Objective This study explored the impact of visual, auditory, and cognitive demands on users’ mental workload during a daily living activity in immersive virtual reality. Methods Twenty-four participants used a head-mounted display for a virtual shopping task, i.e., picking ten listed products from a shelf, under different conditions: visual demands (moving characters), auditory demands (background noise), cognitive demands (simultaneous arithmetic task), and a combination of all three. Mental workload measures included heart rate, pupil diameter, and self-reported mental demand & effort. Results The cognitively demanding secondary task induced the largest mental workload, significantly exceeding that of auditory and visual demands. For example, on a scale of 1 (low) to 10 (high), self-reported mental demand & effort was 4.40 for the moving characters, 5.00 for the background noise, 6.67 for the arithmetic task, and 7.17 for the combined condition. Biosignal differences were consistent within participants but were masked by high inter-individual variability. Conclusions In virtual shopping tasks, reducing enforced cognitive demands may be more effective for decreasing mental workload than reducing non-task-relevant visual or auditory demands. ...
Journal article (2026) - J.C.F. de Winter, D. Dodou, Fleur Moorlag, Joost Broekens
Previous meta-analyses show that social robots aid learning but were often limited in scope or grouped diverse control conditions together. This meta-analysis examined learning outcomes, focusing on control condition type. We retrieved 146 studies (Google Scholar and reference searches) where a physical social robot was used for training cognitive skills, comprising 183 post-test effect sizes between the robot and the control group, and 372 pre-post effect sizes. Analysis of the 78 studies with control groups indicated that robots generally improved learning, most notably when compared to a no-training control group (d = 0.75). Comparing robots to human teachers yielded an overall positive effect (d = 0.31), although effect sizes varied widely. This variability was explained by the robot’s role: robots in a co-teaching capacity showed a strong positive effect (d = 0.88), while robots replacing the teacher showed no benefit (d = −0.06). LLM-based sentiment analysis indicated that papers from outside Europe received higher positivity scores when describing the robots. We conclude that the effect size is influenced by the robot implementation and the control condition chosen. ...
Journal article (2025) - J.C.F. de Winter, Y.B. Eisma
Many of the commentators use the opportunity to highlight the value of Ergonomics and Human Factors (EHF) and challenge the notion that the discipline is fading. In response, we argue that EHF science is lagging behind the rapid developments in AI, remains entrenched in past-century achievements, and is in decline. Indices such as membership counts, conference attendance numbers, and new regulations reflect activity, but not necessarily impact. Important questions about human-AI collaboration are being addressed by other disciplines, often without the involvement of EHF. Rather than advocating for systemic frameworks, we advocate for new skill development and the adoption of data-driven AI methods. To illustrate the potential of AI within EHF, we demonstrate how a vision-language model can replicate findings from a classic knobs-and-dials study. In conclusion, we acknowledge EHF’s knowledge base but foresee existential risks unless the field undergoes major reforms to remain relevant in an AI-dominated future. ...
Journal article (2025) - David A. Stefan, Daniël D. Heikoop, Joost C.F. de Winter, Sjoerd Houwing
To obtain a driver’s licence, one must successfully complete a practical driving test and a theory test. Although the theory test is widely regarded as an important element of driving competence, little is known about the predictors of theory test performance, and in particular the extent to which the acquired knowledge is retained over the years. All individuals who passed a car theory test in the Netherlands between November 2019 and October 2023 were invited to complete a questionnaire, which included a retention test (i.e., a representative retake test) consisting of 20 items not used before. The results based on 50,857 respondents revealed that those with a lower level of education exhibited lower performance on the retention test. Moreover, respondents who took a course with an instructor, an approach mostly used by those with a lower level of education, had a relatively high likelihood of passing the official car theory test on the first attempt. It was also found that the extent to which knowledge increased or decreased over the years was item-dependent, a pattern possibly explained by whether the test item measures functionally relevant driving experiences or if it primarily assesses isolated rules. The results of this study are relevant for training institutes and policymakers. ...
As automated vehicles (AVs) become increasingly popular, the question arises as to how cyclists will interact with such vehicles. This study investigated (1) whether cyclists spontaneously notice if a vehicle is driverless, (2) how well they perform a driver-detection task when explicitly instructed, and (3) how they carry out these tasks. Using a Wizard-of-Oz method, 37 participants cycled a designated route and encountered an AV multiple times in two experimental sessions. In Session 1, participants cycled the route uninstructed, while in Session 2, they were instructed to verbally report whether they detected the presence or absence of a driver. Additionally, we recorded participants’ gaze behaviour with eye-tracking and their responses in post-session interviews. The interviews revealed that 30% of the cyclists spontaneously mentioned the absence of a driver (Session 1), and when instructed (Session 2), they detected the absence and presence of the driver with 93% accuracy. The eye-tracking data showed that cyclists looked more frequently and for longer at the vehicle in Session 2 compared to Session 1. Additionally, participants exhibited intermittent sampling of the vehicle, and they looked at the area in front of the vehicle when it was far away and towards the windshield region when it was closer. The post-session interviews also indicated that participants were curious, but felt safe, and reported a need to receive information about the AV's driving state. In conclusion, cyclists can detect the absence of a driver in the AV, and this detection may influence their perception of safety. Further research is needed to explore these findings in real-world traffic conditions. ...
Attention bias towards social threat has been linked to loneliness and anxiety, though findings are mixed and concerns about measurement reliability persist. This study examined whether state and trait loneliness, along with personality, self-esteem, social anxiety, and life satisfaction, are associated with attention bias towards social threat images (indicating rejection or exclusion) in young adults (N = 241). AI-generated images were used to enhance control over stimulus content and category distinctions. Participants completed an eye-tracking free-viewing task comprising 40 image matrices (four images per matrix, displayed for 6000 ms). We then computed attention bias (dwell time percentage, total fixation duration percentage, and fixation count percentage) and initial orientation of attention (first fixation percentage). The attention bias measures showed adequate-to-good internal consistency (α = 0.61–0.86). No significant associations emerged between loneliness and attention to socially threatening stimuli, suggesting that heightened vigilance to social threat may not be a feature of loneliness in non-clinical young adults. However, it was found that females exhibited greater attention to social positive images, and baseline pupil diameter was associated with social anxiety. Future research should assess whether loneliness-specific attention bias is a replicable phenomenon, ideally by using an extreme-sampling approach with very lonely individuals. ...

A self-confrontation study on awareness and reasons for speed behaviour

Journal article (2025) - Angèle Picco, Arjan Stuiver, Joost De Winter, Dick De Waard
Despite extensive prevention, speeding remains a major contributor to traffic casualties. Understanding drivers’ perceived awareness and the subjective reasons for their speed behaviour could improve intervention strategies, and specifically inform the potential of speed feedback. A self-confrontation study was conducted in which 25 regular drivers recorded one of their drives using GoPro cameras, capturing both the road view and their speed, and selected video excerpts were later discussed with these participants. The study explored participants’ awareness and reasons for their speed behaviour, as well as general attitudes towards speeding, perceptions of its problematic nature, the acceptability of exceeding speed limits, and decision-making in speed choice. This study design aimed to provide an objective basis for the interviews and reduce recall biases. The results revealed that drivers show a latent awareness of their speeding behaviour, which they most often justified as usual, normal and safe. This general tolerance towards speeding suggests the normalisation of speed violations. As a result, individual safety interventions, such as feedback on driving behaviour, may not be effective. Prevention efforts should focus on changing norms, common beliefs and systemic factors regarding speeding. ...
Journal article (2025) - J. C.F. de Winter, V. Onkhar, D. Dodou
The advent of self-driving cars has sparked discussions about eye contact in traffic, particularly due to challenges that automated vehicles face in non-verbal communication with human road users. In his 1992 book, Turn Signals Are The Facial Expressions Of Automobiles, Don Norman describes how drivers in Mexico City deliberately avoid eye contact when entering a roundabout to create uncertainty in the minds of other drivers, leading the latter to yield right of way. Norman argued that such manipulative or aggressive behavior would not be tolerated in the United States. In the present study, we tested these claims through an online survey involving 3,857 respondents from 20 countries. The results confirmed that Mexican drivers reported a higher frequency of non-speeding ‘aggressive’ violations compared to those from most other countries. Regarding eye contact in the roundabout scenario presented in the survey, national differences were found not so much in the frequency of eye contact but in the reasons behind its use. Mexican drivers tended to avoid eye contact to reduce tension or avoid conflict with other drivers. However, they also frequently reported making eye contact to assert or subtly enforce their right of way. In higher-income countries like the United States, driver-driver eye contact is often deemed unnecessary. In conclusion, our findings partially correspond with Norman's anecdote based on his experiences in 1950s Mexico City. These results may have implications for understanding the stability of traffic cultures and the challenges related to eye contact and non-verbal communication faced by developers of automated vehicles. ...
Recent advancements in AI have accelerated the evolution of versatile robot designs. Chess provides a standardized environment for evaluating the impact of robot behavior on human behavior. This article presents an open-source chess robot for human-robot interaction research, specifically focusing on verbal and non-verbal interactions. The OpenChessRobot recognizes chess pieces using computer vision, executes moves, and interacts with the human player through voice and robotic gestures. We detail the software design, provide quantitative evaluations of the efficacy of the robot, and offer a guide for its reproducibility. An online survey examining people’s views of the robot in three possible scenarios was conducted with 597 participants. The robot received the highest ratings in the robotics education and the chess coach scenarios, while the home entertainment scenario received the lowest scores. The code is accessible on GitHub: https://github.com/renchizhhhh/OpenChessRobot. ...
This study investigated human performance in identifying AI-generated images. In a speeded forced-choice task, 255 participants viewed paired images (one real, one AI-generated by Midjourney) of standard or futuristic cars and buildings and had to identify the AI-generated one, while eye movements were recorded using an eye-tracker. Results revealed a powerful “futurism-as-artificiality” heuristic. Specifically, participants performed poorly (55% correct) when an AI-generated standard image was paired with a real futuristic image. Conversely, accuracy was high (91% correct) when the AI-generated futuristic image was paired with a real standard image. Participants’ gaze landed first on the AI-generated image more often when it depicted a futuristic design than when it depicted a standard one. The demonstrated heuristic presents a double-edged sword for information veracity: it may lead to the uncritical acceptance of AI-generated misinformation that appears conventional, while simultaneously causing real forward-thinking designs to be dismissed as fake. ...
Journal article (2025) - Rins de Zwart, Reinier J. Jansen, Cheryl Bolstad, Mica R. Endsley, Petya Ventsislavova, Joost de Winter, Mark S. Young
The use of situation awareness (SA) measures to assess relative safety in driving is common, with higher levels of SA being interpreted as safer. These relative interpretations do not allow researchers to determine whether the level of SA could be considered “safe” or “unsafe”. In contrast to such interpretations based on relative performance, the current position paper explores the potential for a normative interpretation of situation awareness with regard to safety assessment in driving. A series of expert interviews yielded viewpoints on the current relation between SA and safe driving, theoretical underpinnings for a normative approach, and potential actions towards an SA criterion for safe or unsafe driving. Methodological challenges regarding a normative approach are discussed together with considerations towards a weighted criterion-based approach to SA. The selection of SA requirements relevant for safety and the differentiation and weighting of these requirements on high and lower importance is presented. A method towards objective determination of relevance and weight of SA requirements may increase the usefulness of SA measures for assessment of safety in a driving context. ...
Robots are becoming more capable and can autonomously perform tasks such as navigating between locations. However, human oversight remains crucial. This study compared two touchless methods for directing mobile robots: voice control and gesture control, to investigate the efficiency of these methods and the preference of users. We tested these methods in two conditions: one in which participants remained stationary and one in which they walked freely alongside the robot. We hypothesized that walking alongside the robot would result in higher intuitiveness ratings and improved task performance, based on the idea that walking promotes spatial alignment and reduces the effort required for mental rotation. In a 2×2 within-subject design, 218 participants guided the quadruped robot Spot along a circuitous route with multiple 90° turns using rotate left, rotate right, and walk forward commands. After each trial, participants rated the intuitiveness of the command mapping, while post-experiment interviews were used to gather the participants’ preferences. Results showed that voice control combined with walking with Spot was the most favored and intuitive, whereas gesture control while standing caused confusion for left/right commands. Nevertheless, 29% of participants preferred gesture control, citing increased task engagement and visual congruence as reasons. An odometry-based analysis revealed that participants often followed behind Spot, particularly in the gesture control condition, when they were allowed to walk. In conclusion, voice control with walking produced the best outcomes. Improving physical ergonomics and adjusting gesture types could make gesture control more effective. ...
Neuroscience evidence suggests that personalized, task-specific, high-intensity training is essential for maximizing recovery after acquired brain injury. Robotic devices combined with immersive virtual reality (VR) games, visualized through head-mounted displays (HMDs), can support such intensive training within naturalistic virtual environments with audio-visual stimuli tailored to individual needs. However, the impact of these auditory and visual demands on cognitive load remains an open question. To address this, we conducted an experiment with 22 healthy participants to explore how varying levels of visual, auditory, and cognitive demands affect users’ cognitive load and performance during a shopping task in immersive VR. We found that mental demand had the most significant impact on increasing cognitive load and hampering task performance. Visual demands, although affecting gaze behavior, did not significantly affect cognitive load or performance. Auditory demands showed small effects on cognitive load. ...
Journal article (2025) - Salvatore Luca Cucinella, Joost C.F. de Winter, Erik Grauwmeijer, Marc Evers, Laura Marchal-Crespo
BACKGROUND: Head-mounted displays can be used to offer personalized immersive virtual reality (IVR) training for patients who have suffered an Acquired Brain Injury (ABI) by tailoring the complexity of visual and auditory stimuli to the patient's cognitive capabilities. However, it is still an open question how these virtual environments should be designed. METHODS: We used a human-centered design approach to help define the characteristics of suitable virtual training environments for ABI patients. We conducted (i) observations, (ii) interviews with eleven neurorehabilitation experts, and (iii) an online questionnaire with 24 neurorehabilitation experts to examine how therapists modify current training environments to promote patients' recovery in conventional sensorimotor neurorehabilitation settings. Finally, (iv) we involved eight neurorehabilitation experts in a participatory design workshop to co-create examples of IVR training environments. RESULTS: Five phases of the recovery process (Screening, Planning, Training, Reflecting, and Discharging) and six key themes describing the characteristics of suitable (physical) training environments (Specific, Meaningful, Versatile, Educational, Safe, and Supportive) were identified. The experts agreed that modulating the number of elements (e.g., objects, people) or distractions (e.g., background noise) in the physical training environment enables therapists to provide their patients with suitable conditions to execute functional tasks. Additionally, the experts highlighted the importance of developing IVR training environments that are meaningful and realistic. CONCLUSIONS: Through consultations with neurorehabilitation experts, we gained insights into how therapists adjust physical training environments to promote the execution of functional sensorimotor tasks in patients with diverse cognitive capabilities. Their recommendations on how to modulate and make IVR environments meaningful may contribute to increased motivation and skill transfer. Future studies on IVR-based neurorehabilitation should involve patients themselves. ...

New psychological phenomena

Journal article (2025) - Joost de Winter, P. A. Hancock, Yke Bauke Eisma
This study describes the impact of ChatGPT use on the nature of work from the perspective of academics and educators. We elucidate six phenomena: (1) the cognitive workload associated with conducting Turing tests to determine if ChatGPT has been involved in work productions; (2) the ethical void and alienation that result from recondite ChatGPT use; (3) insights into the motives of individuals who fail to disclose their ChatGPT use, while, at the same time, the recipient does not reveal their awareness of that use; (4) the sense of ennui as the meanings of texts dissipate and no longer reveal the sender’s state of understanding; (5) a redefinition of utility, wherein certain texts show redundancy with patterns already embedded in the base model, while physical measurements and personal observations are considered as unique and novel; (6) a power dynamic between sender and recipient, inadvertently leaving non-participants as disadvantaged third parties. This paper makes clear that the introduction of AI tools into society has far-reaching effects, initially most prominent in text-related fields, such as academia. Whether these implementations represent beneficial innovations for human prosperity, or a rather different line of social evolution, represents the pith of our present discussion. ...

A driving simulator study on the effect of real-time feedback based on information-processing stages

Journal article (2025) - Angèle Picco, Arjan Stuiver, Joost de Winter, Dick de Waard
This driving simulator study, which focused on supporting drivers through feedback rather than automating the driving task, examined the effect of real-time feedback based on different stages of information processing on driving behaviour. The stages investigated included providing information alone, assessment of that information, and a decision based on that assessment, following Parasuraman, Sheridan, and Wickens’s (2000) model of information-processing automation. The acceptability and effectiveness of the different stages of feedback were assessed on two key driving behaviours: speed and distance from the vehicle ahead. The results indicated that feedback had a limited effect on driving behaviour. However, the stage of information processing in the feedback did affect a number of outcomes, with decision-oriented feedback leading to improved behaviours but less favourable attitudinal results. Future safety interventions should consider altering risk perception and beliefs, or providing external motivation for behavioural change. ...
As automated vehicles require human drivers to resume control in critical situations, predicting driver takeover behaviour could be beneficial for safe transitions of control. While previous research has explored predicting takeover behaviour in relation to driver state and traits, little work has examined the predictive value of manual driving style. We hypothesised that drivers’ behaviour during manual driving is predictive of their takeover behaviour when resuming control from an automated vehicle. We assessed 38 drivers with varying experience in a high-fidelity driving simulator. After completing manual driving sessions to assess their driving style, participants performed an automated driving task, typically on a subsequent date. Measures of driving style from manual driving sessions, including headway and lane change speed, were found to be predictive of takeover behaviour. The level of driving experience was associated with the behavioural measures, but correlations between measures of manual driving style and takeover behaviour remained after controlling for driver experience. Our findings demonstrate that how drivers reclaim control from their automated vehicle is not an isolated phenomenon but is associated with manual driving behaviour and driving experience. Strategies to improve takeover safety and comfort could be based on driving style measures, for example by the automated vehicle adapting its behaviour to match a driver's driving style. ...
Journal article (2024) - Tom Driessen, Dimitra Dodou, Pavlo Bazilinskyy, Joost De Winter
Vision-language models are of interest in various domains, including automated driving, where computer vision techniques can accurately detect road users, but where the vehicle sometimes fails to understand context. This study examined the effectiveness of GPT-4V in predicting the level of 'risk' in traffic images as assessed by humans. We used 210 static images taken from a moving vehicle, each previously rated by approximately 650 people. Based on psychometric construct theory and using insights from the self-consistency prompting method, we formulated three hypotheses: (i) repeating the prompt under effectively identical conditions increases validity, (ii) varying the prompt text and extracting a total score increases validity compared to using a single prompt, and (iii) in a multiple regression analysis, the incorporation of object detection features, alongside the GPT-4V-based risk rating, significantly contributes to improving the model's validity. Validity was quantified by the correlation coefficient with human risk scores, across the 210 images. The results confirmed the three hypotheses. The eventual validity coefficient was r = 0.83, indicating that population-level human risk can be predicted using AI with a high degree of accuracy. The findings suggest that GPT-4V must be prompted in a way equivalent to how humans fill out a multi-item questionnaire. ...