Introduction: Augmented reality (AR) has been increasingly studied in transportation, particularly for drivers and pedestrians interacting with automated vehicles (AVs). Previous research evaluated AR interfaces using online video-based questionnaires but lacked human-subject research in immersive environments. This study examined if prior online evaluations of nine AR interfaces could be replicated in an immersive virtual environment and if AR interface effectiveness depends on pedestrian attention allocation. 

Methods: Thirty participants completed 120 trials in a CAVE-based simulator with yielding and non-yielding AVs, rating the interface’s intuitiveness and crossing the road when they felt safe. To emulate visual distraction, participants had to look into an attention-attractor circle that disappeared 1 s after the interface appeared. 

Results: The results showed that intuitiveness ratings from the current CAVE-based study and the previous online study correlated strongly (r ≈ 0.90). Head-locked interfaces and familiar designs (augmented traffic lights, zebra crossing) yielded higher intuitiveness ratings and quicker crossing initiations than vehicle-locked interfaces. Vehicle-locked interfaces were less effective when the attention-attractor was on the environment’s opposite side, while head-locked interfaces were relatively unaffected by attention-attractor position. 

Discussion: In conclusion, this ‘AR in VR’ study shows strong congruence between intuitiveness ratings in a CAVE-based study and online research, and demonstrates the importance of interface placement in relation to user gaze direction.

@en

Introduction: Augmented reality (AR) has been increasingly studied in transportation, particularly for drivers and pedestrians interacting with automated vehicles (AVs). Previous research evaluated AR interfaces using online video-based questionnaires but lacked human-subject research in immersive environments. This study examined if prior online evaluations of nine AR interfaces could be replicated in an immersive virtual environment and if AR interface effectiveness depends on pedestrian attention allocation. 

Methods: Thirty participants completed 120 trials in a CAVE-based simulator with yielding and non-yielding AVs, rating the interface’s intuitiveness and crossing the road when they felt safe. To emulate visual distraction, participants had to look into an attention-attractor circle that disappeared 1 s after the interface appeared. 

Results: The results showed that intuitiveness ratings from the current CAVE-based study and the previous online study correlated strongly (r ≈ 0.90). Head-locked interfaces and familiar designs (augmented traffic lights, zebra crossing) yielded higher intuitiveness ratings and quicker crossing initiations than vehicle-locked interfaces. Vehicle-locked interfaces were less effective when the attention-attractor was on the environment’s opposite side, while head-locked interfaces were relatively unaffected by attention-attractor position. 

Discussion: In conclusion, this ‘AR in VR’ study shows strong congruence between intuitiveness ratings in a CAVE-based study and online research, and demonstrates the importance of interface placement in relation to user gaze direction.

@en

Augmented Reality (AR) technology could be utilised to assist pedestrians in navigating safely through traffic. However, whether potential users would understand and use such AR solutions is currently unknown. Nine novel AR interfaces for pedestrian-vehicle communication, previously developed using an experience-based design method, were evaluated through an online questionnaire study completed by 992 respondents in Germany, the Netherlands, Norway, Sweden, and the United Kingdom. The AR indicated whether it was safe to cross the road in front of an approaching automated vehicle. Each interface was rated for its intuitiveness and convincingness, aesthetics, and usefulness. Moreover, comments were collected for qualitative analysis. The results indicated that interfaces that employed traditional design elements from existing traffic, and head-up displays, received the highest ratings overall. Statistical results also showed that there were no significant effects of country, age, and gender on interface acceptance. Thematic analysis of the textual comments offered detail on each interface design's stronger and weaker points, and revealed unintended effects of certain designs. In particular, some of the interfaces were commented on as being dangerous or scary, or were criticised that they could be misinterpreted in that they signal that something is wrong with the vehicle, or that they could occlude the view of the vehicle. The current findings highlight the limitations of experience-based design, and the importance of applying legacy design principles and involving target users in design and evaluation. Future research should be conducted in scenarios in which pedestrians actually interact with approaching vehicles.

Marie Skłodowska-Curie Actions; Innovative Training Networks (ITN); SHAPE-IT; Grant number 860410

Publication date: 21 February 2023

DOI: 10.1016/j.trf.2023.02.005

@en

When humans share space in road traffic, as drivers or as vulnerable road users, they draw on their full range of communicative and interactive capabilities. Much remains unknown about these behaviors, but they need to be captured in models if automated vehicles are to coexist successfully with human road users. Empirical studies of human road user behavior implicate a large number of underlying cognitive mechanisms, which taken together are well beyond the scope of existing computational models. Here, we note that for all of these putative mechanisms, computational theories exist in different subdisciplines of psychology, for more constrained tasks. We demonstrate how these separate theories can be generalized from abstract laboratory paradigms and integrated into a computational framework for modeling human road user interaction, combining Bayesian perception, a theory of mind regarding others’ intentions, behavioral game theory, long-term valuation of action alternatives, and evidence accumulation decision-making. We show that a model with these assumptions—but not simpler versions of the same model—can account for a number of previously unexplained phenomena in naturalistic driver–pedestrian road-crossing interactions, and successfully predicts interaction outcomes in an unseen data set. Our modeling results contribute to demonstrating the real-world value of the theories from which we draw, and address calls in psychology for cumulative theory-building, presenting human road use as a suitable setting for work of this nature. Our findings also underscore the formidable complexity of human interaction in road traffic, with strong implications for the requirements to set on development and testing of vehicle automation.@en

Current research on vehicle-pedestrian interactions focuses on the reaction of one actor other than the interaction of two actors, and considering the impact of the real-time behaviour of both actors on each other. To address this issue, the current study replicated a natural vehicle-pedestrian interaction to the virtual environment by connecting a high-fidelity driving simulator to a CAVE-based pedestrians' simulator. Behaviours from both actors in response to each other were observed indifferent situations including two crossing locations and five time gaps. The proposed method enabled simultaneous interaction in a controlled and safe environment as well as provided implications for future AV design.@en

Current research on vehicle-pedestrian interactions focuses on the reaction of one actor other than the interaction of two actors, and considering the impact of the real-time behaviour of both actors on each other. To address this issue, the current study replicated a natural vehicle-pedestrian interaction to the virtual environment by connecting a high-fidelity driving simulator to a CAVE-based pedestrians' simulator. Behaviours from both actors in response to each other were observed indifferent situations including two crossing locations and five time gaps. The proposed method enabled simultaneous interaction in a controlled and safe environment as well as provided implications for future AV design.@en

Recent developments in vehicle automation require simulations of human-robot interactions in the road traffic context, which can be achieved by computational models of human behavior such as game theory. Game theory provides a good insight into road user behavior by considering agents' interdependencies. However, it is still unclear whether conventional game theory is suitable for modeling vehicle-pedestrian interactions at unsignalized locations or if more complex models like behavioral game theory are needed. Hence, we compared four game-theoretic models based on two different payoff formulations and two solving algorithms, to answer this question. Unlike the most previous studies that employed naturalistic datasets to test and validate such models, this study utilized a distributed simulation dataset to test and compare the models. The study was conducted by connecting a CAVE-based pedestrian simulator to a motion-based driving simulator to replicate the traffic scenarios for 32 pedestrian-driver pairs. The findings demonstrated that there is a high variability between participant pairs' behaviors. Our proposed behavioral game-theoretic model outperformed other models in predicting the interaction outcome. This translates to a decrease by 70% and 67% in the root mean squared error (RMSE) when compared to the baseline model, for marked and unmarked crossings, respectively. The model can also predict which interaction will take the longest time to resolve. According to our results, road users cannot be expected to behave in line with the Nash equilibrium of conventional game theory that underscores the complexity of human behavior with implications for the testing and development of automated vehicles.@en

Recent developments in vehicle automation require simulations of human-robot interactions in the road traffic context, which can be achieved by computational models of human behavior such as game theory. Game theory provides a good insight into road user behavior by considering agents' interdependencies. However, it is still unclear whether conventional game theory is suitable for modeling vehicle-pedestrian interactions at unsignalized locations or if more complex models like behavioral game theory are needed. Hence, we compared four game-theoretic models based on two different payoff formulations and two solving algorithms, to answer this question. Unlike the most previous studies that employed naturalistic datasets to test and validate such models, this study utilized a distributed simulation dataset to test and compare the models. The study was conducted by connecting a CAVE-based pedestrian simulator to a motion-based driving simulator to replicate the traffic scenarios for 32 pedestrian-driver pairs. The findings demonstrated that there is a high variability between participant pairs' behaviors. Our proposed behavioral game-theoretic model outperformed other models in predicting the interaction outcome. This translates to a decrease by 70% and 67% in the root mean squared error (RMSE) when compared to the baseline model, for marked and unmarked crossings, respectively. The model can also predict which interaction will take the longest time to resolve. According to our results, road users cannot be expected to behave in line with the Nash equilibrium of conventional game theory that underscores the complexity of human behavior with implications for the testing and development of automated vehicles.@en

One of the current challenges of automation is to have highly automated vehicles (HAVs) that communicate effectively with pedestrians and react to changes in pedestrian behaviour, to promote more trustable HAVs. However, the details of how human drivers and pedestrians interact at unsignalised crossings remain poorly understood. We addressed some aspects of this challenge by replicating vehicle–pedestrian interactions in a safe and controlled virtual environment by connecting a high fidelity motion-based driving simulator to a CAVE-based pedestrian lab in which 64 participants (32 pairs of one driver and one pedestrian) interacted with each other under different scenarios. The controlled setting helped us study the causal role of kinematics and priority rules on interaction outcome and behaviour, something that is not possible in naturalistic studies. We also found that kinematic cues played a stronger role than psychological traits like sensation seeking and social value orientation in determining whether the pedestrian or driver passed first at unmarked crossings. One main contribution of this study is our experimental paradigm, which permitted repeated observation of crossing interactions by each driver-pedestrian participant pair, yielding behaviours which were qualitatively in line with observations from naturalistic studies.@en

One of the current challenges of automation is to have highly automated vehicles (HAVs) that communicate effectively with pedestrians and react to changes in pedestrian behaviour, to promote more trustable HAVs. However, the details of how human drivers and pedestrians interact at unsignalised crossings remain poorly understood. We addressed some aspects of this challenge by replicating vehicle–pedestrian interactions in a safe and controlled virtual environment by connecting a high fidelity motion-based driving simulator to a CAVE-based pedestrian lab in which 64 participants (32 pairs of one driver and one pedestrian) interacted with each other under different scenarios. The controlled setting helped us study the causal role of kinematics and priority rules on interaction outcome and behaviour, something that is not possible in naturalistic studies. We also found that kinematic cues played a stronger role than psychological traits like sensation seeking and social value orientation in determining whether the pedestrian or driver passed first at unmarked crossings. One main contribution of this study is our experimental paradigm, which permitted repeated observation of crossing interactions by each driver-pedestrian participant pair, yielding behaviours which were qualitatively in line with observations from naturalistic studies.@en

One of the current challenges of automation is to have highly automated vehicles (HAVs) that communicate effectively with pedestrians and react to changes in pedestrian behaviour, to promote more trustable HAVs. However, the details of how human drivers and pedestrians interact at unsignalised crossings remain poorly understood. We addressed some aspects of this challenge by replicating vehicle–pedestrian interactions in a safe and controlled virtual environment by connecting a high fidelity motion-based driving simulator to a CAVE-based pedestrian lab in which 64 participants (32 pairs of one driver and one pedestrian) interacted with each other under different scenarios. The controlled setting helped us study the causal role of kinematics and priority rules on interaction outcome and behaviour, something that is not possible in naturalistic studies. We also found that kinematic cues played a stronger role than psychological traits like sensation seeking and social value orientation in determining whether the pedestrian or driver passed first at unmarked crossings. One main contribution of this study is our experimental paradigm, which permitted repeated observation of crossing interactions by each driver-pedestrian participant pair, yielding behaviours which were qualitatively in line with observations from naturalistic studies.@en

One of the current challenges of automation is to have highly automated vehicles (HAVs) that communicate effectively with pedestrians and react to changes in pedestrian behaviour, to promote more trustable HAVs. However, the details of how human drivers and pedestrians interact at unsignalised crossings remain poorly understood. We addressed some aspects of this challenge by replicating vehicle–pedestrian interactions in a safe and controlled virtual environment by connecting a high fidelity motion-based driving simulator to a CAVE-based pedestrian lab in which 64 participants (32 pairs of one driver and one pedestrian) interacted with each other under different scenarios. The controlled setting helped us study the causal role of kinematics and priority rules on interaction outcome and behaviour, something that is not possible in naturalistic studies. We also found that kinematic cues played a stronger role than psychological traits like sensation seeking and social value orientation in determining whether the pedestrian or driver passed first at unmarked crossings. One main contribution of this study is our experimental paradigm, which permitted repeated observation of crossing interactions by each driver-pedestrian participant pair, yielding behaviours which were qualitatively in line with observations from naturalistic studies.@en

One of the current challenges of automation is to have highly automated vehicles (HAVs) that communicate effectively with pedestrians and react to changes in pedestrian behaviour, to promote more trustable HAVs. However, the details of how human drivers and pedestrians interact at unsignalised crossings remain poorly understood. We addressed some aspects of this challenge by replicating vehicle–pedestrian interactions in a safe and controlled virtual environment by connecting a high fidelity motion-based driving simulator to a CAVE-based pedestrian lab in which 64 participants (32 pairs of one driver and one pedestrian) interacted with each other under different scenarios. The controlled setting helped us study the causal role of kinematics and priority rules on interaction outcome and behaviour, something that is not possible in naturalistic studies. We also found that kinematic cues played a stronger role than psychological traits like sensation seeking and social value orientation in determining whether the pedestrian or driver passed first at unmarked crossings. One main contribution of this study is our experimental paradigm, which permitted repeated observation of crossing interactions by each driver-pedestrian participant pair, yielding behaviours which were qualitatively in line with observations from naturalistic studies.@en

One of the current challenges of automation is to have highly automated vehicles (HAVs) that communicate effectively with pedestrians and react to changes in pedestrian behaviour, to promote more trustable HAVs. However, the details of how human drivers and pedestrians interact at unsignalised crossings remain poorly understood. We addressed some aspects of this challenge by replicating vehicle–pedestrian interactions in a safe and controlled virtual environment by connecting a high fidelity motion-based driving simulator to a CAVE-based pedestrian lab in which 64 participants (32 pairs of one driver and one pedestrian) interacted with each other under different scenarios. The controlled setting helped us study the causal role of kinematics and priority rules on interaction outcome and behaviour, something that is not possible in naturalistic studies. We also found that kinematic cues played a stronger role than psychological traits like sensation seeking and social value orientation in determining whether the pedestrian or driver passed first at unmarked crossings. One main contribution of this study is our experimental paradigm, which permitted repeated observation of crossing interactions by each driver-pedestrian participant pair, yielding behaviours which were qualitatively in line with observations from naturalistic studies.@en

One of the current challenges of automation is to have highly automated vehicles (HAVs) that communicate effectively with pedestrians and react to changes in pedestrian behaviour, to promote more trustable HAVs. However, the details of how human drivers and pedestrians interact at unsignalised crossings remain poorly understood. We addressed some aspects of this challenge by replicating vehicle–pedestrian interactions in a safe and controlled virtual environment by connecting a high fidelity motion-based driving simulator to a CAVE-based pedestrian lab in which 64 participants (32 pairs of one driver and one pedestrian) interacted with each other under different scenarios. The controlled setting helped us study the causal role of kinematics and priority rules on interaction outcome and behaviour, something that is not possible in naturalistic studies. We also found that kinematic cues played a stronger role than psychological traits like sensation seeking and social value orientation in determining whether the pedestrian or driver passed first at unmarked crossings. One main contribution of this study is our experimental paradigm, which permitted repeated observation of crossing interactions by each driver-pedestrian participant pair, yielding behaviours which were qualitatively in line with observations from naturalistic studies.@en

The L3Pilot project tested SAE Level 3 (L3) conditionally automated driving functions addressing driving and travel behavior, impacts on safety, efficiency, environment and socio-economics, and user acceptance. To investigate individual variance in acceptance of conditionally automated cars, an online survey was performed among 18,631 respondents from 17 countries evaluating differences in age, gender, knowledge about the functionality of conditionally automated cars, awareness, information consumption behavior, and expected benefits of conditionally automated cars. Respondents were divided into Enthusiasts, Neutrals, and Sceptics differing in a high, moderate, and low acceptance of conditionally automated cars, respectively. Enthusiasts, Neutrals, and Sceptics differed most with regard to the expected benefits in the productive use of travel time, comfort, and safety of conditionally automated cars. Enthusiasts were male, younger, more knowledgeable about conditionally automated cars, more aware of automated cars, and more likely to receive information about automated cars from different sources, expecting improvements in the productive use of travel time, comfort, and safety due to conditionally automated cars. All groups were most knowledgeable about the lane keeping behavior of conditionally automated cars and least knowledgeable about the operation of conditionally automated cars in dedicated operational design domains. The results indicate that the communication and marketing of automated cars should create a realistic image of the capabilities and limitations of conditionally automated cars where user education programs should be harmonized to calibrate expectations and educate the public.

@en

Nine AR interfaces designed using an experience- based, and theoretically informed design approach, were presented in an online questionnaire for user evaluation.
Statistical analysis of presented measures, and the computation of an overall composite score revealed a preference towards traditional and familiar traffic elements.@en

The impact of automated vehicles (AV) on pedestrians’ crossing behavior has been the topic of some recent studies, but findings are still scarce and inconclusive. The aim of this study is to determine whether the drivers’ presence and apparent attentiveness in a vehicle influences pedestrians’ crossing behavior, perceived behavioral control, and perceived risk, in a controlled environment, using a Head-mounted Display in an immersive Virtual Reality study. Twenty participants took part in a road-crossing experiment. The VR environment consisted of a single lane one-way road with car traffic approaching from the right-hand side of the participant which travelled at 30 kmph. Participants were asked to cross the road if they felt safe to do so. The effect of three driver conditions on pedestrians’ crossing behavior were studied: Attentive driver, distracted driver, and no driver present. Two vehicles were employed with a fixed time gap (3.5 s and 5.5 s) between them to study the effects of time gaps on pedestrians’ crossing behavior. The manipulated vehicle yielded to the pedestrians in half of the trials, stopping completely before reaching the pedestrian's position. The crossing decision, time to initiate the crossing, crossing duration, and safety margin were measured. The main findings show that the vehicle's motion cues (i.e. the gap between the vehicles, and the yielding behavior of the vehicle) were the most important factors affecting pedestrians’ crossing behavior. Therefore, future research should focus more on investigating how AVs should behave while interacting with pedestrians. Distracted driver condition leads to shorter crossing initiation time but the effect was small. No driver condition leads to smaller safety margin. Findings also showed that perceived behavioral control was higher and perceived risk was significantly lower when the driver appeared attentive. Given that drivers will be allowed to do other tasks while AVs are operating in the future, whether explicit communication will be needed in this situation should be further investigated.

@en

The future urban environment may consist of mixed traffic in which pedestrians interact with automated vehicles (AVs). However, it is still unclear how AVs should communicate their intentions to pedestrians. Augmented reality (AR) technology could transform the future of interactions between pedestrians and AVs by offering targeted and individualized communication. This paper presents nine prototypes of AR concepts for pedestrian-AV interaction that are implemented and demonstrated in a real crossing environment. Each concept was based on expert perspectives and designed using theoretically-informed brainstorming sessions. Prototypes were implemented in Unity MARS and subsequently tested on an unmarked road using a standalone iPad Pro with LiDAR functionality. Despite the limitations of the technology, this paper offers an indication of how future AR systems may support future pedestrian-AV interactions.

Marie Skłodowska-Curie Actions; Innovative Training Networks (ITN); SHAPE-IT; Grant number 860410

Publication date: 20 September 2021

DOI: 10.1145/3409118.3475149

@en

This Deliverable starts with a short overview of the design principles and guidelines developed for current Human Machine Interfaces (HMIs), which are predominantly developed for manually driven vehicles, or those with a number of Advanced Driver Assistance Systems (ADAS), at SAE Levels 0 and 1 (SAE, 2018). It then provides an overview of how the addition of more capable systems, and the move to higher levels of vehicle automation, is changing the role the human inside an Automated Vehicle (AV), and the ways in which future automated vehicles at higher levels of automation (SAE level 4 and 5) must communicate with other road users, in the absence of an “in charge” human driver.

It is argued that such changes in the role of the driver, and more transfer of control to the AV and its different functionalities, means that there will be more emphasis on the roles and responsibilities of HMIs for future AVs. In parallel, the multifaceted nature of these HMI, presented from different locations, both in and outside the vehicles, using a variety of modalities, and engaging drivers in a two-way interaction, means that a new set of design guidelines are required, to ensure that the humans interacting with AVs (inside and outside the vehicle) are not distracted and overloaded, that they remain situation aware and understand the capabilities and limitations of the system, having the right mental model of system capabilities and their responsibilities, as responsible road users, at all times

Following a summary of suggested frameworks and design principles which highlight the significant change needed for new AV HMIs, an overview of results from studies investigating human interaction with internal (or iHMIs), and external (or eHMIs), is provided, with examples of new and innovative methods of communication between humans and their vehicles.

The Deliverable then provides a summary of the innovative approaches that will be tackled by the ESRs of the project, which focus on factors such as use of AI and AR for future design of more intuitive and transparent HMI, studying how HMI can support the long term interaction of humans with AVs, and the use of neuroergonomic methods for developing safer HMIs. The Deliverable concludes by summarising how each ESR’s project contributes to the development of HMIs for future AVs.@en