L.E. Suryana
Please Note
7 records found
1
Reasons and principles for automated vehicle decisions in ethically ambiguous everyday scenarios
The case of cyclist overtaking
Automated vehicles (AVs) consistently encounter ethically ambiguous situations in everyday driving, scenarios involving conflicting human interests and no clearly optimal course of action. While existing work often focuses on rare, high-stakes dilemmas (e.g., crash avoidance or trolley problems), routine decisions such as overtaking cyclists or navigating social interactions remain underexplored. This study addresses that gap by applying the tracking condition of Meaningful Human Control (MHC), which holds that AV behaviour should align with human reasons—the values, intentions, or expectations that justify actions. We conducted semi-structured interviews with 18 AV experts, who explained the reasons behind the considerations AV should make when planning a manoeuvre. Thirteen reason categories emerged, organised across normative, strategic, tactical, and operational levels. Using a case study on cyclist overtaking, we demonstrate how these reasons interact in practice and expose tensions in the decision-making process. Building on this analysis, we derive a reason-prioritisation principle grounded in the cyclist-overtaking scenario for AV behaviour in ethically ambiguous routine situations: prioritising vulnerable road users’ safety above all, treating systemic safety and regulation as important but conditional, and permitting secondary values only when safety is not compromised. This hierarchy supports human-aligned behaviour by allowing pragmatic actions when strict legal compliance would undermine higher-priority values. Our findings offer conceptual principles intended to inform future research and design for AV decision-making in ethically challenging routine situations.
To address this challenge, the thesis adopts the philosophical framework of Meaningful Human Control (MHC), which requires that automated systems both track relevant human reasons and allow responsibility for outcomes to be meaningfully traced to human agents. While MHC has been widely discussed at a conceptual level, its technical operationalisation in automated driving remains underdeveloped. This dissertation advances MHC by translating its normative principles into an integrated framework that connects ethical reasoning, engineering implementation, and empirical evaluation.
The dissertation first investigates which human reasons are relevant for automated-vehicle manoeuvre planning in ethically ambiguous, everyday traffic situations. Empirical findings from interviews with AV experts show that such reasons are inherently multi-layered, context-dependent, and often simultaneous, spanning normative, strategic, tactical, and operational considerations. Rather than functioning as fixed values or isolated cost terms, human reasons are shown to form context-sensitive relationships between underlying motivations and expected vehicle behaviour. These insights provide an empirically grounded basis for structuring and prioritising human reasons in automated-vehicle decision-making.
Building on this foundation, the dissertation develops a technical approach for embedding human reasons within automated-vehicle control architectures. Human reasons are translated into formal, machine-readable representations by drawing on insights from human-factors research and are integrated through a supervisory evaluation layer that operates alongside existing motion planning and control frameworks. This approach enables transparent trajectory evaluation and adaptive behavioural adjustment without requiring the design of new controllers, thereby demonstrating a practical pathway for operationalising MHC in real-time decision-making systems.
Finally, the dissertation examines whether meaningful human control can be empirically assessed in practice. Qualitative studies with users of partially automated driving systems reveal how the tracking and tracing conditions of MHC manifest dynamically in drivers’ experiences of safety, trust, responsibility, and intervention readiness. Complementary simulator experiments show that objective behavioural telemetry can capture aspects of tracking at the level of concrete interaction events, while tracing cannot be inferred from behaviour alone. Together, these findings demonstrate that meaningful human control is not merely a normative or post-hoc concept, but an empirically observable property of ongoing human–automation interaction when evaluated through a multi-layer framework combining subjective perception and objective data.
Overall, this dissertation advances the technical operationalisation of meaningful human control by systematically linking human reasons, automated-vehicle decision-making, and empirical evaluation. The proposed framework provides researchers, designers, and policymakers with concrete tools to assess and support reason-aligned automated-vehicle behaviour, contributing to the development of automated driving systems whose behaviour is more transparent, context-sensitive, and reasonable in everyday traffic situations.
...
To address this challenge, the thesis adopts the philosophical framework of Meaningful Human Control (MHC), which requires that automated systems both track relevant human reasons and allow responsibility for outcomes to be meaningfully traced to human agents. While MHC has been widely discussed at a conceptual level, its technical operationalisation in automated driving remains underdeveloped. This dissertation advances MHC by translating its normative principles into an integrated framework that connects ethical reasoning, engineering implementation, and empirical evaluation.
The dissertation first investigates which human reasons are relevant for automated-vehicle manoeuvre planning in ethically ambiguous, everyday traffic situations. Empirical findings from interviews with AV experts show that such reasons are inherently multi-layered, context-dependent, and often simultaneous, spanning normative, strategic, tactical, and operational considerations. Rather than functioning as fixed values or isolated cost terms, human reasons are shown to form context-sensitive relationships between underlying motivations and expected vehicle behaviour. These insights provide an empirically grounded basis for structuring and prioritising human reasons in automated-vehicle decision-making.
Building on this foundation, the dissertation develops a technical approach for embedding human reasons within automated-vehicle control architectures. Human reasons are translated into formal, machine-readable representations by drawing on insights from human-factors research and are integrated through a supervisory evaluation layer that operates alongside existing motion planning and control frameworks. This approach enables transparent trajectory evaluation and adaptive behavioural adjustment without requiring the design of new controllers, thereby demonstrating a practical pathway for operationalising MHC in real-time decision-making systems.
Finally, the dissertation examines whether meaningful human control can be empirically assessed in practice. Qualitative studies with users of partially automated driving systems reveal how the tracking and tracing conditions of MHC manifest dynamically in drivers’ experiences of safety, trust, responsibility, and intervention readiness. Complementary simulator experiments show that objective behavioural telemetry can capture aspects of tracking at the level of concrete interaction events, while tracing cannot be inferred from behaviour alone. Together, these findings demonstrate that meaningful human control is not merely a normative or post-hoc concept, but an empirically observable property of ongoing human–automation interaction when evaluated through a multi-layer framework combining subjective perception and objective data.
Overall, this dissertation advances the technical operationalisation of meaningful human control by systematically linking human reasons, automated-vehicle decision-making, and empirical evaluation. The proposed framework provides researchers, designers, and policymakers with concrete tools to assess and support reason-aligned automated-vehicle behaviour, contributing to the development of automated driving systems whose behaviour is more transparent, context-sensitive, and reasonable in everyday traffic situations.
Ethical dilemmas are a common challenge in everyday driving, requiring human drivers to balance competing priorities such as safety, efficiency, and rule compliance. However, much of the existing research in automated vehicles (AVs) has focused on high-stakes "trolley problems,"which involve extreme and rare situations. Such scenarios, though rich in ethical implications, are rarely applicable in real-world AV decision-making. In practice, when AVs confront everyday ethical dilemmas, they often appear to prioritise strict adherence to traffic rules. By contrast, human drivers may bend the rules in context-specific situations, using judgement informed by practical concerns such as safety and efficiency. According to the concept of meaningful human control, AVs should respond to human reasons, including those of drivers, vulnerable road users, and policymakers. This work introduces a novel human reasons-based supervision framework that detects when AV behaviour misaligns with expected human reasons to trigger trajectory reconsideration. The framework integrates with motion planning and control systems to support real-time adaptation, enabling decisions that better reflect safety, efficiency, and regulatory considerations. Simulation results demonstrate that this approach could help AVs respond more effectively to ethical challenges in dynamic driving environments by prompting replanning when the current trajectory fails to align with human reasons. These findings suggest that our approach offers a path toward more adaptable, human-centered decision-making in AVs.
Meaningful human control of partially automated driving systems
Insights from interviews with Tesla users
Partially automated driving systems are designed to perform specific driving tasks—such as steering, accelerating, and braking—while still requiring human drivers to monitor the environment and intervene when necessary. This shift of driving responsibilities from human drivers to automated systems raises concerns about accountability, particularly in scenarios involving unexpected events. To address these concerns, the concept of meaningful human control (MHC) has been proposed. MHC emphasises the importance of humans retaining oversight and responsibility for decisions made by automated systems. Despite extensive theoretical discussion of MHC in driving automation, there is limited empirical research on how real-world partially automated systems align with MHC principles. This study offers two main contributions: (1) an empirical evaluation of MHC in partially automated driving, based on 103 semi-structured interviews with users of Tesla's Autopilot and Full Self-Driving (FSD) Beta systems; and (2) a methodological framework for assessing MHC through qualitative interview data. We operationalise the previously proposed tracking and tracing conditions of MHC using a set of evaluation criteria to determine whether these systems support meaningful human control in practice. Our findings indicate that several factors influence the degree to which MHC is achieved. Failures in tracking—where drivers' expectations regarding system safety are not adequately met—arise from technological limitations, susceptibility to environmental conditions (e.g., adverse weather or inadequate infrastructure), and discrepancies between technical performance and user satisfaction. Tracing performance—the ability to clearly assign responsibility—is affected by inconsistent adherence to safety protocols, varying levels of driver confidence, and the specific driving mode in use (e.g., Autopilot versus FSD Beta). These findings contribute to ongoing efforts to design partially automated driving systems that more effectively support meaningful human control and promote more appropriate use of automation.
A Meaningful Human Control Perspective on User Perception of Partially Automated Driving Systems
A Case Study of Tesla Users
The use of partially automated driving systems raises concerns about potential responsibility issues, posing risk to the system safety, acceptance, and adoption of these technologies. The concept of meaningful human control has emerged in response to the responsibility gap problem, requiring the fulfillment of two conditions, tracking and tracing. While this concept has provided important philosophical and design insights on automated driving systems, there is currently little knowledge on how meaningful human control relates to subjective experiences of actual users of these systems. To address this gap, our study aimed to investigate the alignment between the degree of meaningful human control and drivers' perceptions of safety and trust in a real-world partially automated driving system. We utilized previously collected data from interviews with Tesla "Full Self-Driving"(FSD) Beta users, investigating the alignment between the user perception and how well the system was tracking the users' reasons. We found that tracking of users' reasons for driving tasks (such as safe maneuvers) correlated with perceived safety and trust, albeit with notable exceptions. Surprisingly, failure to track lane changing and braking reasons was not necessarily associated with negative perceptions of safety. However, the failure of the system to track expected maneuvers in dangerous situations always resulted in low trust and perceived lack of safety. Overall, our analyses highlight alignment points but also possible discrepancies between perceived safety and trust on the one hand, and meaningful human control on the other hand. Our results can help the developers of automated driving technology to design systems under meaningful human control and are perceived as safe and trustworthy.
Intersections are critical bottlenecks within urban transportation networks. Current models for simulating two-dimensional (2D) vehicular movements at intersections are met with limitations in accurately representing complex interactions and capturing vehicle dynamics. Accordingly, this paper proposes a novel microsimulation framework for trajectory planning and vehicular control at intersections. The model considers vehicle dynamics and control variables, such as acceleration and steering angle, and releases the popular assumption that there is full knowledge sharing or cooperation among vehicles at intersections. These features make the proposed framework more realistic compared to previous microsimulation attempts and applicable to traffic flow and environmental impact assessment studies. In addition, it efficiently operates in realtime for multiple vehicles, overcoming the limitations of offline methods. Moreover, the model is capable of accounting for driver/vehicle detection range, reaction time, and perception and prediction inaccuracies, which enhances its suitability for safety assessments. The evaluation in several scenarios indicates the ability of the proposed framework in realtime planning and following realistic and consistent 2D paths while avoiding collisions with other vehicles.