The introduction of automated vehicles means that some or all operational control over these vehicles is diverted away from a human driver to a technological system. The concept of Meaningful Human Control (MHC) was derived to address control issues over automated systems, allowing a system to explicitly consider human intentions and reasons. Applying MHC to technological systems, such as automated driving is a real challenge, and the main focus of this article. An approach with mathematical elaboration has been developed that offers a first quantifiable operationalisation of MHC for the traffic domain and for use with automated vehicles. A major contribution lies in the taxonomification of control for MHC in the broader traffic environment, including consideration of the driver, the vehicle, the traffic environment, considering behaviour, moral standards and societal values, which are considered in a case study. The demonstration case shows the validity of the developed approach for an automated vehicle overtaking a cyclist on an urban street. This article is one of the first to operationalise MHC to such a level of detail and opens the door to further development of the concept for technological implementation.@en

Increased on-road testing and market availability of partially automated vehicles (AV) offers researchers and developers the opportunity to evaluate the AV’s performance. The occurrence of new types of accidents involving AV’s has sparked questions in regard to who is actually in control over and responsible for AV control. In this contribution, we suggest a potential discrepancy in AV control with the review of recently documented accidents involving AV’s. The identification of a gap in control is performed using a recently formulated moral philosophical framework of Meaningful Human Control (MHC). This shows a discrepancy between the attribution of responsibility and the ability of a human to fulfil the role assigned to them. While a gap in control is not evident from the viewpoint of operational control, it requires the more intricate concept of MHC to expose it. Recommendations are further made that AV developers and vehicle approval authorities should consider control from a MHC perspective to avoid future gaps in control with the resulting consequences.@en

As automated vehicles become increasingly common on the road, the call for an appropriate preparation for its drivers is becoming more urgent. Expert opinions and insights have been acquired via a focus group discussion with eleven Dutch driving examiners to assist in inventorying what types of preparations are needed. The concept of meaningful human control (MHC) as an integral part of the discussion lead to consensual findings regarding ADAS functionality and the drivers’ tasks, as well as discussion topics on driver training and levels of automation. It was concluded to have more research into human factors to safeguard proper control over automated vehicles.@en

In this paper, in line with the general framework of value-sensitive design, we aim to operationalize the general concept of “Meaningful Human Control” (MHC) in order to pave the way for its translation into more specific design requirements. In particular, we focus on the operationalization of the first of the two conditions (Santoni de Sio and Van den Hoven 2018) investigated: the so-called ‘tracking’ condition. Our investigation is led in relation to one specific subcase of automated system: dual-mode driving systems (e.g. Tesla ‘autopilot’). First, we connect and compare meaningful human control with a concept of control very popular in engineering and traffic psychology (Michon 1985), and we explain to what extent tracking resembles and differs from it. This will help clarifying the extent to which the idea of meaningful human control is connected to, but also goes beyond, current notions of control in engineering and psychology. Second, we take the systematic analysis of practical reasoning as traditionally presented in the philosophy of human action (Anscombe, Bratman, Mele) and we adapt it to offer a general framework where different types of reasons and agents are identified according to their relation to an automated system’s behaviour. This framework is meant to help explaining what reasons and what agents (should) play a role in controlling a given system, thereby enabling policy makers to produce usable guidelines and engineers to design systems that properly respond to selected human reasons. In the final part, we discuss a practical example of how our framework could be employed in designing automated driving systems.

@en

Automated driving systems (ADS) with partial automation are currently available for the consumer. They are potentially beneficial to traffic flow, fuel consumption, and safety, but human behaviour whilst driving with ADS is poorly understood. Human behaviour is currently expected to lead to dangerous circumstances as ADS could place human drivers ‘out-of-the-loop’ or cause other types of adverse behavioural adaptation. This article introduces the concept of ‘meaningful human control’ to better address the challenges raised by ADS, and presents a new framework of human control over ADS by means of literature-based categorisation. Using standards set by European authorities for driver skills and road rules, this framework offers a unique, quantified perspective into the effects of ADS on human behaviour. One main result is a rapid and inconsistent decrease in required skill- and rule-based behaviour mismatching with the increasing amount of required knowledge-based behaviour. Furthermore, the development of higher levels of automation currently requires different human behaviour than feasible, as a mismatch between supply and demand in terms of behaviour arises. Implications, discrepancies and emerging mismatches this framework elicits are discussed, and recommendations towards future design strategies and research opportunities are made to provide a meaningful transition of human control over ADS.

@en

The future adoption of automated vehicles poses many challenges, with one of the more important being the preservation of control over vehicles that are no longer (fully) operated by drivers. There is consensus that vehicles should not perform actions that are unacceptable to humans. In this paper, we introduce the concept of Meaningful Human Control (MHC) as a function of a framework of the Automated Driving System (ADS). This framework is constructed through the core components that make up the ADS, primarily considered within the categories of the vehicle and driver. Identification of these components and the chain of control allow traceability of MHC to be performed, and aids vehicle manufacturers, software developers, other vehicle component designers, and vehicle- and driver licensing authorities to address many challenges related to the design and preservation of human control in automated vehicles. Operationalisation of MHC is discussed in the paper including a suggested approach that should aid understanding and the application of the concept. Four application examples are given and recommendations are made in regard to vehicle design, human machine interaction, transition of control, driver training, vehicle approval, and other topics. The framework and presented concept also allow researchers to identify areas to perform more explicit and relevant research and develop models that can be applied to perform projections of future impacts.@en

High expectations rest Automated Driving Systems (ADS) to drastically transform roadway transport in the coming decades. In our project we aim at guiding a responsible transition of control toward automated driving. As a first step, we will develop a theory of “meaningful human control” over ADS, and translate the theory into design guidelines, both at the technical and at the institutional level. To maximize human safety, and avoid the creation of accountability gaps, meaningful human control should constantly be maintained over autonomous and semi-autonomous systems. The notion of meaningful human control, originated within the debate about military drones, has –as of yet– not been applied in the context of ADS. The divergent appearance of vehicles with increasing levels of automated control calls for consideration hereof. Therefore, an interdisciplinary team of philosophers, traffic engineers and behavioral scientists will work at defining the conditions for human control, and its “meaningful” elements, and mold it into a workable framework of human control. The validity of this framework will then be tested through empirical research and with the use of automated vehicle prototypes. In conducting the research, particular attention will be dedicated to ensure that the developed theoretical framework can be fully operationalized into empirical science and design recommendations. This will be facilitated by both the interdisciplinary nature of the project itself and by the active participation of a number of private and public partners, who will in turn be provided with up to date research results. In particular, designers, manufacturers and road operators will receive conclusions drawn from empirical and theoretical research they can apply in developing automated systems that achieve meaningful human control; policy-makers can use our findings to elaborate regulations that promote both innovation and human values; lawyers and insurance companies will receive original inputs for the design of liability and insurance schemes; driving licensing bodies will receive data to use for developing new procedures. In the first stage of our project, we are aiming to develop an empirically and technically usable conceptual toolbox (or framework) that can be shared across the different areas of expertise that characterize the project. We will isolate a minimal set of notions of control, based on literature from, among others, philosophy, behavioral science, and engineering. Simultaneously, we aim to identify an as clear as possible notion of “meaningfulness”. On that regard, we started from a philosophical notion of meaningful human control over ADS, to investigate whether, and to what extent, its elements could be operationalized and tested within an empirical framework. We have presented our project to our private and public partners, and to a number of other stakeholders involved in automated driving. From this presentation, we have received positive and constructive feedback, and encouraged us to continue our communication to the public. In particular, we aim to structure a bi-directional communication avenue between data production and data use. Ideally, this would benefit both sides, providing the researchers with societally relevant research questions and practical constraints, and the stakeholders with timely access to scientific results.@en

Truck platooning is a form of vehicle automation and cooperation that is leading the way for cooperative and automated vehicle implementation. However, much is still unknown about the effects and potential dangers of many situations in regard to cooperative control of these platoons. In this contribution, we discuss many of the challenges in regard to full platoon control, we give concepts that can answer some of the questions and make recommendations on how full platoon control should be considered by truck manufactures, ADS software developers and policy makers. A main concept that is applied is that of Meaningful Human Control (MHC). We furthermore consider driver 'reasons', both distal and proximal, to identify correct chains of MHC. We conclude that each part of a system should be responsive to the maximum amount of relevant reasons available and the availability of relevant reasons should be maximized to obtain sufficient MHC.

@en

Human Factors issues with automated driving systems (ADS) are becoming more apparent with the increasing prevalence of automated vehicles on the public roads. As automated driving demands increased performance of supervisory skills of the driver, rather than vehicle handling skills, a mismatch occurs between the demand and supply of the drivers’ skillset. Therefore, it has been suggested that drivers should at all times have meaningful human control (MHC) over ADS. The basic idea behind MHC is derived from the debate on autonomous weapon systems, and entails three essential components: human operators are (1) making informed, conscious decisions, (2) sufficiently informed about lawfulness of an action and its context, and (3) properly trained, to ensure effective control over the use of ADS. This paper presents definitions, components and potential human roles within ADS, from an interdisciplinary and a MHC perspective. The ideas presented in this paper are valuable to both designers, manufacturers, and road operators, as well as policy makers, driving licensing bodies, and lawyers and insurers, and our future research into these topics will deliver usable results for all stakeholders. @en