This book claims that artificial intelligence (AI) may affect our freedom at work, in our daily life, and in the political sphere. The author provides a philosophical framework to help make sense of and govern the ethical and political impact of AI in these domains. AI presents great opportunities and risks, raising the question of how to reap its potential benefits without endangering basic human and societal values. The author identifies three major risks for human freedom. First, AI offers employers new forms of control of the workforce, opening the door to new forms of domination and exploitation. Second, it may reduce our capacity to remain in control of and responsible for our decisions and actions, thereby affecting our free will and moral responsibility. Third, it may increase the power of governments and tech companies to steer the political debate, thereby affecting the possibility of a free and inclusive political participation. The author claims that it is still possible to promote human freedom in our interactions with AI. This requires designing AI systems that help promote workers’ freedom, strengthen human control and responsibility, and foster a free, active, and inclusive democratic participation. Human Freedom in the Age of AI will be of interest to scholars and graduate students working on the ethics of technology, philosophy of technology, political philosophy, design, and artificial intelligence.@en

This handbook presents the concept of ‘meaningful human control’ (MHC) over AI systems from the perspectives of (i) philosophy and ethics, (ii) law and governance, and (iii) design and engineering. The introductory chapter addresses the motivations and recent developments in MHC, introducing each perspective and related chapters. These three disciplinary perspectives scrutinize how MHC intertwines with philosophical debates on moral responsibility, societal concerns regarding control over technological advancements in legal frameworks, and the engineering complexities of designing and developing AI systems while ensuring human control and responsibility. Additionally, cross-cutting aspects on MHC over AI systems are also introduced and discussed through (iv) interdisciplinary and systemic perspectives. By offering a contextualized introduction to the perspectives considered in this handbook, this chapter aims to present the handbook’s various approaches and points of interest for a diverse audience, highlighting potential entry points into this multidisciplinary volume.

@en

The urban traffic environment is characterized by the presence of a highly differentiated pool of users, including vulnerable ones. This makes vehicle automation particularly difficult to implement, as a safe coordination among those users is hard to achieve in such an open scenario. Different strategies have been proposed to address these coordination issues, but all of them have been found to be costly for they negatively affect a range of human values (e.g. safety, democracy, accountability…). In this paper, we claim that the negative value impacts entailed by each of these strategies can be interpreted as lack of what we call Meaningful Human Control over different parts of a sociotechnical system. We argue that Meaningful Human Control theory provides the conceptual tools to reduce those unwanted consequences, and show how “designing for meaningful human control” constitutes a valid strategy to address coordination issues. Furthermore, we showcase a possible application of this framework in a highly dynamic urban scenario, aiming to safeguard important values such as safety, democracy, individual autonomy, and accountability. Our meaningful human control framework offers a perspective on coordination issues that allows to keep human actors in control while minimizing the active, operational role of the drivers. This approach makes ultimately possible to promote a safe and responsible transition to full automation.

@en

The present chapter analyses the complex ways in which driving automation affects human autonomy with the aim of raising awareness on the design and policy challenges that must be faced to effectively align future transportation systems to this ethical value. Building on the European report Ethics of Connected and Automated Vehicles, we consider three dimensions of the relation between human autonomy and driving automation: autonomy as self-determination of driving decisions; autonomy as freedom to pursue a good life through mobility; and, finally, autonomy as the capacity and opportunity to influence social deliberation concerning transportation policies and planning. In doing so, the chapter shows that delegating driving tasks to CAVs might both infringe and support user autonomy, thus calling for a reconsideration of widespread frameworks concerning the role of humans and technological systems in this domain. Moreover, it stresses the importance of promoting inclusive and participated decision-making processes on transportation policies and planning, so to avoid situations where the development and adoption of transport innovations are led by agents willing to respond only to a limited set of stakeholders’ needs.@en

The paper presents a framework to realise “meaningful human control” over Automated Driving Systems. The framework is based on an original synthesis of the results of the multidisciplinary research project “Meaningful Human Control over Automated Driving Systems” lead by a team of engineers, philosophers, and psychologists at Delft University of the Technology from 2017 to 2021. Meaningful human control aims at protecting safety and reducing responsibility gaps. The framework is based on the core assumption that human persons and institutions, not hardware and software and their algorithms, should remain ultimately—though not necessarily directly—in control of, and thus morally responsible for, the potentially dangerous operation of driving in mixed traffic. We propose an Automated Driving System to be under meaningful human control if it behaves according to the relevant reasons of the relevant human actors (tracking), and that any potentially dangerous event can be related to a human actor (tracing). We operationalise the requirements for meaningful human control through multidisciplinary work in philosophy, behavioural psychology and traffic engineering. The tracking condition is operationalised via a proximal scale of reasons and the tracing condition via an evaluation cascade table. We review the implications and requirements for the behaviour and skills of human actors, in particular related to supervisory control and driver education. We show how the evaluation cascade table can be applied in concrete engineering use cases in combination with the definition of core components to expose deficiencies in traceability, thereby avoiding so-called responsibility gaps. Future research directions are proposed to expand the philosophical framework and use cases, supervisory control and driver education, real-world pilots and institutional embedding@en

Automated vehicles (AVs) aim to dramatically improve traffic safety by reducing or eliminating human error, which remains the leading cause of road crashes. However, commonly accepted standards for the ‘safe driving behaviour of machines’ are pending and urgently needed. Unless a common understanding of safety as a design value is achieved, different manufacturers’ driving styles may emerge, resulting in inconsistent, unpredictable and potentially unsafe ‘behaviour’ of AVs in certain situations. This paper aims to explore the main gaps and challenges towards establishing shared safety standards for the ‘behaviour’ of AVs, and contribute to their responsible traffic integration, by reviewing the state-of-the-art on AV safety in the core relevant disciplines: ethics of technology, safety science (engineering & human factors), and standardisation. The ethical and safety aspects investigated include the users’ perception of AV safety, the ethical trade-offs in critical decision-making contexts, the pertinence of data-driven approaches for AVs to mimic human behaviour, and the responsibilities of various actors. Moreover, the paper reviews the current safety patterns, metrics (surrogate measures of safety – SMoS) and their thresholds introduced in existing research for three use cases: mixed traffic of AV and conventional vehicles, AV interaction with pedestrians and cyclists, and transition of control from machine to human driver. The results reveal several knowledge gaps within each discipline and highlights the lack of common understanding of safety across disciplines. On the basis of the results, the paper proposes a framework for further research on AV safety, identifying concrete opportunities for interdisciplinary research, with common goals and methodologies, and explicitly indicating the path for transfer of knowledge between sectors.

@en

The notion of “responsibility gap” with artificial intelligence (AI) was originally introduced in the philosophical debate to indicate the concern that “learning automata” may make more difficult or impossible to attribute moral culpability to persons for untoward events. Building on literature in moral and legal philosophy, and ethics of technology, the paper proposes a broader and more comprehensive analysis of the responsibility gap. The responsibility gap, it is argued, is not one problem but a set of at least four interconnected problems – gaps in culpability, moral and public accountability, active responsibility—caused by different sources, some technical, other organisational, legal, ethical, and societal. Responsibility gaps may also happen with non-learning systems. The paper clarifies which aspect of AI may cause which gap in which form of responsibility, and why each of these gaps matter. It proposes a critical review of partial and non-satisfactory attempts to address the responsibility gap: those which present it as a new and intractable problem (“fatalism”), those which dismiss it as a false problem (“deflationism”), and those which reduce it to only one of its dimensions or sources and/or present it as a problem that can be solved by simply introducing new technical and/or legal tools (“solutionism”). The paper also outlines a more comprehensive approach to address the responsibility gaps with AI in their entirety, based on the idea of designing socio-technical systems for “meaningful human control", that is systems aligned with the relevant human reasons and capacities.@en

The paper has two goals. The first is presenting the main results of the recent report Ethics of Connected and Automated Vehicles: recommendations on road safety, privacy, fairness, explainability and responsibility written by the Horizon 2020 European Commission Expert Group to advise on specific ethical issues raised by driverless mobility, of which the author of this paper has been member and rapporteur. The second is presenting some broader ethical and philosophical implications of these recommendations, and using these to contribute to the establishment of Ethics of Transportation as an independent branch of applied ethics. The recent debate on the ethics of Connected and Automated Vehicles (CAVs) presents a paradox and an opportunity. The paradox is the presence of a flourishing debate on the ethics of one very specific transportation technology without ethics of transportation being in itself a well-established academic discipline. The opportunity is that now that a spotlight has been switched on the ethical dimensions of CAVs it may be easier to establish a broader debate on ethics of transportation. While the 20 recommendations of the EU report are grouped in three macro-areas: road safety, data ethics, and responsibility, in this paper they will be grouped according to eight philosophical themes: Responsible Innovation, road justice, road safety, freedom, human control, privacy, data fairness, responsibility. These are proposed as the first topics for a new ethics of transportation.

@en

In this article the title and running title were incorrectly given as ‘Marc Coeckelbergh, AI Ethics, Mit Press, 2021’ but should have been ’Mark Coeckelbergh, AI Ethics, Mit Press, 2021’. The original article has been corrected.@en

Increased on-road testing and market availability of partially automated vehicles (AV) offers researchers and developers the opportunity to evaluate the AV’s performance. The occurrence of new types of accidents involving AV’s has sparked questions in regard to who is actually in control over and responsible for AV control. In this contribution, we suggest a potential discrepancy in AV control with the review of recently documented accidents involving AV’s. The identification of a gap in control is performed using a recently formulated moral philosophical framework of Meaningful Human Control (MHC). This shows a discrepancy between the attribution of responsibility and the ability of a human to fulfil the role assigned to them. While a gap in control is not evident from the viewpoint of operational control, it requires the more intricate concept of MHC to expose it. Recommendations are further made that AV developers and vehicle approval authorities should consider control from a MHC perspective to avoid future gaps in control with the resulting consequences.@en

Accountability and responsibility are key concepts in the academic and societal debate on Autonomous Weapon Systems, but these notions are often used as high-level overarching constructs and are not operationalised to be useful in practice. “Meaningful Human Control” is often mentioned as a requirement for the deployment of Autonomous Weapon Systems, but a common definition of what this notion means in practice, and a clear understanding of its relation with responsibility and accountability is also lacking. In this paper, we present a definition of these concepts and describe the relations between accountability, responsibility, control and oversight in order to show how these notions are distinct but also connected. We focus on accountability as a particular form of responsibility—the obligation to explain one’s action to a forum—and we present three ways in which the introduction of Autonomous Weapon Systems may create “accountability gaps”. We propose a Framework for Comprehensive Human Oversight based on an engineering, socio-technical and governance perspective on control. Our main claim is that combining the control mechanisms at technical, socio-technical and governance levels will lead to comprehensive human oversight over Autonomous Weapon Systems which may ensure solid controllability and accountability for the behaviour of Autonomous Weapon Systems. Finally, we give an overview of the military control instruments that are currently used in the Netherlands and show the applicability of the comprehensive human oversight Framework to Autonomous Weapon Systems. Our analysis reveals two main gaps in the current control mechanisms as applied to Autonomous Weapon Systems. We have identified three first options as future work for the design of a control mechanism, one in the technological layer, one in the socio-technical layer and one the governance layer, in order to achieve comprehensive human oversight and ensure accountability over Autonomous Weapon Systems.

@en

This paper proposes that autonomous vehicles should be designed to reduce light pollution. In support of this specific proposal, a moral assessment of autonomous vehicles more comprehensive than the dilemmatic life-and-death questions of trolley problem-style situations is presented. The paper therefore consists of two interre-lated arguments. The first is that autonomous vehicles are currently still a technol-ogy in development, and not one that has acquired its definitive shape, meaning the design of both the vehicles and the surrounding infrastructure is open-ended. Design for values is utilized to articulate a path forward, by which engineering ethics should strive to incorporate values into a technology during its development phase. Second, it is argued that nighttime lighting—a critical supporting infrastructure—should be a prima facie consideration for autonomous vehicles during their development phase. It is shown that a reduction in light pollution, and more boldly a better balance of lighting and darkness, can be achieved via the design of future autonomous vehicles. Two case studies are examined (parking lots and highways) through which autono-mous vehicles may be designed for “driving in the dark.” Nighttime lighting issues are thus inserted into a broader ethics of autonomous vehicles, while simultaneously introducing questions of autonomous vehicles into debates about light pollution.@en

In the political debate on Autonomous Weapon Systems strong views and opinions are voiced, but empirical research to support these opinions is lacking. Insight into which moral values are related to the deployment of Autonomous Weapon Systems is missing. We describe the empirical results of two studies on moral values regarding Autonomous Weapon Systems that aim to understand the perception of people pertaining to the introduction of Autonomous Weapon Systems. One study consists of a sample of military personnel of the Dutch Ministry of Defense and the second study contains a sample of civilians. The results indicate both groups are more anxious about the deployment of Autonomous Weapon Systems than about the deployment of Human Operated drones, and that they perceive Autonomous Weapon Systems to have less respect for the dignity of human life. The concerns for Autonomous Weapon Systems creating new kinds of psychological and moral harm is very present in the public debate, and this is in our opinion one element that deserves to be carefully considered in future debates on the ethics of the design and deployment of Autonomous Weapon Systems. The results of these studies reveal a common ground regarding the moral values of human dignity and anxiety pertaining the introduction of Autonomous Weapon Systems which could further the ethical debate.

@en

In this paper, in line with the general framework of value-sensitive design, we aim to operationalize the general concept of “Meaningful Human Control” (MHC) in order to pave the way for its translation into more specific design requirements. In particular, we focus on the operationalization of the first of the two conditions (Santoni de Sio and Van den Hoven 2018) investigated: the so-called ‘tracking’ condition. Our investigation is led in relation to one specific subcase of automated system: dual-mode driving systems (e.g. Tesla ‘autopilot’). First, we connect and compare meaningful human control with a concept of control very popular in engineering and traffic psychology (Michon 1985), and we explain to what extent tracking resembles and differs from it. This will help clarifying the extent to which the idea of meaningful human control is connected to, but also goes beyond, current notions of control in engineering and psychology. Second, we take the systematic analysis of practical reasoning as traditionally presented in the philosophy of human action (Anscombe, Bratman, Mele) and we adapt it to offer a general framework where different types of reasons and agents are identified according to their relation to an automated system’s behaviour. This framework is meant to help explaining what reasons and what agents (should) play a role in controlling a given system, thereby enabling policy makers to produce usable guidelines and engineers to design systems that properly respond to selected human reasons. In the final part, we discuss a practical example of how our framework could be employed in designing automated driving systems.

@en

Automated driving systems (ADS) with partial automation are currently available for the consumer. They are potentially beneficial to traffic flow, fuel consumption, and safety, but human behaviour whilst driving with ADS is poorly understood. Human behaviour is currently expected to lead to dangerous circumstances as ADS could place human drivers ‘out-of-the-loop’ or cause other types of adverse behavioural adaptation. This article introduces the concept of ‘meaningful human control’ to better address the challenges raised by ADS, and presents a new framework of human control over ADS by means of literature-based categorisation. Using standards set by European authorities for driver skills and road rules, this framework offers a unique, quantified perspective into the effects of ADS on human behaviour. One main result is a rapid and inconsistent decrease in required skill- and rule-based behaviour mismatching with the increasing amount of required knowledge-based behaviour. Furthermore, the development of higher levels of automation currently requires different human behaviour than feasible, as a mismatch between supply and demand in terms of behaviour arises. Implications, discrepancies and emerging mismatches this framework elicits are discussed, and recommendations towards future design strategies and research opportunities are made to provide a meaningful transition of human control over ADS.

@en

Truck platooning is a form of vehicle automation and cooperation that is leading the way for cooperative and automated vehicle implementation. However, much is still unknown about the effects and potential dangers of many situations in regard to cooperative control of these platoons. In this contribution, we discuss many of the challenges in regard to full platoon control, we give concepts that can answer some of the questions and make recommendations on how full platoon control should be considered by truck manufactures, ADS software developers and policy makers. A main concept that is applied is that of Meaningful Human Control (MHC). We furthermore consider driver 'reasons', both distal and proximal, to identify correct chains of MHC. We conclude that each part of a system should be responsive to the maximum amount of relevant reasons available and the availability of relevant reasons should be maximized to obtain sufficient MHC.

@en

High expectations rest Automated Driving Systems (ADS) to drastically transform roadway transport in the coming decades. In our project we aim at guiding a responsible transition of control toward automated driving. As a first step, we will develop a theory of “meaningful human control” over ADS, and translate the theory into design guidelines, both at the technical and at the institutional level. To maximize human safety, and avoid the creation of accountability gaps, meaningful human control should constantly be maintained over autonomous and semi-autonomous systems. The notion of meaningful human control, originated within the debate about military drones, has –as of yet– not been applied in the context of ADS. The divergent appearance of vehicles with increasing levels of automated control calls for consideration hereof. Therefore, an interdisciplinary team of philosophers, traffic engineers and behavioral scientists will work at defining the conditions for human control, and its “meaningful” elements, and mold it into a workable framework of human control. The validity of this framework will then be tested through empirical research and with the use of automated vehicle prototypes. In conducting the research, particular attention will be dedicated to ensure that the developed theoretical framework can be fully operationalized into empirical science and design recommendations. This will be facilitated by both the interdisciplinary nature of the project itself and by the active participation of a number of private and public partners, who will in turn be provided with up to date research results. In particular, designers, manufacturers and road operators will receive conclusions drawn from empirical and theoretical research they can apply in developing automated systems that achieve meaningful human control; policy-makers can use our findings to elaborate regulations that promote both innovation and human values; lawyers and insurance companies will receive original inputs for the design of liability and insurance schemes; driving licensing bodies will receive data to use for developing new procedures. In the first stage of our project, we are aiming to develop an empirically and technically usable conceptual toolbox (or framework) that can be shared across the different areas of expertise that characterize the project. We will isolate a minimal set of notions of control, based on literature from, among others, philosophy, behavioral science, and engineering. Simultaneously, we aim to identify an as clear as possible notion of “meaningfulness”. On that regard, we started from a philosophical notion of meaningful human control over ADS, to investigate whether, and to what extent, its elements could be operationalized and tested within an empirical framework. We have presented our project to our private and public partners, and to a number of other stakeholders involved in automated driving. From this presentation, we have received positive and constructive feedback, and encouraged us to continue our communication to the public. In particular, we aim to structure a bi-directional communication avenue between data production and data use. Ideally, this would benefit both sides, providing the researchers with societally relevant research questions and practical constraints, and the stakeholders with timely access to scientific results.@en