document

Human–machine coordination in mixed traffic as a problem of Meaningful Human Control

Mecacci, Giulio (author), Calvert, S.C. (author), Santoni De Sio, F. (author)

The urban traffic environment is characterized by the presence of a highly differentiated pool of users, including vulnerable ones. This makes vehicle automation particularly difficult to implement, as a safe coordination among those users is hard to achieve in such an open scenario. Different strategies have been proposed to address these...

journal article 2023

document

Meaningful human control: actionable properties for AI system development

Cavalcante Siebert, L. (author), Lupetti, M.L. (author), Aizenberg, E. (author), Beckers, N.W.M. (author), Zgonnikov, A. (author), Veluwenkamp, H.M. (author), Abbink, D.A. (author), Giaccardi, Elisa (author), Houben, G.J.P.M. (author), Jonker, C.M. (author), van den Hoven, M.J. (author), Forster, D. (author), Lagendijk, R.L. (author)

How can humans remain in control of artificial intelligence (AI)-based systems designed to perform tasks autonomously? Such systems are increasingly ubiquitous, creating benefits - but also undesirable situations where moral responsibility for their actions cannot be properly attributed to any particular person or group. The concept of...

journal article 2022

document

Machine Learning and Counter-Terrorism: Ethics, Efficacy, and Meaningful Human Control

Robbins, S.A. (author)

Machine Learning (ML) is reaching the peak of a hype cycle. If you can think of a personal or grand societal challenge – then ML is being proposed to solve it. For example, ML is purported to be able to assist in the current global pandemic by predicting COVID-19 outbreaks and identifying carriers (see, e.g., Ardabili et al. 2020). ML can make...

doctoral thesis 2021

document

Moral Decision Making in Human-Agent Teams: Human Control and the Role of Explanations

van der Waa, J.S. (author), Verdult, Sabine (author), van den Bosch, Karel (author), van Diggelen, Jurriaan (author), Haije, Tjalling (author), van der Stigchel, Birgit (author), Cocu, Ioana (author)

With the progress of Artificial Intelligence, intelligent agents are increasingly being deployed in tasks for which ethical guidelines and moral values apply. As artificial agents do not have a legal position, humans should be held accountable if actions do not comply, implying humans need to exercise control. This is often labeled as...

journal article 2021

document

Four Responsibility Gaps with Artificial Intelligence: Why they Matter and How to Address them

Santoni De Sio, F. (author), Mecacci, Giulio (author)

The notion of “responsibility gap” with artificial intelligence (AI) was originally introduced in the philosophical debate to indicate the concern that “learning automata” may make more difficult or impossible to attribute moral culpability to persons for untoward events. Building on literature in moral and legal philosophy, and ethics of...

journal article 2021

document

How to ensure control of cooperative vehicle and truck platoons using meaningful human control

Calvert, S.C. (author), Mecacci, Giulio (author), Heikoop, Daniel D. (author), Janssen, Robbert (author)

Vehicle cooperation, not vehicle automation, will yield the greatest benefits on road traffic. However, satisfactory human control over platoons of cooperative vehicles still has a large number of uncertainties and issues to be addressed. This paper aims to address these broader issues of control over a cooperative vehicle platoon by...

journal article 2021

document

Meaningful human control over opaque machines

Robbins, S.A. (author)

conference paper 2020

document

Accountability and Control Over Autonomous Weapon Systems: A Framework for Comprehensive Human Oversight

Verdiesen, E.P. (author), Santoni De Sio, F. (author), Dignum, M.V. (author)

Accountability and responsibility are key concepts in the academic and societal debate on Autonomous Weapon Systems, but these notions are often used as high-level overarching constructs and are not operationalised to be useful in practice. “Meaningful Human Control” is often mentioned as a requirement for the deployment of Autonomous Weapon...

journal article 2020

document

Allocation of moral decision-making in human-agent teams: a pattern approach

van der Waa, J.S. (author), van Diggelen, Jurriaan (author), Cavalcante Siebert, L. (author), Neerincx, M.A. (author), Jonker, C.M. (author)

Artificially intelligent agents will deal with more morally sensitive situations as the field of AI progresses. Research efforts are made to regulate, design and build Artificial Moral Agents (AMAs) capable of making moral decisions. This research is highly multidisciplinary with each their own jargon and vision, and so far it is unclear...

conference paper 2020

document

A Conceptual Control System Description of Cooperative and Automated Driving in Mixed Urban Traffic With Meaningful Human Control for Design and Evaluation

Calvert, S.C. (author), Mecacci, G. (author)

The introduction of automated vehicles means that some or all operational control over these vehicles is diverted away from a human driver to a technological system. The concept of Meaningful Human Control (MHC) was derived to address control issues over automated systems, allowing a system to explicitly consider human intentions and reasons....

journal article 2020

document

Gaps in the control of automated vehicles on roads

Calvert, S.C. (author), Mecacci, G. (author), van Arem, B. (author), Santoni De Sio, F. (author), Heikoop, D.D. (author), Hagenzieker, Marjan (author)

Increased on-road testing and market availability of partially automated vehicles (AV) offers researchers and developers the opportunity to evaluate the AV’s performance. The occurrence of new types of accidents involving AV’s has sparked questions in regard to who is actually in control over and responsible for AV control. In this contribution,...

review 2020

document

Meaningful human control as reason-responsiveness: the case of dual-mode vehicles

Mecacci, G. (author), Santoni De Sio, F. (author)

In this paper, in line with the general framework of value-sensitive design, we aim to operationalize the general concept of “Meaningful Human Control” (MHC) in order to pave the way for its translation into more specific design requirements. In particular, we focus on the operationalization of the first of the two conditions (Santoni de Sio...

journal article 2019

document

Human behaviour with automated driving systems: A quantitative framework for meaningful human control

Heikoop, D.D. (author), Hagenzieker, Marjan (author), Mecacci, G. (author), Calvert, S.C. (author), Santoni De Sio, F. (author), van Arem, B. (author)

Automated driving systems (ADS) with partial automation are currently available for the consumer. They are potentially beneficial to traffic flow, fuel consumption, and safety, but human behaviour whilst driving with ADS is poorly understood. Human behaviour is currently expected to lead to dangerous circumstances as ADS could place human...

journal article 2019

document

AI and the path to envelopment: knowledge as a first step towards the responsible regulation and use of AI-powered machines

Robbins, S.A. (author)

With Artificial Intelligence (AI) entering our lives in novel ways—both known and unknown to us—there is both the enhancement of existing ethical issues associated with AI as well as the rise of new ethical issues. There is much focus on opening up the ‘black box’ of modern machine-learning algorithms to understand the reasoning behind their...

journal article 2019

document

A Misdirected Principle with a Catch: Explicability for AI

Robbins, S.A. (author)

There is widespread agreement that there should be a principle requiring that artificial intelligence (AI) be ‘explicable’. Microsoft, Google, the World Economic Forum, the draft AI ethics guidelines for the EU commission, etc. all include a principle for AI that falls under the umbrella of ‘explicability’. Roughly, the principle states that ...

journal article 2019

document

Meaningful Human Control over Automated Driving Systems

Heikoop, D.D. (author), Hagenzieker, Marjan (author), Mecacci, G. (author), Santoni De Sio, F. (author), Calvert, S.C. (author), van Arem, B. (author)

Human Factors issues with automated driving systems (ADS) are becoming more apparent with the increasing prevalence of automated vehicles on the public roads. As automated driving demands increased performance of supervisory skills of the driver, rather than vehicle handling skills, a mismatch occurs between the demand and supply of the drivers’...

conference paper 2018

document

Meaningful Human Control over Automated Driving Systems

Heikoop, D.D. (author), Hagenzieker, Marjan (author), Mecacci, G. (author), Santoni De Sio, F. (author), Calvert, S.C. (author), van Arem, B. (author)

poster 2018

document

Meaningful Human Control Over Autonomous Systems: A Philosophical Account

Santoni De Sio, F. (author), van den Hoven, M.J. (author)

Debates on lethal autonomous weapon systems have proliferated in the past 5 years. Ethical concerns have been voiced about a possible raise in the number of wrongs and crimes in military operations and about the creation of a “responsibility gap” for harms caused by these systems. To address these concerns, the principle of “meaningful human...

journal article 2018

document

Full platoon control in Truck Platooning: A Meaningful Human Control perspective

Calvert, S.C. (author), Mecacci, G. (author), Heikoop, D.D. (author), Santoni De Sio, F. (author)

Truck platooning is a form of vehicle automation and cooperation that is leading the way for cooperative and automated vehicle implementation. However, much is still unknown about the effects and potential dangers of many situations in regard to cooperative control of these platoons. In this contribution, we discuss many of the challenges in...

conference paper 2018