Realising Meaningful Human Control Over Automated Driving Systems

Realising Meaningful Human Control Over Automated Driving Systems: A Multidisciplinary Approach

Santoni De Sio, F. (TU Delft Ethics & Philosophy of Technology)
Mecacci, G. (TU Delft Ethics & Philosophy of Technology; Radboud Universiteit Nijmegen)
Calvert, S.C. (TU Delft Transport and Planning)
Heikoop, D.D. (TU Delft Transport and Planning)
Hagenzieker, Marjan (TU Delft Transport and Planning)
van Arem, B. (TU Delft Transport and Planning)

2022

The paper presents a framework to realise “meaningful human control” over Automated Driving Systems. The framework is based on an original synthesis of the results of the multidisciplinary research project “Meaningful Human Control over Automated Driving Systems” lead by a team of engineers, philosophers, and psychologists at Delft University of the Technology from 2017 to 2021. Meaningful human control aims at protecting safety and reducing responsibility gaps. The framework is based on the core assumption that human persons and institutions, not hardware and software and their algorithms, should remain ultimately—though not necessarily directly—in control of, and thus morally responsible for, the potentially dangerous operation of driving in mixed traffic. We propose an Automated Driving System to be under meaningful human control if it behaves according to the relevant reasons of the relevant human actors (tracking), and that any potentially dangerous event can be related to a human actor (tracing). We operationalise the requirements for meaningful human control through multidisciplinary work in philosophy, behavioural psychology and traffic engineering. The tracking condition is operationalised via a proximal scale of reasons and the tracing condition via an evaluation cascade table. We review the implications and requirements for the behaviour and skills of human actors, in particular related to supervisory control and driver education. We show how the evaluation cascade table can be applied in concrete engineering use cases in combination with the definition of core components to expose deficiencies in traceability, thereby avoiding so-called responsibility gaps. Future research directions are proposed to expand the philosophical framework and use cases, supervisory control and driver education, real-world pilots and institutional embedding

http://resolver.tudelft.nl/uuid:c23d8c24-1d61-4f59-8fe4-4c25ad039dde

https://doi.org/10.1007/s11023-022-09608-8

2023-01-28

1572-8641

Minds and Machines: journal for artificial intelligence, philosophy and cognitive sciences, 33 (4), 587-611

Green Open Access added to TU Delft Institutional Repository 'You share, we take care!' - Taverne project https://www.openaccess.nl/en/you-share-we-take-care Otherwise as indicated in the copyright section: the publisher is the copyright holder of this work and the author uses the Dutch legislation to make this work public.

Institutional Repository

journal article

© 2022 F. Santoni De Sio, G. Mecacci, S.C. Calvert, D.D. Heikoop, Marjan Hagenzieker, B. van Arem