Print Email Facebook Twitter Computer Says I Don’t Know Title Computer Says I Don’t Know: An Empirical Approach to Capture Moral Uncertainty in Artificial Intelligence Author Martins Martinho Bessa, A.C. (TU Delft Transport and Logistics) Kroesen, M. (TU Delft Transport and Logistics) Chorus, C.G. (TU Delft Transport and Logistics) Date 2021 Abstract As AI Systems become increasingly autonomous, they are expected to engage in decision-making processes that have moral implications. In this research we integrate theoretical and empirical lines of thought to address the matters of moral reasoning and moral uncertainty in AI Systems. We reconceptualize the metanormative framework for decision-making under moral uncertainty and we operationalize it through a latent class choice model. The core idea being that moral heterogeneity in society can be codified in terms of a small number of classes with distinct moral preferences and that this codification can be used to express moral uncertainty of an AI. Choice analysis allows for the identification of classes and their moral preferences based on observed choice data. Our reformulation of the metanormative framework is theory-rooted and practical in the sense that it avoids runtime issues in real time applications. To illustrate our approach we conceptualize a society in which AI Systems are in charge of making policy choices. While one of the systems uses a baseline morally certain model, the other uses a morally uncertain model. We highlight cases in which the AI Systems disagree about the policy to be chosen, thus illustrating the need to capture moral uncertainty in AI systems. Subject Artificial IntelligenceDiscrete Choice AnalysisLatent Class Choice ModelMetanormative TheoryMoralityUncertainty To reference this document use: http://resolver.tudelft.nl/uuid:abb48e9f-b1a6-4522-bb22-cb23a18fb55d DOI https://doi.org/10.1007/s11023-021-09556-9 ISSN 0924-6495 Source Minds and Machines: journal for artificial intelligence, philosophy and cognitive sciences, 31 (2), 215-237 Part of collection Institutional Repository Document type journal article Rights © 2021 A.C. Martins Martinho Bessa, M. Kroesen, C.G. Chorus Files PDF Martinho2021_Article_Comp ... mpiric.pdf 1.02 MB Close viewer /islandora/object/uuid:abb48e9f-b1a6-4522-bb22-cb23a18fb55d/datastream/OBJ/view