Title
Improving Confidence in the Estimation of Values and Norms
Author
Cavalcante Siebert, L. (TU Delft Interactive Intelligence) 
Mercuur, R.A. (TU Delft Information and Communication Technology)
Dignum, M.V. (TU Delft Information and Communication Technology; Umeå University)
van den Hoven, M.J. (TU Delft Ethics & Philosophy of Technology)
Jonker, C.M. (TU Delft Interactive Intelligence)
Contributor
Aler Tubella, Andrea (editor)
Cranefield, Stephen (editor)
Frantz, Christopher (editor)
Meneguzzi, Felipe (editor)
Vasconcelos, Wamberto (editor)
Date
2020
Abstract
Autonomous agents (AA) will increasingly be interacting with us in our daily lives. While we want the benefits attached to AAs, it is essential that their behavior is aligned with our values and norms. Hence, an AA will need to estimate the values and norms of the humans it interacts with, which is not a straightforward task when solely observing an agent's behavior. This paper analyses to what extent an AA is able to estimate the values and norms of a simulated human agent (SHA) based on its actions in the ultimatum game. We present two methods to reduce ambiguity in profiling the SHAs: one based on search space exploration and another based on counterfactual analysis. We found that both methods are able to increase the confidence in estimating human values and norms, but differ in their applicability, the latter being more efficient when the number of interactions with the agent is to be minimized. These insights are useful to improve the alignment of AAs with human values and norms.
Subject
Autonomous agents
Norms
Ultimatum game
Values
To reference this document use:
http://resolver.tudelft.nl/uuid:b1f3f93c-ff85-4382-b90c-89ca8a3eeb72
DOI
https://doi.org/10.1007/978-3-030-72376-7_6
Publisher
Cornell University Library - arXiv.org
Embargo date
2021-07-28
ISBN
9783030723750
Source
Coordination, Organizations, Institutions, Norms, and Ethics for Governance of Multi-Agent Systems XIII - International Workshops COIN 2017 and COINE 2020, Revised Selected Papers: International Workshops COIN 2017 and COINE 2020 Sao Paulo, Brazil, May 8–9, 2017 and Virtual Event, May 9, 2020 Revised Selected Papers
Series
Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), 0302-9743, 12298 LNAI
Bibliographical note
Green Open Access added to TU Delft Institutional Repository ‘You share, we take care!’ – Taverne project https://www.openaccess.nl/en/you-share-we-take-care Otherwise as indicated in the copyright section: the publisher is the copyright holder of this work and the author uses the Dutch legislation to make this work public.
Part of collection
Institutional Repository
Document type
conference paper
Rights
© 2020 L. Cavalcante Siebert, R.A. Mercuur, M.V. Dignum, M.J. van den Hoven, C.M. Jonker