Improving Confidence in the Estimation of Values and Norms

Conference Paper (2020)
Author(s)

Luciano Cavalcante Siebert (TU Delft - Interactive Intelligence)

R.A. Mercuur (TU Delft - Information and Communication Technology)

M.V. Dignum (Umeå University, TU Delft - Information and Communication Technology)

MJ van den Hoven (TU Delft - Ethics & Philosophy of Technology)

CM Jonker (TU Delft - Interactive Intelligence)

Research Group
Interactive Intelligence
Copyright
© 2020 L. Cavalcante Siebert, R.A. Mercuur, M.V. Dignum, M.J. van den Hoven, C.M. Jonker
DOI related publication
https://doi.org/10.1007/978-3-030-72376-7_6
More Info
expand_more
Publication Year
2020
Language
English
Copyright
© 2020 L. Cavalcante Siebert, R.A. Mercuur, M.V. Dignum, M.J. van den Hoven, C.M. Jonker
Research Group
Interactive Intelligence
Pages (from-to)
98-113
ISBN (print)
9783030723750
ISBN (electronic)
978-3-030-72376-7
Reuse Rights

Other than for strictly personal use, it is not permitted to download, forward or distribute the text or part of it, without the consent of the author(s) and/or copyright holder(s), unless the work is under an open content license such as Creative Commons.

Abstract

Autonomous agents (AA) will increasingly be interacting with us in our daily lives. While we want the benefits attached to AAs, it is essential that their behavior is aligned with our values and norms. Hence, an AA will need to estimate the values and norms of the humans it interacts with, which is not a straightforward task when solely observing an agent's behavior. This paper analyses to what extent an AA is able to estimate the values and norms of a simulated human agent (SHA) based on its actions in the ultimatum game. We present two methods to reduce ambiguity in profiling the SHAs: one based on search space exploration and another based on counterfactual analysis. We found that both methods are able to increase the confidence in estimating human values and norms, but differ in their applicability, the latter being more efficient when the number of interactions with the agent is to be minimized. These insights are useful to improve the alignment of AAs with human values and norms.

Files

Siebert2021_Chapter_ImprovingC... (pdf)
(pdf | 0.83 Mb)
- Embargo expired in 28-07-2021
License info not available