Spotting When Algorithms Are Wrong

Journal article (2022)

Authors

S.N.R. Buijsman Ethics & Philosophy of Technology - TPM
H.M. Veluwenkamp Ethics & Philosophy of Technology - TPM
Research Group
Ethics & Philosophy of Technology (TPM) (TU Delft)
Copyright: © 2022 S.N.R. Buijsman, H.M. Veluwenkamp
DOI: https://doi.org/10.1007/s11023-022-09591-0
Sociotechnical systems Defeaters Epistemic dependence Oversight
More Info
expand_more

Published Date

2022

Language

English

Reuse Rights

Other than for strictly personal use, it is not permitted to download, forward or distribute the text or part of it, without the consent of the author(s) and/or copyright holder(s), unless the work is under an open content license such as Creative Commons.

Faculty

Technology, Policy and Management

Department

Values Technology and Innovation

Research Group

Ethics & Philosophy of Technology

Abstract

Users of sociotechnical systems often have no way to independently verify whether the system output which they use to make decisions is correct; they are epistemically dependent on the system. We argue that this leads to problems when the system is wrong, namely to bad decisions and violations of the norm of practical reasoning. To prevent this from occurring we suggest the implementation of defeaters: information that a system is unreliable in a specific case (undercutting defeat) or independent information that the output is wrong (rebutting defeat). Practically, we suggest to design defeaters based on the different ways in which a system might produce erroneous outputs, and analyse this suggestion with a case study of the risk classification algorithm used by the Dutch tax agency.