The perils and pitfalls of explainable AI

The perils and pitfalls of explainable AI: Strategies for explaining algorithmic decision-making

de Bruijn, J.A. (TU Delft Organisation & Governance)
Warnier, Martijn (TU Delft Multi Actor Systems)
Janssen, M.F.W.H.A. (TU Delft Information and Communication Technology)

Multi Actor Systems

2022

Governments look at explainable artificial intelligence's (XAI) potential to tackle the criticisms of the opaqueness of algorithmic decision-making with AI. Although XAI is appealing as a solution for automated decisions, the wicked nature of the challenges governments face complicates the use of XAI. Wickedness means that the facts that define a problem are ambiguous and that there is no consensus on the normative criteria for solving this problem. In such a situation, the use of algorithms can result in distrust. Whereas there is much research advancing XAI technology, the focus of this paper is on strategies for explainability. Three illustrative cases are used to show that explainable, data-driven decisions are often not perceived as objective by the public. The context might raise strong incentives to contest and distrust the explanation of AI, and as a consequence, fierce resistance from society is encountered. To overcome the inherent problems of XAI, decisions-specific strategies are proposed to lead to societal acceptance of AI-based decisions. We suggest strategies to embrace explainable decisions and processes, co-create decisions with societal actors, move away from an instrumental to an institutional approach, use competing and value-sensitive algorithms, and mobilize the tacit knowledge of professionals

Accountability
Algorithms
Artificial intelligence
Computational intelligence
Data-driven decision
E-government
Socio-tech
Transparency
Trust
XAI

http://resolver.tudelft.nl/uuid:0f6a066c-9652-4b87-95fa-a30cc9775128

https://doi.org/10.1016/j.giq.2021.101666

0740-624X

Government Information Quarterly: an international journal of information technology management, policies, and practices, 39 (2)

Institutional Repository

journal article

© 2022 J.A. de Bruijn, Martijn Warnier, M.F.W.H.A. Janssen