Explainability and its emerging counterpart contestability are key normative and design principles for trustworthy AI, enabling users and subjects to understand and challenge AI decisions. Yet realizing these principles is difficult, as they take on different meanings across technical, legal, and organizational dimensions of AI regulation. To address this conceptual polysemy, we report findings from an interview study with 14 experts examining the intersection and implementation of explainability and contestability, and their interpretations in different research communities. We outline differentiations between descriptive and normative explainability, judicial and non-judicial channels of contestation, and individual and collective contestation action. We also identify key points of friction in realizing both principles, including alignment between top-down and bottom-up regulation, assignment of responsibility, and the need for interdisciplinary collaboration. Finally, we offer three AI policy recommendations to operationalize explainability and contestability through a Regulation-by-Design perspective. Our contributions inform policy research and regulation of these core principles, and support more effective and equitable design, development, and deployment of trustworthy public AI systems.

The question we explore in this paper is: What are the current constraints and challenges of ladder-based participation models in participatory smart city R&D, and how might learning-oriented approaches offer a complementary perspective? We explore this question through a year-long participant-observation study of a participatory research and development project focused on object recognition technology in public space in Amsterdam. This project not only engaged citizens in shaping urban technology but also challenged the professional, institutional, and political boundaries of what constitutes participation in the smart city. We identified six key developments that emerged during the course of the project and analysed how the participatory process was implemented across them. The results resonate with critical literature on participatory governance in smart cities, particularly the difficulty of sustaining citizen influence in co-decisive roles over time. We argue that the temporal dynamics of influence and institutional uptake observed in this case point to significant opportunities for collaborative and institutional learning in ladder-driven innovation projects.

Contestability has been proposed as a key element in designing algorithmic decision-making processes that safeguard decision subjects' rights to dignity and autonomy. However, little is known about how contestability can be operationalized based on decision subjects' needs and preferences. We address this research gap by identifying decision subjects' information and procedural needs for enacting meaningful contestability. To this end, we chose an illegal holiday rental detection scenario as our case; a high-risk decision-making process in the public sector. We conducted 21 semi-structured interviews with citizens with experience renting their homes out and different levels of AI literacy. We found that decision subjects request interventions that facilitate (1) cooperation in sense-making, (2) support in contestation acts, and (3) appropriate responsibility attribution. Our results highlight the cooperative work behind contestability, and motivate future efforts to structure individual and collective action, to personalize explanations for contestability, and to open up sites of contestation in AI pipelines.

Public sector organizations increasingly use artificial intelligence to augment, support, and automate decision-making. However, such public AI can potentially infringe on citizens’ right to autonomy. Contestability is a system quality that protects against this by ensuring systems are open and responsive to disputes throughout their life cycle. While a growing body of work is investigating contestable AI by design, little of this knowledge has so far been evaluated with practitioners. To make explicit the guiding ideas underpinning contestable AI research, we construct the generative metaphor of the Agonistic Arena, inspired by the political theory of agonistic pluralism. Combining this metaphor and current contestable AI guidelines, we develop an infographic supporting the early-stage concept design of public AI system contestability mechanisms. We evaluate this infographic in five workshops paired with focus groups with a total of 18 practitioners, yielding ten concept designs. Our findings outline the mechanisms for contestability derived from these concept designs. Building on these findings, we subsequently evaluate the efficacy of the Agonistic Arena as a generative metaphor for the design of public AI and identify two competing metaphors at play in this space: the Black Box and the Sovereign.

The increasing use of artificial intelligence (AI) by public actors has led to a push for more transparency. Previous research has conceptualized AI transparency as knowledge that empowers citizens and experts to make informed choices about the use and governance of AI. Conversely, in this paper, we critically examine if transparency-as-knowledge is an appropriate concept for a public realm where private interests intersect with democratic concerns. We conduct a practice-based design research study in which we prototype and evaluate a transparent smart electric vehicle charge point, and investigate experts’ and citizens’ understanding of AI transparency. We find that citizens experience transparency as burdensome; experts hope transparency ensures acceptance, while citizens are mostly indifferent to AI; and with absent means of control, citizens question transparency’s relevance. The tensions we identify suggest transparency cannot be reduced to a product feature, but should be seen as a mediator of debate between experts and citizens.