This research explores how AI-powered “digital representatives” might address Rotterdam’s democratic deficit, where citizen input loses richness as it moves through municipal processes. In Rotterdam, this disconnect manifests in participation disparities (voter turnout ranging from 60% in affluent areas to 21% elsewhere) and declining institutional trust. Through iterative development and evaluation, the study created AI personas capable of maintaining citizen perspectives in policy discussions. These digital representatives integrated speech recognition, Large Language Models, and voice synthesis to participate in deliberations about municipal issues. Key findings demonstrate that while LLMs can consistently represent citizen viewpoints, their authenticity depends entirely on input quality. Representatives derived from rich interview transcripts preserved nuance and influenced policy discussions, while those created from typical municipal documentation proved inadequate. Voice interaction generated stronger engagement than text interfaces, and individual exploration enhanced subsequent group discussions. Evaluation with Rotterdam civil servants revealed both promising capabilities and significant concerns. Digital representatives successfully challenged assumptions about community needs, but participants expressed legitimate concerns about accuracy and the risk of replacing direct citizen engagement with technological simulation. The research’s primary contribution is not the technology itself but revealing critical deficiencies in how municipalities capture and process citizen knowledge. Current documentation practices often fail to retain the experiential context that gives citizen input its value. Addressing Rotterdam’s democratic challenges requires transforming how institutions value citizen perspectives—preserving context rather than pursuing abstraction. Digital representatives cannot solve Rotterdam’s democratic challenges alone but offer a means to sustain citizen perspectives throughout policy processes, especially for voices easily lost in current systems. Their most potent function may be provoking necessary institutional conversations about how citizen knowledge is valued—helping fill, not just simulate presence in, the empty chair.

The adoption of generative AI (GenAI) by public organisations in digital citizen services presents both opportunities and challenges. While GenAI solutions such as Microsoft Copilot have the potential to improve the efficiency and quality of service delivery, public organisations are hesitant to explore its value. Consultants within Capgemini’s Microsoft cluster play a key role in optimising digital service systems but face difficulties in advising public sector clients on GenAI. Through a strategic design approach, this graduation project focused on developing a solution that enhances consultant support in early stage GenAI discussions to ensure that public sector clients can explore GenAI in a way that is strategic, informed, and aligned with their needs.

In the research phase, GenAI adoption barriers of public organisations and difficulties consultants face in guiding clients on GenAI were examined. The literature research, interviews with consultants, and insights of experts revealed that trust, regulatory concerns, and uncertainty about the value of GenAI are key barriers preventing initial adoption. Public sector clients often operate in a regulated and risk averse environment, making them reluctant to explore this new technology. While consultants recognise GenAI’s potential in projects, they struggle to make discussions about GenAI concrete, address client concerns, and align it with organisational needs.

The insights from the research were translated into design opportunities in the definition and ideation phase. Through brainstorming and evaluation sessions, several ideas were explored which would support the consultants as trusted advisors and build client trust in generative AI through an adaptable, interactive and collaborative way. After evaluating five concepts, strong elements were combined into a new concept.

The concept development phase led to a final design: the GenAI Exploration Kit, a toolkit to empower consultants in supporting GenAI exploration with clients. Consultants can use the toolkit to guide clients through an exploratory journey of five interactive sessions which help clients understand GenAI’s potential, identify relevant use cases and address possible risks. The toolkit includes support materials to facilitate exploration sessions and session materials consisting of collaborative canvases and stimulus cards to guide discussions and spark co-creation.

The final design was evaluated with Microsoft consultants to assess its relevance, usability and scalability within Capgemini’s consulting practice. The evaluation validated that the toolkit provides the necessary structure for consultants in enabling them to guide discussions on GenAI more effectively. The consultants valued the interactive format and strategic step by step approach, helping them make GenAI more concrete and connected to the current challenges of the client.

To ensure successful implementation, a strategic implementation plan was developed which outlines a phased approach to adopt it within Capgemini on different levels and sustain its relevance. Additionally, recommendations for further research and development were made including pilot testing the toolkit in real client engagements. With continued development, the GenAI Exploration Kit has the potential to become an essential resource for consultants to guide public organisations in valuable exploration of AI.

This paper argues that protecting human autonomy in AI systems requires moving beyond application-level contestability to address three critical dimensions: (1) shifting focus from applications to infrastructures that shape technological possibilities; (2) designing for collectives rather than individuals to foster democratic governance; and (3) adopting realist rather than idealist approaches to address actual power relations. The author proposes a research agenda called "People's Compute" that aims to democratize AI infrastructure through constructive HCI design research, positioning designers as embedded accomplices who build capacity within communities for technological self-determination. This approach offers an alternative path between neoliberal technocracy and populist anti-politics in our current era of increasing technological sovereigntism.

The emergence of generative things as a new class of physical objects that embed AI confronts designers with questions about how to relate to the dominance of Big Tech companies in the GenAI space and the resulting constraints on the design of embodied AI.

AI can be understood historically is a subfield of computer and cognitive science. It can also be characterized as a specific set of computational techniques that extract statistical correlations from large datasets, currently dominated by machine learning and neural network approaches. Today, for the most part, these techniques are applied to natural language processing, analysis and generation of ‘content’ (e.g., text, images, datasets, and programming code), and automated decision/recommendation systems. AI also is a “floating signifier” with strategic vagueness that escapes precise definition while suggesting technological autonomy, serving the interests of its promoters while obscuring the material practices, labor and political economies that make it up. This account is important for our purposes because by treating AI as an “uncontroversial thing” with autonomous agency, rather than a situated set of practices and relations, we contribute to its mystification and shield it from critical examination (Suchman, 2023). [...]

In an era dominated by technological solutionism, artificial intelligence (AI) is often seen as a quick fix for complex problems. Inspired by the surrealist notion of pataphysics, this graduation project employs a novel, critical design approach to address the imaginary problem of trustworthy AI and its solutions in the context of generative chatbots.

This project presents three pataphysical chatbots, each addressing a specific requirement—transparency, human control, or fairness—currently implemented in AI systems through technical mechanisms intended to ensure trust. Each chatbot is designed to deliberately exaggerate the mechanism behind the assumed requirement, aiming to help users understand the absurdist nature of the chatbot and, ultimately, provoke critical reflection on the pursuit of trustworthy AI.

Through an iterative design process of prototyping, testing, and reflecting upon pataphysical chatbots, progress was made toward the final pata-design, which was evaluated in an online experiment. A key insight emerged: while the pataphysical chatbots succeeded in conveying their absurdist nature to users, critical reflection on the pursuit of trustworthy AI did not occur. However, additional findings suggest a potential link between understanding the absurdity and the occurrence of this particular type of critical reflection.

Ultimately, this project contributes to the critical design practice and in particular the field of pataphysical design, by demonstrating how pataphysical chatbots can effectively challenge both the prevailing methods for achieving trustworthy AI as the dominant problem-solution paradigm of AI solutionism. Moreover, the pataphysical design approach proposed in this project offers a new direction for current pataphysical design practices, emphasizing the creation of engaging pataphysical experiences that aim to provoke specific user responses to the premises of the imaginary problem.

This graduation project explored the environmental nuisances experienced by houseboat residents in Amsterdam due to vessel traffic. The findings show that the experience of nuisance is influenced by both contextual factors (such as past experiences, type of boat, time, and location) and environmental factors (such as noise and movement).
By reflecting on insights gathered through qualitative research, this project aligns with its initial goal of exploring new methods for measuring environmental nuisance.

Public sector organizations increasingly use artificial intelligence to augment, support, and automate decision-making. However, such public AI can potentially infringe on citizens’ right to autonomy. Contestability is a system quality that protects against this by ensuring systems are open and responsive to disputes throughout their life cycle. While a growing body of work is investigating contestable AI by design, little of this knowledge has so far been evaluated with practitioners. To make explicit the guiding ideas underpinning contestable AI research, we construct the generative metaphor of the Agonistic Arena, inspired by the political theory of agonistic pluralism. Combining this metaphor and current contestable AI guidelines, we develop an infographic supporting the early-stage concept design of public AI system contestability mechanisms. We evaluate this infographic in five workshops paired with focus groups with a total of 18 practitioners, yielding ten concept designs. Our findings outline the mechanisms for contestability derived from these concept designs. Building on these findings, we subsequently evaluate the efficacy of the Agonistic Arena as a generative metaphor for the design of public AI and identify two competing metaphors at play in this space: the Black Box and the Sovereign.

This thesis investigates the use of artificial intelligence (AI) in public policy execution. To contribute to preserving citizen autonomy, it introduces the concept of ‘contestability’—a system quality that ensures citizens retain control over their lives in the face of AI systems and can influence AI system development. The central research aim is to explore sociotechnical design interventions that enhance the contestability of public AI systems.

Utilizing constructive design research, the thesis reports on several studies in which researchers collaborate with design practitioners to create artifacts that function as data generation instruments. Methods encompass interaction design, speculative design, and information design, with case studies in smart electric vehicle charging, urban monitoring camera cars, and fraud risk models, all situated in Amsterdam.

Key findings include varying perceptions of AI transparency between citizens and experts, a design framework for contestable AI, challenges in local government implementation, and the metaphors designers use for public AI.

The research advocates for integrating citizen feedback into AI systems, promoting dialogue between citizens and system controllers, and enhancing democratic involvement in AI development. It also highlights the importance of design in AI implementation, emphasizing speculative design as a method for generating relevant data and guiding ideation and specification processes.

Concluding, the thesis calls for a greater engagement of design researchers and practitioners with political philosophy to understand the democratic implications of their work in AI and related fields.

As more smart city technologies are being developed, more attention should be directed on how to include citizens to align such technology with public values and facilitate contestability. This involvement can take place during different phases. To ensure the legitimacy of a machine vision system, citizens should be involved to discuss when a model is performing acceptably. However, enabling this civic participation is a challenge due to a lack of awareness, knowledge, and understanding.

This graduation project explores opening up the discussion around acceptability of a machine vision system, using the scan car development process in Amsterdam as a use case. This discussion clarified acceptability based on trade-offs made during the development phase. To open up this discussion, citizens first need to be able to understand before they can judge such a system and its trade-offs. This resulted in creating a tangible approach to machine vision development, merging elements from fields as TUI and XAI.

The final design was evaluated within three sessions. The results suggest that providing a tangible representation and context-specific examples improved the subjective understanding of participants. The design enabled participants to form and articulate their own opinion about what is acceptable and take part in this discussion. The findings suggest that a tangible approach to participatory machine learning could involve
citizens and other non-expert stakeholders in informing and steering a machine vision model during the development phase to close the legitimacy gap.

Facilitating this communication, the project shows a possible contestability loop between citizens and developers in the co-construction of public AI. Ultimately, the design contributes to the field of participatory design approaches to public and responsible AI by providing a practical example.

Local governments increasingly use artificial intelligence (AI) for automated decision-making. Contestability, making systems responsive to dispute, is a way to ensure they respect human rights to autonomy and dignity. We investigate the design of public urban AI systems for contestability through the example of camera cars: human-driven vehicles equipped with image sensors. Applying a provisional framework for contestable AI, we use speculative design to create a concept video of a contestable camera car. Using this concept video, we then conduct semi-structured interviews with 17 civil servants who work with AI employed by a large northwestern European city. The resulting data is analyzed using reflexive thematic analysis to identify the main challenges facing the implementation of contestability in public AI. We describe how civic participation faces issues of representation, public AI systems should integrate with existing democratic practices, and cities must expand capacities for responsible AI development and operation.

Roads are currently governed by relatively static rules communicated via road signs, road surface markings and navigation apps. However they could also be dynamic, allowing road space to be allocated for a multitude of activities over time. The Code the Streets project envisions a digital system for managing urban mobility in Amsterdam that would allow for this dynamic allocation of road space. Road rules are intended to be used to foster specific civic values such as livability or sustainability within the city. But how do we ensure fairness within such a system? This project explores how we can reimagine the smart allocation of road space in Amsterdam for fairness. Fairness is framed within this project as an ideal that cannot be reached but can be approximated through continuous adaptation. Inspired by literature on contestable AI, the continuous adaptation of the system is approached through stakeholder participation. The project relied on research in the form of literature review, expert interviews, and hands-on experimentation as well as the review of related design work. This research inspired design goals and criteria for the project’s design outcome: a system map and a speculative prototype. The map presents a system in which stakeholders have substantial influence over the values that the system and its design process foster. The speculative prototype, a low-tech user experience installation, makes parts of this system tangible and presents a provocative proposal for an alternative perspective on the current practice of road space allocation. The project ultimately produced several insights contributing to existing literature in the fields of public AI and design methodology research as well as delivering recommendations for Code the Streets.

The increasing use of artificial intelligence (AI) by public actors has led to a push for more transparency. Previous research has conceptualized AI transparency as knowledge that empowers citizens and experts to make informed choices about the use and governance of AI. Conversely, in this paper, we critically examine if transparency-as-knowledge is an appropriate concept for a public realm where private interests intersect with democratic concerns. We conduct a practice-based design research study in which we prototype and evaluate a transparent smart electric vehicle charge point, and investigate experts’ and citizens’ understanding of AI transparency. We find that citizens experience transparency as burdensome; experts hope transparency ensures acceptance, while citizens are mostly indifferent to AI; and with absent means of control, citizens question transparency’s relevance. The tensions we identify suggest transparency cannot be reduced to a product feature, but should be seen as a mediator of debate between experts and citizens.

As the use of AI systems continues to increase, so do concerns over their lack of fairness, legitimacy and accountability. Such harmful automated decision-making can be guarded against by ensuring AI systems are contestable by design: responsive to human intervention throughout the system lifecycle. Contestable AI by design is a small but growing field of research. However, most available knowledge requires a significant amount of translation to be applicable in practice. A proven way of conveying intermediate-level, generative design knowledge is in the form of frameworks. In this article we use qualitative-interpretative methods and visual mapping techniques to extract from the literature sociotechnical features and practices that contribute to contestable AI, and synthesize these into a design framework

In 2008 more than 50 percent of all people, 3.3 billion people, lived in urban areas according to the United Nations Population Fund (UNFPA - United Nations Population Fund, n.d.). By 2030 this will be approximately 5 billion (Lea, 2017). On the one hand, with fast urban growth also come challenges in the way we build and manage cities. On the other hand, technologies that are discovered in this era of digitization can produce opportunities for the economic and social development of cities. Not only making sure that cities stay safe and livable, but also to improve them, these technologies are indispensable. Worldwide engineers turn to technology to tackle challenges of fast urban growth (Arroub et al., 2016). Aside from tackling these challenges, technologies provide opportunities for economic and social development of cities keeping the cities safe and livable. Both of these aspects are often gathered under the term: ‘Smart City’. Though there are many definitions, visions or imaginaries of what a smart city is or should be, there are key elements that can be attributed to ‘smart cities’. One of the key elements of a smart city is collecting data through sensors embedded in our surroundings to better understand, monitor, regulate and plan the city, with the goal to stimulate and support innovation and economic growth and provide sustainable and efficient urban management and development (Kitchin, 2014). However, data raises concerns regarding its design, development and deployment for smart cities (Al Nuaimi et al., 2015) as well as other concerns relating to ethical issues (Kitchin, 2014). Privacy certainly proves to be the most prominent issue in digitization and big data (Kool et al., 2017). Several international organizations have even identified privacy as a key policy, regulatory and legislation challenge of the 21st century (van Zoonen, 2016). This leads to the following design goal:
“Designing a tour that enables citizens of Utrecht to critically reflect on the collection and use of their personal data and its privacy consequences within the smart city”. Pusca is an application based Personal Utrecht Smart City Assistant that takes the user on a one time tour through Utrecht to explore and experience smart city technologies that could improve their quality of life. Each of the stops presents a service provided by a smart city technology that can be experienced by sharing personal data. The user is asked to make a trade-off between their estimated privacy value of the asked personal data and the benefits of the presented smart technology. At the end of the tour the user gets confronted with privacy consequences of sharing their personal data that provokes a critical reflection on the trade-offs that the user made during the tour. This design is a start for knowing how we want our privacy to be protected, by letting people create an opinion about this. I hope that this project can be a first step into creating a society where open and standard policies about safeguarding citizens’ privacy within smart city technologies will be secured.

The Smart Bench is a speculative design aiming to create critical awareness about the social implications of Data-driven Nudging in the Smart City. The project originated from a critical perspective on Smart City as a leading urban development paradigm. Many cities labelling themselves as smart cities competitively implemented advanced technologies like artificial intelligence (AI), pervasive sensor networks, or internet of things (IoT) for infrastructural or economic purposes (Kitchin et al, 2015; Townsend, 2013). This enabled ubiquitous extraction and monitoring of urban behavioral data, facilitating more efficient, resilient, and frictionless urban governance.Data-driven nudging emerged as vast aggregate of citizen behavioral insights became available to be implemented into the concept of ‘nudge’, a libertarian paternalistic strategy of behavior change (Ranchordás, 2019; Thaler & Sunstein, 2008). Citizen behaviors can be steered more elaborately through targeted and dynamic adjustment of choice architecture enabled by development of analytical algorithms. Pervasively embedded network of urban sensors has made this new type of nudge more invisible, thus more powerful. Hence, data-driven nudging is raising ethical concerns in terms of behavioral manipulation, ubiquitous surveillance, and urban depoliticisation (Kitchin et al, 2015; Lanzing, 2018; Ranchordás, 2019; Void et al, 2018; Yeung, 2016). Making matters worse, citizens’s level of awareness not only on data-driven nudging itself but also on smart city in general turned out to be significantly lacking (Government Europa, 2018; Jameson et al, 2019). The project addresses the lack of awareness of the presence of data-driven nudging and its social implications by designing a speculative experience of Smart Bench. The Smart Bench is an intelligent bench which is designed to nudge citizen behaviors according to a predetermined algorithm. Because smart technologies tend to reproduce pre-existing prevailing biases, the purpose of the design is to make them experiential to the general public. Participants are invited to try out the bench and a special interface where they could take a look at the software principles behind it and have a unique opportunity to adjust the nudging algorithm for others. As they proceed along the experience, participants can experience the discriminatory implications of data-driven nudging and openly discuss its application for urban governance. The evaluation results confirmed a fruitful capacity of speculative design in terms of materialising an abstract unfamiliar concept and evoking critical awareness on its potential implications. Most of all, the design provided participants with a space for open discussions about the necessity of data-driven nudging for urban governance and further reflect on pre-existing discrimination in our current society.

The increasing use of algorithmic decision-making systems in the public realm and in citieshas led to an urgent call for more transparencyand accountability. While recent work in algorith-mic fairness and human-centred ML has exploredways to include the concerns of people into thedesign of ML systems, the “street-level” expe-rience of algorithmic systems is not well under-stood. In this paper, we present a case study of a“transparent electric vehicle charge point” whichis designed to provide electric vehicle drivers withinsights of the operation of smart charging algo-rithms. Exploring limitations of the transparencyideal, we identify the need for contestability as acritical aspect of future public decision-makingsystems.

The increasing use of algorithms in cities has come under scrutiny. Transparency is widely seen as a way to ensure their fairness and accountability. We investigate how al- gorithmic transparency helps citizens understand smart electric vehicle charge points and how its conception differs between experts and citizens. Using a research-through- design approach we collaborated over a 10-month period with companies and Amsterdam municipality to prototype and evaluate a transparent smart electric vehicle charge point. We find that experts believe transparency is pro- duced by truthful information about inputs, processes and outcomes, that this information aids understanding and is actionable. We also find that citizens are indifferent to al-gorithmic decision-making when it serves common interests. Furthermore, transparency invites gaming, creates expectations of control, and adds to the cognitive burden of an already stressful task. Our findings suggest algorithmic transparency benefits professional stakeholders more than the citizens it is claimed to serve.

For data-intensive research, Wi-Fi access data has become an important source since it’s low-cost, convenient and rich in information. It is mainly used to study people’s movement and then used to infer individuals’ or a group of people’s behavior pattern. The application of the research result is also common. It could be used to do interior positioning, crowd management, facility management and more. As the Global Data Privacy Regulation (GDPR) became enforceable in 2018, it set strict rules for personal data collection and usage. Since Wi-Fi access data contains personal information and it’s hard to be anonymised for research purposes, GDPR brings more limitations for people like researchers when they deal with Wi-Fi access data. The GDPR triggers the development of a data platform in TU Delft, which aims at sharing data to researchers in a legal way. This data platform will legitimate the management of such personal data, helping the researchers to get and process the data in a GDPR-compliant way. Three stakeholders are involved in this context: the ICT Department as the data controller, researchers and designers as the data requester, and the students and employees in the TU campus as the data subject. All their values need to be taken seriously when developing the platform. Thus, the aim of the research is to bridge the gap between the stakeholder’s values and the implementations of the platform developers. Literature review is conducted first, discussing why ethics are important in platform design. Then the background information about Wi-Fi access data and GDPR are researched, so as an overview of the data platform. Then the research uses Friedman’s Value Sensitive Design methods to explore the values of different stakeholder entities. Methods like interview, questionnaire and generative session are used to collect the stakeholders’ insights. Then the insights are analysed, being defined as different values according to Schwartz’s theory of Basic Human Values. Then, a toolkit is designed through two minimum viable product (MVP) iterations. It communicates the values to the platform developers, then inspire them to come up with functionalities that fulfill the values. The form of the toolkit are two sets of cards. The first card set are ‘values cards’. On the frontside of each card, a defined value, a sub-value and one quote from the stakeholder that help to understand the value are shown. On the backside of each card, questions that help the developers to measure whether this value is met are asked. The second card set are ‘inspiration cards’. It shows one or more functionality that could be developed to fulfill each value, triggering the developers coming up with more tangible features. It is expected that the toolkit will be used in the group discussions when deciding what functionalities will be implemented into the platform, to speak for the absent stakeholders as well as to trigger discussion. Finally, the limitations of the research are discussed, the whole project is concluded and personal reflection is made.