Agonistic Machine Vision Development

Abstract

As more smart city technologies are being developed, more attention should be directed on how to include citizens to align such technology with public values and facilitate contestability. This involvement can take place during different phases. To ensure the legitimacy of a machine vision system, citizens should be involved to discuss when a model is performing acceptably. However, enabling this civic participation is a challenge due to a lack of awareness, knowledge, and understanding.

This graduation project explores opening up the discussion around acceptability of a machine vision system, using the scan car development process in Amsterdam as a use case. This discussion clarified acceptability based on trade-offs made during the development phase. To open up this discussion, citizens first need to be able to understand before they can judge such a system and its trade-offs. This resulted in creating a tangible approach to machine vision development, merging elements from fields as TUI and XAI.

The final design was evaluated within three sessions. The results suggest that providing a tangible representation and context-specific examples improved the subjective understanding of participants. The design enabled participants to form and articulate their own opinion about what is acceptable and take part in this discussion. The findings suggest that a tangible approach to participatory machine learning could involve
citizens and other non-expert stakeholders in informing and steering a machine vision model during the development phase to close the legitimacy gap.

Facilitating this communication, the project shows a possible contestability loop between citizens and developers in the co-construction of public AI. Ultimately, the design contributes to the field of participatory design approaches to public and responsible AI by providing a practical example.