A sociotechnical systems lens on AI is often used to bring attention to the human factors and societal impacts that are often neglected through technical abstraction. However, abstraction is also a general principle of sociotechnical systems, where functional objectives (e.g. fair hiring decisions) are operationalised into low-level implementations (e.g. fair algorithms, recourse, legal basis). The trouble with abstraction arises when critical contextual factors are erroneously neglected, leading to an impoverished representation of the problem space. De-contextualisation can render the resulting solutions problematic when they are re-contextualised back into the site of use, where misabstractions may produce safety hazards, harms, moral wrongs, and context frictions. Despite growing recognition that context matters for how sociotechnical systems operate in practice, the normative implications of abstraction are still understudied. In this paper, we propose misabstraction as an analytic framework for thinking about the perils and challenges of sociotechnical abstraction. We use the framework to analyse the requirements specification outlined in the procurement tender of a recommender system for public employment services and show how misabstractions cascade through the sociotechnical stack, producing ripple effects that implicate hidden and neglected contextual factors across multiple frames (e.g. institutional, organisational, operational, and algorithmic). Misabstraction can help policymakers, system designers, critical scholars, and civil society alike to attend to the political conditions that shape design, and their implications for understanding and addressing systemic risk in sociotechnical AI systems.

This paper critically evaluates the attempts to align Artificial Intelligence (AI) systems, especially Large Language Models (LLMs), with human values and intentions through Reinforcement Learning from Feedback methods, involving either human feedback (RLHF) or AI feedback (RLAIF). Specifically, we show the shortcomings of the broadly pursued alignment goals of honesty, harmlessness, and helpfulness. Through a multidisciplinary sociotechnical critique, we examine both the theoretical underpinnings and practical implementations of RLHF techniques, revealing significant limitations in their approach to capturing the complexities of human ethics, and contributing to AI safety. We highlight tensions inherent in the goals of RLHF, as captured in the HHH principle (helpful, harmless and honest). In addition, we discuss ethically-relevant issues that tend to be neglected in discussions about alignment and RLHF, among which the trade-offs between user-friendliness and deception, flexibility and interpretability, and system safety. We offer an alternative vision for AI safety and ethics which positions RLHF approaches within a broader context of comprehensive design across institutions, processes and technological systems, and suggest the establishment of AI safety as a sociotechnical discipline that is open to the normative and political dimensions of artificial intelligence.

This paper reviews the burgeoning field of data-driven algorithms and their application in solving increasingly complex decision-making, optimization, and control problems within active distribution networks. By summarizing a wide array of use cases, including network reconfiguration and restoration, crew dispatch, Volt-Var control, dispatch of distributed energy resources, and optimal power flow, we underscore the versatility and potential of data-driven approaches to improve active distribution system operations. The categorization of these algorithms into four main groups-mathematical optimization, end-to-end learning, learning-assisted optimization, and physics-informed learning-provides a structured overview of the current state of research in this domain. Additionally, we delve into enhanced algorithmic strategies such as non-centralized methods, robust and stochastic methods, and online learning, which represent significant advancements in addressing the unique challenges of active distribution systems. The discussion extends to the critical role of datasets and test systems in fostering an open and collaborative research environment, essential for the validation and benchmarking of novel data-driven solutions. In conclusion, we outline the primary challenges that must be navigated to bridge the gap between theoretical research and practical implementation, alongside the opportunities that lie ahead. These insights aim to pave the way for the development of more resilient, efficient, and adaptive active distribution networks, leveraging the full spectrum of data-driven algorithmic innovations.

In mathematical models of eco-evolutionary dynamics with a quantitative trait, two species with different strategies can coexist only if they are separated by a valley or peak of the adaptive landscape. A community is ecologically and evolutionarily stable if each species’ trait sits on global, equal fitness peaks, forming a saturated ESS community. However, the adaptive landscape may allow communities with fewer (undersaturated) or more (hypersaturated) species than the ESS. Non-ESS communities at ecological equilibrium exhibit invasion windows of strategies that can successfully invade. Hypersaturated communities can arise through mutual invasibility where each non-ESS species’ strategy lies in another’s invasion window. Hypersaturation in ESS communities with more than 1 species remains poorly understood. We use the G-function approach to model niche coevolution and Darwinian dynamics in a Lotka–Volterra competition model. We confirm that up to 2 species can coexist in a hypersaturated community with a single-species ESS if the strategy is scalar-valued, or 3 species if the strategy is bivariate. We conjecture that at most n·s+1 species can form a hypersaturated community, where n is the number of ESS species at the strategy’s dimension s. For a scalar-valued 2-species ESS, 4 species coexist by “straddling” the would-be ESS traits. When our model has a 5-species ESS, we can get 7 or 8, but not 9 or 10, species coexisting in the hypersaturated community. In a bivariate model with a single-species ESS, an infinite number of 3-species hypersaturated communities can exist. We offer conjectures and discuss their relevance to ecosystems that may be non-ESS due to invasive species, climate change, and human-altered landscapes.

As renewable energy and electrification expand rapidly, many electrical distribution grids experience grid congestion. This situation leads to long waiting lists for parties seeking a new grid connection or aiming to expand their existing grid connection. In addition to traditional grid enforcements, distribution system operators are developing ways to manage congestion by steering electricity supply and demand. As grid congestion limits the previously abundant resource of grid capacity, the challenge of how to fairly distribute this now-scarce resource raises new questions about nondiscrimination and broader notions of justice. This study, grounded in energy justice, explores the distributive and procedural injustices people experience with increasing grid congestion. Our research focuses on The Netherlands, where more than 10,000 parties await new grid connections. Through 16 semi-structured interviews with people either affected by or involved in mitigating grid congestion, our thematic analysis reveals three key categories: (1) injustices arising from legacy policies, legislation, and social norms; (2) injustices due to unclear regulations, inconsistent policies, and policy gaps; and (3) injustices related to changing relationships between DSOs and affected parties. These findings highlight that grid congestion is fundamentally sociotechnical; while congestion is both constrained and addressed by technical factors, institutional and social factors such as legacy policies, social norms and communication, significantly influence perceptions of injustice. Our findings call for a comprehensive integration of justice principles within the institutional (e.g. regulation, policy, markets, social norms), technical (e.g. grid infrastructure, IT systems), and social (e.g. community engagement, communication) components of grid infrastructure.

Purpose (stating the main purposes and research question): Anthropogenic resource use contributes to pollution, violent conflict over scarce resources, loss of biodiversity, and diminished quality of life for humans. Moreover, the “safe” amount of carbon dioxide—350 parts per million—has been exceeded. The health care industry is responsible for 4–5% of total world emissions,[i] which is similar to the global food sector.[ii] Health care carbon emissions come from health care infrastructures, supply chains and health care delivery. Increasingly, health care delivery is reliant on technologies which require the use of artificial intelligence to provide supportive care, such as triage algorithms, electronic patient records, and robotics.[iii] While these technological innovations have advanced health care significantly, they also contribute to the negative effects on the environment, among others, through carbon emissions. The environmental impacts of artificial intelligence (AI) in health care—in particular—are understudied. This research seeks to fill this gap. Methods: Our team ran an exploratory search in Scopus and PubMed to identify studies that integrate environmental sustainability, artificial intelligence, and health. Results: Our research initially yielded 735 studies. 77 of these studies focused on an environmental concern of a health technology or AI-application in a health care setting, but most of the articles in this subset addressed lowering energy consumption of a specific technology, such as a sensor or monitoring technology. Conclusions: While there have been studies looking at AI in health care; sustainability in AI; and sustainability in health care, little attention has been paid to the interface between all three. [i] Karliner, J., Slotterback, S., Boyd, R., Ashby, B., & Steele, K. 2019. Health Care’s Climate Footprint: How the Health Sector Contributes to the Global Climate Crisis and Opportunities for Action Healthcare Without HarmARUP; September. [ii] Pichler, P. P., Jaccard, I. S., Weisz, U., & Weisz, H. 2019 International Comparison of Health Care Carbon Footprints, Environmental Research Letters 14, no. 6: 064004. [iii] Khaliq, Abdul, Ali Waqas, Qasim Ali Nisar, Shahbaz Haider, and Zunaina Asghar. 2022. Application of AI and robotics in hospitality sector: A resource gain and resource loss perspective. Technology in Society 68: 101807.

Information technologies promise to make products and services more efficient and sustainable, yet the ICT industry has a substantial ecological footprint that is only expected to grow. This calls for actions beyond improving product efficiency. To this end, we explore whether demand for data can be moderated by encouraging ‘sufficient’ usage. We combine behaviour change theories with a critical design approach and conduct a Research through Design exploration of online consumption speculating about possible directions for reducing data use. Partnering with an internet service provider, we designed and tested with end users a set of three provotypes embodying data-sufficient design directions. We reflect on the results to argue that across the entire system of stakeholders, there is insufficient awareness of the environmental impact of data technologies. Further, we conclude by highlighting that contrary to the belief that interventions aimed at reducing consumption harm user experience, our results show that moderating data intake might actually even improve it.

Fairness has recently gained significant attention in the scientific literature on algorithmic control systems for congestion management. However, many diverse conceptualizations of fairness have been presented. This paper aims to categorize these varying conceptualizations by reviewing existing literature on congestion management. It examines how researchers approach decisions concerning the scoping of fairness problems, the selection of fairness principles, and the choice of evaluation metrics. Findings highlight a need for more justification of fairness conceptualizations in literature as well as a need for standardized evaluation metrics and more empirical grounding and validation. The insights provided can help researchers and practitioners consider fairness comprehensively in the design of algorithmic control systems for congestion management.

This article explores the rapidly developing field of Critical AI Studies and its relation to issues of class and capitalism through a hybrid approach based on distant reading of a newly collected corpus of 300 full-text scientific articles, the creation of which is itself a first attempt at properly delineating the field. We find that words related to issues of class are predominantly but not exclusively confined to a set of studies that make up their own distinct subfield of Critical AI Studies, in contrast to, e.g., issues of race and gender, which are more broadly present in the corpus.

This paper provides an empirical and conceptual account on seeing machine learning models as part of a sociotechnical system to identify relevant vulnerabilities emerging in the context of use. As ML is increasingly adopted in socially sensitive and safety-critical domains, many ML applications end up not delivering on their promises, and contributing to new forms of algorithmic harm. There is still a lack of empirical insights as well as conceptual tools and frameworks to properly understand and design for the impact of ML models in their sociotechnical context. In this paper, we follow a design science research approach to work towards such insights and tools. We center our study in the financial industry, where we first empirically map recently emerging MLOps practices to govern ML applications, and corroborate our insights with recent literature. We then perform an integrative literature research to identify a long list of vulnerabilities that emerge in the sociotechnical context of ML applications, and we theorize these along eight dimensions. We then perform semi-structured interviews in two real-world use cases and across a broad set of relevant actors and organizations, to validate the conceptual dimensions and identify challenges to address sociotechnical vulnerabilities in the design and governance of ML-based systems. The paper proposes a set of guidelines to proactively and integrally address both the dimensions of sociotechnical vulnerability, as well as the challenges identified in the empirical use case research, in the organization of MLOps practices.

This paper introduces systems theory and system safety concepts to ongoing academic debates about the safety of Machine Learning (ML) systems in the public sector. In particular, we analyze the risk factors of ML systems and their respective institutional context, which impact the ability to control such systems. We use interview data to abductively show what risk factors of such systems are present in public professionals' perceptions and what factors are expected based on systems theory but are missing. Based on the hypothesis that ML systems are best addressed with a systems theory lens, we argue that the missing factors deserve greater attention in ongoing efforts to address ML systems safety. These factors include the explication of safety goals and constraints, the inclusion of systemic factors in system design, the development of safety control structures, and the tendency of ML systems to migrate towards higher risk. Our observations support the hypothesis that ML systems can be best regarded through a systems theory lens. Therefore, we conclude that system safety concepts can be useful aids for policymakers who aim to improve ML system safety.

This chapter proposes an analytical lens to comprehensively address the role of Artificial Intelligence (AI) applications in mediating arbitrary exercise of power in public administration and the citizen harms that result from such conduct. It provides a timely and urgent account to fill gaps in conventional Rule of Law thought. AI systems are socio-technical by nature and, therefore, differ from the text-driven social constructs that the legal professions dealing with Rule of Law issues concentrate on. Put to work in public administration contexts with consequential decision-making, technical artefacts can contribute to a variety of hazardous situations that provide opportunities for arbitrary conduct. A comprehensive lens to understand and address the role of technology in Rule of Law violations has largely been missing in literature. We propose to combine a socio-legal perspective on the Rule of Law with central insights from system safety—a safety engineering tradition with a strong scientific as well as real-world practice—that considers safety from a technological, systemic, and institutional perspective. The combination results in a lexicon and analytical approach that enables public organisations to identify possibilities for arbitrary conduct in public AI systems. Following on the analysis, interventions can be designed to prevent, mitigate, or correct system hazards and, thereby, protect citizens against arbitrary exercise of power.

In this article, the following errors have been missed in the corrections stage and the same has been corrected with the correction article. Data Availability section was inadvertently published and it has been removed from the article. The reference Wolters, A. (2022). Guiding the specification of sociotechnical Machine Learning systems: Addressing vulnerabilities and challenges in Machine Learning practice (Unpublished master’s thesis). Massachusetts Institute of Technology is updated to Wolters, A. (2022). Guiding the specification of sociotechnical Machine Learning systems: Addressing vulnerabilities and challenges in Machine Learning practice (Published master’s thesis). Delft University of Technology. Missing equal contribution text and Acknowledgments were included in the original article.

Bayesian networks (BNs) are compact representations of probability distributions that allow for supporting reasoning and decision making under uncertainty. Their interpretable structure and probability parameters allow for integrating human knowledge in their construction and explanation. For BN construction, reusable building blocks, or idioms, exist that describe the dependencies and reasoning patterns among small sets of variables. In this paper we formalise the concept of an idiom, explicitly including qualitative constraints that capture the reasoning patterns among variables as stated in the informal descriptions that accompany the idioms in literature. Our proposed formalisation ensures that idioms can be applied more consistently and reliably, improving the BN’s accountability.

Algorithmic and data-driven systems are increasingly used in the public sector to improve the efficiency of existing services or to provide new services through the newfound capacity to process vast volumes of data. Unfortunately, certain instances also have negative consequences for citizens, in the form of discriminatory outcomes, arbitrary decisions, lack of recourse, and more. These have serious impacts on citizens ranging from material to psychological harms. These harms partly emerge from choices and interactions in the design process. Existing critical and reflective frameworks for technology design do not address several aspects that are important to the design of systems in the public sector, namely protection of citizens in the face of potential algorithmic harms, the design of institutions to ensure system safety, and an understanding of how power relations affect the design, development, and deployment of these systems. The goal of this workshop is to develop these three perspectives and take the next step towards reflective design processes within public organisations. The workshop will be divided into two parts. In the first half we will elaborate the conceptual foundations of these perspectives in a series of short talks. Workshop participants will learn new ways of protecting against algorithmic harms in sociotechnical systems through understanding what institutions can support system safety, and how power relations influence the design process. In the second half, participants will get a chance to apply these lenses by analysing a real world case, and reflect on the challenges in applying conceptual frameworks to practice.

As AI technology becomes more powerful, the impact on governance and the governance of AI becomes more important. The purpose of this panel is to bring together experts on Trustworthy AI and E-Governance to discuss the lessons learned so far and the challenges ahead. The goal is to provide a comprehensive review of challenges and potential solutions related to the future acceptance and beneficial application of AI-based solutions - including using AI for public benefit and applying AI for the common good.

Values, such as freedom and safety, are the core motivations that guide us humans. A prerequisite for creating value-aligned multiagent systems that involve humans and artificial agents is value inference, the process of identifying values and reasoning about human value preferences. We introduce a framework that connects the value inference steps, and motivate why a hybrid intelligence approach is instrumental for its success. We also highlight the multidisciplinary research challenges that hybrid value inference entails.

As artificial agents become increasingly embedded in our society, we must ensure that their behavior aligns with human values. Value alignment entails value inference, the process of identifying values and reasoning about how humans prioritize values. We introduce a holistic framework that connects the technical (AI) components necessary for value inference. Subsequently, we discuss how hybrid intelligence'the synergy of human and artificial intelligence'is instrumental to the success of value inference. Finally, we illustrate how value inference both poses significant challenges and provides novel opportunities for multiagent systems research.

Algorithmic systems used in public administration can create or reinforce digital cages. A digital cage refers to algorithmic systems or information architectures that create their own reality through formalization, frequently resulting in incorrect automated decisions with severe impact on citizens. Although much research has identified how algorithmic artefacts can contribute to digital cages and their unintended consequences, the emergence of digital cages from human actions and institutions is poorly understood. Embracing a broader lens on how technology, human activity, and institutions shape each other, this paper explores what design practices in public organizations can result in the emergence of digital cages. Using Orlikowski’s structurational model of technology, we found four design practices in observations and interviews conducted at a consortium of public organizations. This study shows that design processes of public algorithmic systems (1) are often narrowly focused on technical artefacts, (2) disregard the normative basis for these systems, (3) depend on involved actors’ awareness of socio-technics in public algorithmic systems, (4) and are approached as linear rather than iterative. These four practices indicate that institutions and human actions in design processes can contribute to the emergence of digital cages, but also that institutional – opposed to technical – possibilities to address their unintended consequences are often ignored. Further research is needed to examine how design processes in public organizations can evolve into socio-technical processes, can become more democratic, and how power asymmetries in the design process can be mitigated.