R.I.J. Dobbe
Please Note
34 records found
1
Understanding AI Trustworthiness
A Scoping Review of AIES & FAccT Articles
The power of assumptions
A literature review on how algorithmic design influences energy justice in electrical distribution grids
Recent energy justice scholarship has argued for the need to reflect more explicitly on the normative assumptions that underpin claims to justice in energy systems. While such reflections increasingly inform energy policy, less attention has been paid to how these assumptions shape the design of algorithmic systems central to energy system planning and operations. This paper explores how normative assumptions in the design of algorithmic systems used to request flexibility from electricity consumers and producers to manage grid congestion may influence distributive justice outcomes. By systematically reviewing the scientific literature presenting such systems, we define two categories of assumptions: (1) scope assumptions , which set the boundaries of the justice analysis by determining which burdens and benefits, scale, subjects, and timeframe are considered relevant; and (2) design assumptions , which specify how these considerations are translated into the structure of algorithmic systems, such as allocation principles, technical problem framing, data availability and evaluation metrics. We find that the particular assumptions adopted within each category determine the distributive outcomes of these algorithmic systems. Recognizing their normative character, we propose that scope assumptions should be informed by context-specific risks of injustice identified by policymakers, while engineers should reflect on and validate their design assumptions in relation to these risks.
Evolutionary therapy (ET) applies principles of evolutionary biology to steer tumour dynamics and forestall or delay treatment resistance, typically guided by data-driven mathematical models. Our aim is to assess whether ET protocols, and specifically Zhang et al.’s protocol proposed for metastatic castrate-resistant prostate cancer, can be theoretically effective for fast-growing metastatic cancers such as stage IV non-small-cell lung cancer (NSCLC). Using longitudinal tumour-burden data from NSCLC patients treated with erlotinib, we systematically evaluate 26 two-population differential-equation models based on classical tumour-growth dynamics, with varying assumptions about density- and frequency-dependent interactions, pharmacokinetics, and treatment-induced death. Previous work by Yin et al. on the same dataset employed an exponential model that omitted density- and frequency-dependent interactions; although it provided a good fit to tumour-burden data, its structure would theoretically lead to poorer outcomes under ET protocols. In contrast, our analysis identifies the minimal model structure required to reproduce the resistance-driven regrowth observed in NSCLC, with the Gompertzian model featuring log-kill dynamics and both density- and frequency-dependent interactions providing the best fit. In this model, Zhang et al.’s protocol prolonged median time-to-progression to 42.3 months compared with 24.8 months under maximum tolerated dose. These results indicate that ET is theoretically a viable treatment strategy for NSCLC. This study offers a practical framework for assessing ET feasibility using clinical data and supports future clinical translation of ET in NSCLC.
Group fairness in power systems computation
Evaluating disparities in grid capacity between households with and without rooftop solar
“Fairness-aware” algorithms are increasingly used to allocate grid capacity in electrical power systems, yet they often overlook existing disparities between household groups, such as those arising from unequal rooftop solar adoption. Through simulations on a stylized low-voltage grid using electricity data from the Netherlands, we apply demographic parity, a group fairness metric, to compare differences in grid access and costs between households with and without rooftop solar. Three objectives are analyzed: minimal intervention, proportional fairness, and min-max fairness. The results indicate that (1) “fairness-aware” allocation does not eliminate group disparities, (2) voltage violations influence the distribution of burdens and benefits, (3) lower curtailment compensation can reduce disparities between households, and (4) common fairness evaluation metrics do not capture these group disparities. This study advances the discussion on fairness in power systems computation by encouraging further research on how to avoid reinforcing existing disparities.
Understanding the Affordances and Constraints of Explainable AI in Safety-Critical Contexts
A Case Study in Dutch Social Welfare
We focus on explainability as a desideratum for automated decision-making systems, rather than only models. Although the explainable artificial intelligence (XAI) paradigm offers an impressive variety of solutions to increase the transparency of automated decisions, XAI contributions rarely account for the complete systems—social and institutional environments—where models operate. Our work focuses on one such system in the domain of social welfare, which increasingly turns to automated decision-making to carry out targeted digital surveillance. Specifically, we present a case study of a black-box machine learning model previously used in a major Dutch city to support its officials in the task of detecting fraud. Employing analyses established in the field of system safety, we identify five types of hazards that could have occurred after the introduction of the model. For each of them, we reason about the potential value of XAI interventions as hazard mitigation strategies. The case study illustrates how the deployment of models may impact processes that exist far upstream and downstream from their decision logic, making explainability and/or interpretability insufficient to guarantee the systems’ safe operation. In many cases, XAI techniques may only be able to reasonably address a small fraction of hazards related to the use of algorithms; several major hazards that we identify would have still posed risks if the system had relied on an interpretable model. Thus, we empirically demonstrate that the values, which lie at the heart of XAI research, such as responsibility, safety, or transparency, ultimately necessitate a broader outlook on automated decision-making systems.
Helpful, harmless, honest?
Sociotechnical limits of AI alignment and safety through Reinforcement Learning from Human Feedback
Systemic Risks of Algorithms in Social Welfare
Combining Actor Analysis and System Safety Analysis
Adapting to limited grid capacity
Perceptions of injustice emerging from grid congestion in the Netherlands
As renewable energy and electrification expand rapidly, many electrical distribution grids experience grid congestion. This situation leads to long waiting lists for parties seeking a new grid connection or aiming to expand their existing grid connection. In addition to traditional grid enforcements, distribution system operators are developing ways to manage congestion by steering electricity supply and demand. As grid congestion limits the previously abundant resource of grid capacity, the challenge of how to fairly distribute this now-scarce resource raises new questions about nondiscrimination and broader notions of justice. This study, grounded in energy justice, explores the distributive and procedural injustices people experience with increasing grid congestion. Our research focuses on The Netherlands, where more than 10,000 parties await new grid connections. Through 16 semi-structured interviews with people either affected by or involved in mitigating grid congestion, our thematic analysis reveals three key categories: (1) injustices arising from legacy policies, legislation, and social norms; (2) injustices due to unclear regulations, inconsistent policies, and policy gaps; and (3) injustices related to changing relationships between DSOs and affected parties. These findings highlight that grid congestion is fundamentally sociotechnical; while congestion is both constrained and addressed by technical factors, institutional and social factors such as legacy policies, social norms and communication, significantly influence perceptions of injustice. Our findings call for a comprehensive integration of justice principles within the institutional (e.g. regulation, policy, markets, social norms), technical (e.g. grid infrastructure, IT systems), and social (e.g. community engagement, communication) components of grid infrastructure.
This paper reviews the burgeoning field of data-driven algorithms and their application in solving increasingly complex decision-making, optimization, and control problems within active distribution networks. By summarizing a wide array of use cases, including network reconfiguration and restoration, crew dispatch, Volt-Var control, dispatch of distributed energy resources, and optimal power flow, we underscore the versatility and potential of data-driven approaches to improve active distribution system operations. The categorization of these algorithms into four main groups-mathematical optimization, end-to-end learning, learning-assisted optimization, and physics-informed learning-provides a structured overview of the current state of research in this domain. Additionally, we delve into enhanced algorithmic strategies such as non-centralized methods, robust and stochastic methods, and online learning, which represent significant advancements in addressing the unique challenges of active distribution systems. The discussion extends to the critical role of datasets and test systems in fostering an open and collaborative research environment, essential for the validation and benchmarking of novel data-driven solutions. In conclusion, we outline the primary challenges that must be navigated to bridge the gap between theoretical research and practical implementation, alongside the opportunities that lie ahead. These insights aim to pave the way for the development of more resilient, efficient, and adaptive active distribution networks, leveraging the full spectrum of data-driven algorithmic innovations.
Preface
EGOV-CeDEM-ePart 2025 conference
Burying the Lead
Adjusting Goals to Manage Functional Limitations of AI Tools in Healthcare
In mathematical models of eco-evolutionary dynamics with a quantitative trait, two species with different strategies can coexist only if they are separated by a valley or peak of the adaptive landscape. A community is ecologically and evolutionarily stable if each species’ trait sits on global, equal fitness peaks, forming a saturated ESS community. However, the adaptive landscape may allow communities with fewer (undersaturated) or more (hypersaturated) species than the ESS. Non-ESS communities at ecological equilibrium exhibit invasion windows of strategies that can successfully invade. Hypersaturated communities can arise through mutual invasibility where each non-ESS species’ strategy lies in another’s invasion window. Hypersaturation in ESS communities with more than 1 species remains poorly understood. We use the G-function approach to model niche coevolution and Darwinian dynamics in a Lotka–Volterra competition model. We confirm that up to 2 species can coexist in a hypersaturated community with a single-species ESS if the strategy is scalar-valued, or 3 species if the strategy is bivariate. We conjecture that at most n·s+1 species can form a hypersaturated community, where n is the number of ESS species at the strategy’s dimension s. For a scalar-valued 2-species ESS, 4 species coexist by “straddling” the would-be ESS traits. When our model has a 5-species ESS, we can get 7 or 8, but not 9 or 10, species coexisting in the hypersaturated community. In a bivariate model with a single-species ESS, an infinite number of 3-species hypersaturated communities can exist. We offer conjectures and discuss their relevance to ecosystems that may be non-ESS due to invasive species, climate change, and human-altered landscapes.
Environmental impacts of artificial intelligence in health care
Considerations and recommendations
Purpose (stating the main purposes and research question): Anthropogenic resource use contributes to pollution, violent conflict over scarce resources, loss of biodiversity, and diminished quality of life for humans. Moreover, the “safe” amount of carbon dioxide—350 parts per million—has been exceeded. The health care industry is responsible for 4–5% of total world emissions,[i] which is similar to the global food sector.[ii] Health care carbon emissions come from health care infrastructures, supply chains and health care delivery. Increasingly, health care delivery is reliant on technologies which require the use of artificial intelligence to provide supportive care, such as triage algorithms, electronic patient records, and robotics.[iii] While these technological innovations have advanced health care significantly, they also contribute to the negative effects on the environment, among others, through carbon emissions. The environmental impacts of artificial intelligence (AI) in health care—in particular—are understudied. This research seeks to fill this gap. Methods: Our team ran an exploratory search in Scopus and PubMed to identify studies that integrate environmental sustainability, artificial intelligence, and health. Results: Our research initially yielded 735 studies. 77 of these studies focused on an environmental concern of a health technology or AI-application in a health care setting, but most of the articles in this subset addressed lowering energy consumption of a specific technology, such as a sensor or monitoring technology. Conclusions: While there have been studies looking at AI in health care; sustainability in AI; and sustainability in health care, little attention has been paid to the interface between all three. [i] Karliner, J., Slotterback, S., Boyd, R., Ashby, B., & Steele, K. 2019. Health Care’s Climate Footprint: How the Health Sector Contributes to the Global Climate Crisis and Opportunities for Action Healthcare Without HarmARUP; September. [ii] Pichler, P. P., Jaccard, I. S., Weisz, U., & Weisz, H. 2019 International Comparison of Health Care Carbon Footprints, Environmental Research Letters 14, no. 6: 064004. [iii] Khaliq, Abdul, Ali Waqas, Qasim Ali Nisar, Shahbaz Haider, and Zunaina Asghar. 2022. Application of AI and robotics in hospitality sector: A resource gain and resource loss perspective. Technology in Society 68: 101807.
Integral system safety for machine learning in the public sector
An empirical account
This paper introduces systems theory and system safety concepts to ongoing academic debates about the safety of Machine Learning (ML) systems in the public sector. In particular, we analyze the risk factors of ML systems and their respective institutional context, which impact the ability to control such systems. We use interview data to abductively show what risk factors of such systems are present in public professionals' perceptions and what factors are expected based on systems theory but are missing. Based on the hypothesis that ML systems are best addressed with a systems theory lens, we argue that the missing factors deserve greater attention in ongoing efforts to address ML systems safety. These factors include the explication of safety goals and constraints, the inclusion of systemic factors in system design, the development of safety control structures, and the tendency of ML systems to migrate towards higher risk. Our observations support the hypothesis that ML systems can be best regarded through a systems theory lens. Therefore, we conclude that system safety concepts can be useful aids for policymakers who aim to improve ML system safety.
Data sufficient products
Speculative design explorations for sustainable digital futures
Correction to
Toward Sociotechnical AI: Mapping Vulnerabilities for Machine Learning in Context (Minds and Machines, (2024), 34, 2, (12), 10.1007/s11023-024-09668-y)
In this article, the following errors have been missed in the corrections stage and the same has been corrected with the correction article. Data Availability section was inadvertently published and it has been removed from the article. The reference Wolters, A. (2022). Guiding the specification of sociotechnical Machine Learning systems: Addressing vulnerabilities and challenges in Machine Learning practice (Unpublished master’s thesis). Massachusetts Institute of Technology is updated to Wolters, A. (2022). Guiding the specification of sociotechnical Machine Learning systems: Addressing vulnerabilities and challenges in Machine Learning practice (Published master’s thesis). Delft University of Technology. Missing equal contribution text and Acknowledgments were included in the original article.