Circular Image

L. Cavalcante Siebert

info

Please Note

33 records found

Review (2026) - Sarah Zainab Mbawa, Roelof Anne Jelle de Vries, Luciano Cavalcante Siebert, Koen van Turnhout, Willem-Paul Brinkman
Background: The ongoing adoption and use of digital interventions offer promising opportunities to meet the growing demand for mental health support. The effectiveness, implementation, and usage of these interventions depend on how well they are designed and evaluated. However, given the emerging nature of design research in this area, there is still no clear consensus on the specific principles and guidelines for developing digital mental health interventions (DMHIs). There seems to be a lack of clarity regarding the best practices for designing and evaluating these tools. Objective: We aimed to investigate and report on the design principles and evaluation approaches used in digital interventions specific to mental health care. Additionally, we sought to outline how these principles and approaches are applied in research. Methods: This scoping review was conducted in accordance with the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) guidelines for scoping reviews. The literature search was performed in 2 electronic databases, SCOPUS and Web of Science, across 3 iterations from January 2024 to January 2025. A total of 2 independent reviewers screened and selected papers based on predefined inclusion and exclusion criteria, followed by data extraction from the selected studies. The data were then synthesized by categorizing the papers according to the primary research aim of each study. The inclusion criteria covered studies involving populations with mental health challenges or users of DMHIs, any digital tools for mental health care, and principles or strategies related to the design, evaluation, or implementation of DMHIs. Results: Our search identified 401 papers, of which 17 met the inclusion criteria for this review. Among these, 11 focused on evaluation studies, while 6 covered both design and evaluation studies (mixed). An iterative user-centered development process, expert inclusion, usability testing, specification of design elements, and user tracking and feedback were identified as common design principles used in studies focused on DMHIs. Evaluation approaches were shaped by the evaluation goal, which influenced the chosen methodologies. We also summarize the recommendations for implementation highlighted in some studies. Based on our findings, we propose 8 guidelines emphasizing stakeholder involvement in the development process and the need for clear justifications for design decisions, among other considerations. Conclusions: Design principles used in DMHI development include user-centered development, expert inclusion, and usability testing, while evaluation approaches often rely on randomized controlled trials to assess efficacy. Qualitative and mixed-method approaches are commonly adopted by studies to capture user experience and bridge both process and outcome measures. We recommend that future research explicitly report its design justification and adopt a multiperspective approach in the research and design of DMHIs. ...
Inverse Reinforcement Learning (IRL) seeks to infer reward functions from expert demonstrations. When demonstrations come from multiple experts with different intentions, the problem becomes Multi-Intention IRL (MI-IRL). MI-IRL methods generally follow a rewardfirst paradigm: they ask which reward, if followed, could have generated each trajectory. This leads to trajectory similarity being based on reward likelihood rather than on the relationships between behaviors. In deep generative MI-IRL methods, this paradigm further couples behavior clustering and reward learning, making such methods dependent on prior knowledge of the true number of modes K∗. This limits adaptability to unseen behaviors, and restricts the analysis to learned rewards rather than relationships across behaviors. In contrast, we approach MI-IRL from a different standpoint. We propose Contrastive Multi-Intention IRL (CoMI-IRL), a transformer-based unsupervised behavior-first framework: it first learns a representation of trajectory dynamics that captures intrinsic behavioral similarity, then clusters trajectories in that space, and finally learns a reward for each cluster. Our experiments show that CoMI-IRL achieves higher clustering and reward learning performance when compared to Ess-InfoGAIL without needing K∗ or labels, while allowing for visual interpretation of behavior relationships and adaptation to unseen behavior without full retraining. ...

Diagnostics for MORL Policy Selection

Real-world decision-making often requires optimizing multiple competing objectives simultaneously. In reinforcement learning (RL), this is typically addressed by combining reward signals into a single scalar objective via a scalarization function, which can be fragile: small changes in the weights can induce drastically different policies. Multi-objective reinforcement learning (MORL) instead produces sets of policies that explicitly represent trade-offs between objectives. However, these policies are typically presented to the decision maker only through their value vectors, which can obscure substantial behavioral variation: policies that induce distinct trajectories may appear indistinguishable when evaluated solely by expected returns. We propose an exploratory diagnostic workflow that automatically highlights behavioral variation along the Pareto front that objective values alone do not reveal, providing both quantitative and visual tools to support policy inspection. We validate our approach on simple grid examples and scale it to continuous control benchmarks, demonstrating that it remains effective as problem complexity increases. ...
Aggregating multiple annotations into a single ground truth label may hide valuable insights into annotator disagreement, particularly in tasks where subjectivity plays a crucial role. In this work, we explore methods for identifying subjectivity in recognizing the human values that motivate arguments. We evaluate two main approaches: inferring subjectivity through value prediction vs. directly identifying subjectivity. Our experiments show that direct subjectivity identification significantly improves the model performance of flagging subjective arguments. Furthermore, combining contrastive loss with binary cross-entropy loss does not improve performance but reduces the dependency on per-label subjectivity. Our proposed methods can help identify arguments that individuals may interpret differently, fostering a more nuanced annotation process. ...
Human values capture what people and societies perceive as desirable, transcend specific situations and serve as guiding principles for action. People’s value systems motivate their positions on issues concerning the economy, society and politics among others, influencing the arguments they make. Identifying the values behind arguments can therefore help us find common ground in discourse and uncover the core reasons behind disagreements. Transformer-based large language models (LLMs) have exhibited remarkable performance across language generation and analysis. However, leveraging LLMs in sociotechnical systems that assist with discourse and argumentation necessitates systematically evaluating their ability to analyse and identify the values behind arguments, an under-explored research direction. Using a multi-level human value taxonomy inspired by the Schwartz Theory of Basic Human Values, we present a systematic and critical evaluation of GPT-3.5-turbo in human value identification from a dataset of multi-cultural arguments, across the zero-shot, few-shot and chain-of-thought prompting strategies, carrying forward from prior research on this task which leveraged a fine-tuned BERT model. We observe that prompting strategies exhibit performance levels close to, but still behind fine-tuning for value classification. We also detail some challenges associated with value classification with LLMs, offering potential directions for future research. ...
Understanding citizens’ values in participatory systems is crucial for citizen-centric policy-making. We envision a hybrid participatory system where participants make choices and provide motivations for those choices, and AI agents estimate their value preferences by interacting with them. We focus on situations where a conflict is detected between participants’ choices and motivations, and propose methods for estimating value preferences while addressing detected inconsistencies by interacting with the participants. We operationalize the philosophical stance that “valuing is deliberatively consequential.” That is, if a participant’s choice is based on a deliberation of value preferences, the value preferences can be observed in the motivation the participant provides for the choice. Thus, we propose and compare value preferences estimation methods that prioritize the values estimated from motivations over the values estimated from choices alone. Then, we introduce a disambiguation strategy that combines Natural Language Processing and Active Learning to address the detected inconsistencies between choices and motivations. We evaluate the proposed methods on a dataset of a large-scale survey on energy transition. The results show that explicitly addressing inconsistencies between choices and motivations improves the estimation of an individual’s value preferences. The disambiguation strategy does not show substantial improvements when compared to similar baselines—however, we discuss how the novelty of the approach can open new research avenues and propose improvements to address the current limitations. ...

Integrated Inverse Reinforcement Learning and Model Predictive Control for Human-Robot Collaboration

Journal article (2025) - Angelo Caregnato-Neto, Luciano Cavalcante Siebert, Arkady Zgonnikov, Marcos R.O.A. Maximo, Rubens J.M. Afonso
One of the key issues in human-robot collaboration is the development of computational models that allow robots to predict and adapt to human behavior. Much progress has been achieved in developing such models, as well as control techniques that address the autonomy problems of motion planning and decision-making in robotics. However, the integration of computational models of human behavior with such control techniques still poses a major challenge, resulting in a bottleneck for efficient collaborative human-robot teams. In this context, we present a novel architecture for human-robot collaboration: Adaptive Robot Motion for Collaboration with Humans using Adversarial Inverse Reinforcement learning (ARMCHAIR). Our solution leverages adversarial inverse reinforcement learning and model predictive control to compute optimal trajectories and decisions for a mobile multi-robot system that collaborates with a human in an exploration task. During the mission, ARMCHAIR operates without human intervention, autonomously identifying the necessity to support and acting accordingly. Our approach also explicitly addresses the network connectivity requirement of the human-robot team. Extensive simulation-based evaluations demonstrate that ARMCHAIR allows a group of robots to safely support a simulated human in an exploration scenario, preventing collisions and network disconnections, and improving the overall performance of the task. ...

Investigating Value Conflicts in Smart Home through Enactment and Co-speculation

Conference paper (2025) - Nazli Cila, Maria Luce Lupetti, Luciano Cavalcante Siebert, Janna Van Grunsven
Smart home technologies embed values such as sustainability, comfort, privacy, and security, which can sometimes conflict with one another, considering the complexities of domestic environments. This paper investigates the potential implications of these value conflicts and the corresponding design challenges. Through an enactment session and co-speculations with professional actors, we explored what it means to navigate multiple values simultaneously, live with products that impose their own values, and manage value conflicts both with and among smart products. The findings challenge the seamless and harmonious vision of smart homes conceived by technologists, proposing shifts in the common narrative: from value alignment to value transparency, from service provision to mutual care, and from autonomy to responsiveness. We discuss that acknowledging value conflicts, rather than eliminating them, is an opportunity to gain a deeper understanding of users and home environments and guide the design of smart home technologies. ...

Ethical Design Requirements for The New Generation of Artificially Intelligent Agents

Journal article (2025) - S.K. Kuilman, Sven Nyholm, S.N.R. Buijsman, L. Cavalcante Siebert
Recently, several large tech companies have pushed the notion of AI assistants into the public debate. These envisioned agents are intended to far outshine current systems, as they are intended to be able to manage our affairs as if they are personal assistants. In turn, this ought to give users a leg up, as one prominent tech exec has put it. However, it remains to be seen how these Personal AI Assistants (PAIAs) are implemented, and critical reflection on how and whether they can be implemented in a responsible way is needed. Currently, such agents are undertheorized and this may cause us to misunderstand their value and capacity. In this paper, we explore and critique the potential for responsible implementation by considering some design requirements based on the notion of meaningful human control. If we desire to have control over such assistants, then we need to be able to do so meaningfully and effectively. In looking at the design requirements, we run into the issue that their broad and differing capacities make any kind of design requirements hard because there are simply no standards to which we can measure PAIAs. Furthermore, it seems that the implementation of these assistants will be a matter of trade-offs both in capacities and in values, which will likely lead to enhancement for some rather than an improvement for all. ...
Relevancy is a prevalent term in value alignment. We either need to keep track of the relevant moral reasons, we need to embed the relevant values, or we need to learn from the relevant behaviour. What relevancy entails in particular cases, however, is often ill-defined. The reasons for this are obvious, it is hard to define relevancy in a way that is both general and concrete enough to give direction towards a specific implementation. In this paper, we describe the inherent difficulty that comes along with defining what is relevant to a particular situation. Simply due to design and the way an AI system functions, we need to state or learn particular goals and circumstances under which that goal is completed. However, because of both the changing nature of the world and the varied wielders and users of such implements, misalignment occurs, especially after a longer amount of time. We propose a way to counteract this by putting contestability front and centre throughout the lifecycle of an AI system, as it can provide insight into what is actually relevant at a particular instance. This allows designers to update the applications in such a manner that they can account for oversight during design. ...
Automatic design tools are being developed to assist designers handle tedious work at scale. However, knowledge gaps still exist in harnessing deep learning models to learn from human experience for more efficient design generation while keeping the data understandable and interoperable. Moreover, human-in-the-loop approach is largely neglected, which are essential for more user-centered design. This research utilizes graph data to parametrically represent housing designs and graph-representative deep generative models for design generation, which provides an interactive design approach for the users at every step. This method would facilitate the human-centered design process by returning feasible and parametric housing design alternatives. ...

Towards a dynamic balance of humans and AI-based systems within our global society and environment

Book chapter (2024) - Frank Flemisch, Marcel Baltzer, David Abbink, L. Cavalcante Siebert, Jurriaan van Diggelen, Nicolas Daniel Herzberger, Mark Draper, Michael Boardman, Marie Pierre Pacaux-Lemoine, Joscha Wasser
While Meaningful Human Control (MHC) is at the very heart of the Edward Elgar research handbook, this specific chapter addresses the questions how MHC is rooted in the history of human artefacts and human-machine systems, how it is related to the term control, ability, responsibility, authority, autonomy and finally accountability. The chapter sketches, step by step, a holistic, cybernetic model of the most important relationships between MHC and its related concepts interconnected over this holistic big picture map. Starting point are existing control systems and their evolution through history, followed by the interrelationship between the small-scale human-machine or human-AI system, and the increasingly bigger system of systems, organizations, societies and our global environment. The goal of this bow-tie shaped system map is to enable a better balance between global and local perspectives, and therefore enable a more efficient and better design, engineering and evaluation of such systems. ...
Artificial Intelligence (AI) has become one of the most dicussed topics of today and are being used to support solving complex problems. AI has given opportunities for efficiency, control, safety while raising issues around trust, optimism and responsibility. One of the prominent features of AI resides in the digitalization of the built environment. Optimizing the built environment to improve quality of life, adapt to climate change and respond to crises requires strategies to redesign, reproduce and manage the traditional ways the built environment has been shaped. In this chapter, we present demonstate how we can use AI for post-pandemic recovery. To do that, we first start with addressing digital transformation and the role of AI. We then discuss how we can accelerate this transformation in cities. We will reflect on the covid-19 crises and the impact of the crises in the built environment. We argue that the use of AI raises new possibilities, questions and problems around how we can better organize the built environment and more inclusive participation while supporting existing logics of the built environment. ...
With the growing capabilities and pervasiveness of AI systems, societies must collectively choose between reduced human autonomy, endangered democracies and limited human rights, and AI that is aligned to human and social values, nurturing collaboration, resilience, knowledge and ethical behaviour. In this chapter, we introduce the notion of self-reflective AI systems for meaningful human control over AI systems. Focusing on decision support systems, we propose a framework that integrates knowledge from psychology and philosophy with formal reasoning methods and machine learning approaches to create AI systems responsive to human values and social norms. We also propose a possible research approach to design and develop self-reflective capability in AI systems. Finally, we argue that self-reflective AI systems can lead to self-reflective hybrid systems (human + AI), thus increasing meaningful human control and empowering human moral reasoning by providing comprehensible information and insights on possible human moral blind spots. ...

A Roadmap for Inclusive XAI for people with Cognitive Disabilities

Journal article (2024) - Myrthe L. Tielman, Mari Carmen Suárez-Figueroa, Arne Jönsson, Mark A. Neerincx, Luciano Cavalcante Siebert
Artificial intelligence (AI) is increasingly prevalent in our daily lives, setting specific requirements for responsible development and deployment: The AI should be explainable and inclusive. Despite substantial research and development investment in explainable AI, there is a lack of effort into making AI explainable and inclusive to people with cognitive disabilities as well. In this paper, we present the first steps towards this research topic. We argue that three main questions guide this research, namely: 1) How explainable should a system be?; 2) What level of understanding can the user reach, and what is the right type of explanation to help them reach this level?; and 3) How can we implement an AI system that can generate the necessary explanations? We present the current state of the art in research on these three topics, the current open questions and the next steps. Finally, we present the challenges specific to bringing these three research topics together, in order to eventually be able to answer the question of how to make AI systems explainable also to people with cognitive disabilities. ...
Book chapter (2024) - David Abbink, Daniele Amoroso, L. Cavalcante Siebert, M.J. van den Hoven, Giulio Mecacci, F. Santoni De Sio
This handbook presents the concept of ‘meaningful human control’ (MHC) over AI systems from the perspectives of (i) philosophy and ethics, (ii) law and governance, and (iii) design and engineering. The introductory chapter addresses the motivations and recent developments in MHC, introducing each perspective and related chapters. These three disciplinary perspectives scrutinize how MHC intertwines with philosophical debates on moral responsibility, societal concerns regarding control over technological advancements in legal frameworks, and the engineering complexities of designing and developing AI systems while ensuring human control and responsibility. Additionally, cross-cutting aspects on MHC over AI systems are also introduced and discussed through (iv) interdisciplinary and systemic perspectives. By offering a contextualized introduction to the perspectives considered in this handbook, this chapter aims to present the handbook’s various approaches and points of interest for a diverse audience, highlighting potential entry points into this multidisciplinary volume. ...

Building global explanations from disagreeing local post-hoc explainers

Journal article (2024) - Andrea Agiollo, Luciano Cavalcante Siebert, Pradeep K. Murukannaiah, Andrea Omicini
The expressive power and effectiveness of large language models (LLMs) is going to increasingly push intelligent agents towards sub-symbolic models for natural language processing (NLP) tasks in human–agent interaction. However, LLMs are characterised by a performance vs. transparency trade-off that hinders their applicability to such sensitive scenarios. This is the main reason behind many approaches focusing on local post-hoc explanations, recently proposed by the XAI community in the NLP realm. However, to the best of our knowledge, a thorough comparison among available explainability techniques is currently missing, as well as approaches for constructing global post-hoc explanations leveraging the local information. This is why we propose a novel framework for comparing state-of-the-art local post-hoc explanation mechanisms and for extracting logic programs surrogating LLMs. Our experiments—over a wide variety of text classification tasks—show how most local post-hoc explainers are loosely correlated, highlighting substantial discrepancies in their results. By relying on the proposed novel framework, we also show how it is possible to extract faithful and efficient global explanations for the original LLM over multiple tasks, enabling explainable and resource-friendly AI techniques. ...

Multi-Agent Reinforcement Learning for Incentive-Based Residential Demand Response

This paper presents a decentralized Multi-Agent Reinforcement Learning (MARL) approach to an incentive-based Demand Response (DR) program, which aims to maintain the capacity limits of the electricity grid and prevent grid congestion by financially incentivizing residential consumers to reduce their energy consumption. The proposed approach addresses the key challenge of coordinating heterogeneous preferences and requirements from multiple participants while preserving their privacy and minimizing financial costs for the aggregator. The participant agents use a novel Disjunctively Constrained Knapsack Problem optimization to curtail or shift the requested household appliances based on the selected demand reduction. Through case studies with electricity data from 25 households, the proposed approach effectively reduced energy consumption's Peak-to-Average ratio (PAR) by 14.48% compared to the original PAR while fully preserving participant privacy. This approach has the potential to significantly improve the efficiency and reliability of the electricity grid, making it an important con-tribution to the management of renewable energy resources and the growing electricity demand. ...

The problem with evaluative standards

Journal article (2023) - Sietze Kai Kuilman, Koji Andriamahery, Catholijn M. Jonker, Luciano Cavalcante Siebert
Many technological systems these days interact with their environment with increasingly little human intervention. This situation comes with higher stakes and consequences that society needs to manage. No longer are we dealing with 404 pages: AI systems today may cause serious harm. To address this, we wish to exert a kind of control over these systems, so that they can adhere to our moral beliefs. However, given the plurality of values in our societies, which “oughts” ought these machines to adhere to? In this article, we examine Borda voting as a way to maximize expected choice-worthiness among individuals through different possible “implementations” of ethical principles. We use data from the Moral Machine experiment to illustrate the effectiveness of such a voting system. Although it appears to be effective on average, the maximization of expected choice-worthiness is heavily dependent on the formulation of principles. While Borda voting may be a good way of ensuring outcomes that are preferable to many, the larger problems in maximizing expected choice-worthiness, such as the capacity to formulate credences well, remain notoriously difficult; hence, we argue that such mechanisms should be implemented with caution and that other problems ought to be solved first. ...
Conference paper (2023) - Andrea Agiollo, Luciano Cavalcante Siebert, Pradeep Kumar Murukannaiah, Andrea Omicini
Although popular and effective, large language models (LLM) are characterised by a performance vs. transparency trade-off that hinders their applicability to sensitive scenarios. This is the main reason behind many approaches focusing on local post-hoc explanations recently proposed by the XAI community. However, to the best of our knowledge, a thorough comparison among available explainability techniques is currently missing, mainly for the lack of a general metric to measure their benefits. We compare state-of-the-art local post-hoc explanation mechanisms for models trained over moral value classification tasks based on a measure of correlation. By relying on a novel framework for comparing global impact scores, our experiments show how most local post-hoc explainers are loosely correlated, and highlight huge discrepancies in their results—their “quarrel” about explanations. Finally, we compare the impact scores distribution obtained from each local post-hoc explainer with human-made dictionaries, and point out that there is no correlation between explanation outputs and the concepts humans consider as salient. ...