As machines take on more complex tasks, we move from asking how well they can perform those tasks to asking how well they can collaborate with us. After all, the goal of building technology should be to improve our lives, not make them harder, but that requires mutual understanding, coordination, and trust. This dissertation looks at the role of trust in decision-making within teams of humans and semi-autonomous machines, including AI systems, agents and robots. In particular, we look at the concept of artificial trust, that is when an artificial agent reasons about someone’s trustworthiness.

Trust is central to human decision-making. When we work with others, we constantly judge who is reliable and who is not, and we delegate tasks based on how trustworthy we think our teammates are and what risks those choices pose to us individually and to the team as a whole. When we see someone is not very trustworthy for a task they are expected to perform, and that poses risks to them or us, we can also offer help. The same logic can extend to artificial agents. When humans and intelligent artificial agents work together, artificial agents must not only be trusted by humans but also develop ways of assessing how trustworthy their human partners are for different tasks. In other words, artificial agents can use artificial trust to make decisions. This requires defining, modelling and using trustworthiness for decision-making in human–agent teamwork. We go over all of those steps in this dissertation.

This research argues that human trustworthiness is not only about a few internal traits such as ability, benevolence or integrity. In fact, what counts as trustworthiness can vary depending on the task and team characteristics. For example, if success in a task depends only on being somewhere on time, then punctuality may be the only relevant trait. Furthermore, to perform a task successfully, a person not only needs to be able to do it but also needs to choose to do it. Our research shows that in human–agent collaborative scenarios, task choices can often be explained by contextual cost–benefit reasoning. People consider a task by weighing its potential benefits, such as reward, against its potential costs, such as effort and time. This translates into a person’s willingness to do a task. At the end of the day, it is not enough that someone has the skills to succeed in a certain task, but it is also important that they are willing to do it.

Although it is challenging to infer someone’s willingness for different tasks, both for humans and machines, we can try to find ways around it. For example, asking directly about teammates’ competence and willingness can give machines better information to work with, helping them to make fairer, more transparent and more efficient decisions. One of our studies found that people want artificial teammates, such as robots, to consider their preferences and willingness, but only in non-critical situations. In urgent or high-stakes work, efficiency mattered most. However, over time, recognising willingness may help make collaboration more sustainable and engaging.

This dissertation focusses on developing machines that can complement and even augment human teams, instead of replacing people. For that to happen, we need a solid understanding of how people make decisions, what motivates them, and what they value in teamwork and in their artificial teammates. At the same time, giving machines the power to trust or distrust humans raises ethical risks. Used wrongly, it could harm individuals or undermine their autonomy. These concerns are especially pressing in areas such as defence, where collaborative technologies are already being explored, and can contribute to the escalation of armed conflicts. As such, the goal of this dissertation by building artificial trust is not to maximise efficiency at all costs. Instead, we hope to help design systems that support human well-being, safety, and dignity. This requires combining theoretical and technical advances from different disciplines, such as the social sciences and computer science, and carefully reflecting on the contexts where these systems are deployed.

People often remember parts of conversations that are important to them, such as something personal, useful, or emotionally engaging. These memories help shape relationships, guide decisions, and influence how we communicate in the future. While many computer systems can already track emotions or attention in group settings, no previous research has looked at how specific moments in conversations are stored in memory or how this process could be predicted using technology.

On the path towards training computer systems to predict such memorable moments, this dissertation first introduces a new dataset called the MeMo corpus (Chapter 2). It includes group video conversations along with direct reports from participants about which moments they remembered. The data was collected in a way that reflects real-life conversations, using repeated video calls and memory reports that are linked to specific moments in time.

The study in chapter 3 then asks whether affective signals, such as emotional tone or energy in a conversation, could help predict what people will remember. These kinds of emotional signals are often used in artificial intelligence systems. However, the results show that emotional signals alone are not enough to explain what people remember from a conversation.

Next, in chapter 4, the dissertation looks at other behavioural signs, such as where people were looking and who was speaking. These signals were found to be significantly linked with memory: for example, people tend to remember parts of a conversation where there was shared attention or dynamic speaking patterns. Using these signals, simple computer models were able to predict which parts of the conversation were more likely to be remembered. The study also looked at why people remembered certain moments and found that many of them were related to personal relevance or social connection.

This work shows that it is possible to build systems that recognise which parts of a conversation are more memorable. This can be useful for improving automatic meeting tools, personal assistants, and other technologies that support communication and augment memory.

This thesis investigates how conversational interaction, specifically through alignment dialogue, can help behavior support systems better understand and adapt to a user's real-time needs. Behavior support systems are often deployed to help individuals manage health-related goals, such as increasing physical activity or adhering to dietary guidelines. However, these systems tend to rely on pre-collected behavioral data, which makes it difficult to adapt support to the nuanced, changing circumstances of daily lives. We thus introduce alignment dialogues as conversations between an AI system and a user aimed at eliciting reasons for non-adherence, surfacing underlying values, and revealing contextual barriers. Rather than merely logging behavior, these dialogues help uncover the user's lived experience to iteratively build a more accurate user model at run time.
After conducting an exploratory focus group study, we found that probing for deeper knowledge like user values often triggers negative emotional responses, suggesting that the interpersonal impact of an agent's language is a critical design dimension. These findings informed a large-scale user experiment testing different dialogue strategies, which highlighted a tension between conversational depth and usability: while open-ended questions allow for greater perceived completeness, they also increase cognitive effort compared to multiple-choice formats. To assess how these outputs could be used in healthcare, we explored sharing dialogue content with professionals as a Conversational Intermediary AI (CIAI). Interviews and experiments suggest that while these agents facilitate early diagnoses and sensitive discussions, the effectiveness of the information is sensitive to the presentation format and length of the dialogue.
Finally, we demonstrate technical feasibility through a system prototype that transforms user responses into structured knowledge using RDF triples and a user knowledge graph. This allows the system to cross-reference user data with medical expertise to recommend actions that are contextually relevant and clinically sound. Our findings show that the effectiveness of alignment is determined not only by the accuracy of the information gathered but by the interpersonal and emotional quality of the interaction. This approach redefines personalization from categorization to interpretation, where systems must learn to make sense of the user's situation. Ultimately, this research suggests that personalization grounded in dialogue opens new paths for hybrid collaboration, advocating for behavior support systems that adapt by conversing, reflecting, and aligning with users.

This thesis investigates how to learn local abstractions for scalable sequential decision making in large and complex systems. Real-world environments are typically dynamic, multiagent, and characterized by an extremely high number of state variables. As a result, exhaustive reasoning is computationally infeasible for an agent. Abstraction serves to reduce this complexity by focusing on the essential aspects of the environment while disregarding irrelevant details. A central theme of this work is the study of specific local abstractions and effective methods to learn them.
We introduce a perspective that unifies different approaches to state abstraction, showing how seemingly distinct methods can be systematically organized within a broader conceptual framework. This clarifies the connections between existing models and lays the foundation for more systematic development and reuse of abstraction techniques.
We explore approximate influence-based abstraction, an approach that enables building small local models complemented by learned representations of the external influence on local dynamics. We establish theoretical performance guarantees for approximate influence models, proving that the performance loss can be bounded in terms of the approximation error. The analysis is supported by empirical studies that show that accurate influence approximations improve performance in practice.
A further contribution is the investigation of the learnability of influence. We demonstrate that accurate influence representations can be learned efficiently, even in largescale and long-horizon scenarios. Empirical evaluations show that small recurrent architectures are often sufficient to approximate the influence effectively and generalize beyond the training horizon.
In conclusion, this dissertation advances the foundations of state abstraction and principled applications of approximate influence-based abstraction. It provides a coherent framework for understanding state abstraction, establishes performance guarantees for approximate influence representations, and demonstrates the feasibility of influence learning. Together, these contributions offer insights into scalable and principled methods for sequential decision making in complex systems.

Deceptive self-reporting in diabetes lifestyle management (DLM) systems limits their ability to offer meaningful and accurate support. Deception can function as a self-protective mechanism, driven by factors such as low self-esteem or the desire to protect self-image. This research builds on CHIP, a chatbot-based DLM prototype, to explore whether the language framing of its responses can influence the psychological determinants of deception. Two framing strategies, empathic and affirming, were implemented and evaluated through a pilot user study, which offers insights for refining the intervention and experimental design in future research.

Lifestyle management systems aim to provide personalized health guidance by interpreting patient's self-reported data. However, these systems often overlook the temporal consistency of behavioral patterns, risking inaccurate or misleading recommendations. To address this, we present an adapted version of a diabetes lifestyle management system that integrates a conflict detection component into its knowledge graph (KG) architecture. This addition enables the system to identify temporal inconsistencies in patient data based on formally defined rules. We developed handcrafted constraints to capture clinically meaningful relationships, such as the expected timing between physical activity and glucose responses. A tunable threshold parameter, $\varepsilon$, is introduced to account for minor fluctuations and variability in activity intensity, enhancing the physiological realism of constraint evaluations. Handcrafted rules were favored over mined constraints due to the limited availability of real-world data, though future work may incorporate mining techniques when larger datasets become accessible. The system was tested on synthetic case studies that simulate typical lifestyle scenarios, demonstrating its potential to flag inconsistencies and improve the reliability of temporal reasoning. These results contribute to the design of more trustworthy, context-aware decision support tools for personalized health management.

Unreliable patient self-reporting complicates diabetes management. This study investigates how AI-generated summaries of patient-chatbot conversations can be structured to help healthcare professionals detect deception and non-adherence. To address this, we developed a novel pipeline by first identifying four key behavioral indicators from literature and then using advanced prompt engineering to automatically flag these in structured summaries. The system's effectiveness was evaluated in an annotation experiment using synthetic chat data. The results showed that the summaries did not improve detection accuracy, increased annotation time, and revealed a critically low inter-annotator agreement. These findings highlight the inherent subjectivity and complexity of the detection task, demonstrating that the developed summarization method is not an effective intervention. Although the approach was unsuccessful, this research contributes a novel summarization pipeline, an open-source annotation tool, and a synthetic dataset, establishing a baseline for future work in enhancing doctor-patient communication.