Human-Agent Alignment Dialogues

Abstract

This thesis investigates how conversational interaction, specifically through alignment dialogue, can help behavior support systems better understand and adapt to a user's real-time needs. Behavior support systems are often deployed to help individuals manage health-related goals, such as increasing physical activity or adhering to dietary guidelines. However, these systems tend to rely on pre-collected behavioral data, which makes it difficult to adapt support to the nuanced, changing circumstances of daily lives. We thus introduce alignment dialogues as conversations between an AI system and a user aimed at eliciting reasons for non-adherence, surfacing underlying values, and revealing contextual barriers. Rather than merely logging behavior, these dialogues help uncover the user's lived experience to iteratively build a more accurate user model at run time.
After conducting an exploratory focus group study, we found that probing for deeper knowledge like user values often triggers negative emotional responses, suggesting that the interpersonal impact of an agent's language is a critical design dimension. These findings informed a large-scale user experiment testing different dialogue strategies, which highlighted a tension between conversational depth and usability: while open-ended questions allow for greater perceived completeness, they also increase cognitive effort compared to multiple-choice formats. To assess how these outputs could be used in healthcare, we explored sharing dialogue content with professionals as a Conversational Intermediary AI (CIAI). Interviews and experiments suggest that while these agents facilitate early diagnoses and sensitive discussions, the effectiveness of the information is sensitive to the presentation format and length of the dialogue.
Finally, we demonstrate technical feasibility through a system prototype that transforms user responses into structured knowledge using RDF triples and a user knowledge graph. This allows the system to cross-reference user data with medical expertise to recommend actions that are contextually relevant and clinically sound. Our findings show that the effectiveness of alignment is determined not only by the accuracy of the information gathered but by the interpersonal and emotional quality of the interaction. This approach redefines personalization from categorization to interpretation, where systems must learn to make sense of the user's situation. Ultimately, this research suggests that personalization grounded in dialogue opens new paths for hybrid collaboration, advocating for behavior support systems that adapt by conversing, reflecting, and aligning with users.