Social engineering is a major threat in today's cybersecurity landscape. Unlike other types of cyberattacks, social engineering focuses on exploiting human psychology rather than technical vulnerabilities. By manipulating individuals into revealing sensitive information or taking actions against their own interests, malicious actors can cause significant harm, ranging from financial loss to compromised systems. As a socio-technical problem, it requires not only technical measures but also awareness on the human side to defend against these threats. With advancements in technology and, most recently, generative artificial intelligence, social engineering attacks are becoming increasingly sophisticated, highlighting the urgency to educate individuals about how to recognise and defend against them. While existing efforts in research and industry mostly focus on interventions for workplace settings, little attention has been given to approaches targeted at the general public or everyday life contexts. This thesis addresses this gap by designing and evaluating a serious game to raise awareness of social engineering, focusing on families as the target group. The resulting game, “Connected & Protected”, employs an asymmetric tabletop format in which one player takes on the role of a social engineer, while the others play as family members, trying to protect their personal data from the attacker. The aim of the game is to educate players about common influence principles used in social engineering, how to counter them, and the risks that come with sharing personal data. The project takes a human-centred approach and involves users in multiple phases, including feedback sessions and playtesting to support concept development and prototyping. The game was evaluated using a pre- and post-test study design with 15 participants across five family groups, including a delayed post-test two weeks later to measure knowledge retention. The evaluation focused on three main areas: social engineering awareness, self-perceptions, and game experience. Results indicated that the game had positive effects on social engineering awareness, with significant improvements after gameplay that were largely retained at the delayed post-test. Participants improved in their ability to recognise influence principles in social engineering scenarios and showed increased knowledge about defensive strategies and data-sharing risks. Self-efficacy and perceived preparedness to deal with social engineering attempts likewise increased significantly after gameplay, while perceived susceptibility did not change notably. The game experience was overall well-received, with particularly high scores for enjoyment and audiovisual appeal, along with a positively rated perceived learning effect and willingness to play the game again. With this project, we provide novel insights into designing serious games for social engineering education aimed at the general public, leveraging the family setting as a medium for shared learning and intergenerational knowledge exchange. The contributions can serve as a starting point for practitioners and policymakers to extend security awareness interventions beyond the workplace, making social engineering education more accessible by bringing it into people's homes.

Error messages are a primary feedback channel in programming environments, yet they often obstruct  progress, especially for novices. Although large language models (LLMs) are widely used for code  generation and debugging assistance, there is limited empirical evidence that LLM-rephrased error  messages consistently improve code correction capabilities, and skill-adaptive designs remain largely  unexplored. We introduce a framework that uses an LLM to rewrite Python standard interpreter errors  in two different styles, which are designed to be tailored to user expertise: the pragmatic style, which is  concise and action oriented, and the contingent style, which provides scaffolded, actionable guidance  organized by a clear argumentation model. To measure Python skill level reliably, we first ran a pilot  study that informed the design of a short 8 multiple-choice question assessment focused on debugging  and error-message interpretation. We then used this instrument in the main study to classify participants  as either novices or experts.

To gauge the effectiveness of our LLM-enhanced programming error messages (PEMs), we evaluated  the framework in a crowdsourced Prolific study with 103 participants. We measured objective outcomes  such as fix rate, time to fix, and number of attempts to fix, while also capturing subjective perceptions of  PEMs, including readability, cognitive load, and authoritativeness. Objectively, LLM-enhanced PEMs  showed favorable trends but did not produce statistically significant improvements over the standard  interpreter. Subjectively, novices and experts alike, rated the pragmatic messages as significantly  more readable and helpful, lower in intrinsic and extraneous cognitive load, and considerably less  authoritative. Contingent messages exceeded the baseline on average but did not consistently reach  statistical significance across all of our measurements, which points to a need for tighter control of error  message verbosity and granularity, particularly for beginners.   

These results show that LLMs, especially small-sized ones, are already capable of delivering targeted  text-rewriting interventions that improve the perceived quality of error feedback. Future work should  validate the effects at larger scale and across languages, expand coverage of real-world error contexts,  and pursue true adaptivity in which error message style and level of detail adjust dynamically to user  skill and task state.

Pose estimation models offer promising opportunities for automated feedback in cricket training, but their practical impact is limited by the lack of personalized and understandable explanations. This study investigates how explanation formats can be tailored to users’ expertise levels, focusing on beginner, intermediate, and expert levels, to improve the effectiveness of AI-generated feedback. Based on a literature review of explanation needs and generation methods, we propose a taxonomy linking expertise levels to suitable explanation modes: visual, comparative, and statistical. We implement a set of explanation prototypes aligned with this taxonomy and evaluate them through a user study involving 17 participants across the three expertise levels. Results show that participants rated explanations tailored to their skill level as more useful, trustworthy, and easier to interpret. Statistical validation using Kruskal-Wallis and Dunn’s tests confirmed significant differences in perception between user groups, especially between beginners and experts. These findings demonstrate the value of expertise-based explanation design in cricket analytics and offer design guidelines for future explainable pose estimation systems in sports

Pose estimation models predict multiple interdependent body keypoints, making them a prototypical example of multi-target tasks in machine learning. While existing explainable AI (XAI) techniques have advanced our ability to interpret model outputs in single-target domains, their application to structured outputs remains underdeveloped. This work investigates how XAI methods can be adapted to explain pose estimation models, particularly in the context of cricket shot analysis. Guided by three research questions, we identify key challenges such as capturing inter-keypoint dependencies and providing interpretable explanations of structured outputs. We analyze both geometric and heatmap-level behavior of a pretrained pose estimation model to distinguish between two cricket shots - the pull and the cover drive. Through techniques like cosine similarity on heatmaps and polynomial trajectory modeling, we reveal how the model internally differentiates between similar motion patterns. Our framework introduces novel techniques for inter-keypoint explanation, contributes domain-specific insights into model behavior, and demonstrates the feasibility of interpretable structured predictions in high-dimensional, real-world tasks.

As mental health issues continue to rise around the world, AI chatbots are becoming a promising way to provide accessible and scalable support. This study explores how different levels of chatbot self-disclosure affect users’ willingness to share personal information in a mental health context. A within-subjects experiment was conducted with 94 participants, each interacting with three versions of a chatbot: one with no self-disclosure, one with factual self-disclosure, and one with emotional self-disclosure. Participants engaged in a fictional role-play and rated their willingness to disclose across five personal topics, as well as their level of trust and comfort with the chatbot.

The chatbot using factual self-disclosure received the highest average scores for trust, comfort, and willingness to disclose. However, statistical tests (ANOVA) showed no significant differences between chatbot types on these measures, except for changes in willingness to disclose. Participants who interacted with the emotional chatbot were more likely to report a negative change in their willingness to share. This result was unexpected and suggests that emotional self-disclosure may reduce user openness during early interactions, possibly because it feels unnatural or too personal too soon.

These findings show that emotional expression is not always the best approach. Instead, it is important to match the chatbot’s disclosure style to the situation and the user's comfort level, especially in sensitive areas like mental health support.

Mental health chatbots are increasingly adopted to address shortage mental health services, by offering non-judgmental, always-available support. User self-disclosure is a critical factor which allows mental health chatbots to better understand users and provide more therapeutic experiences. Although prior work has explored how factors such as chatbot modality and tone affect self-disclosure, the role of privacy policies and how question sensitivity affects disclosure remains under examined. In this study, we investigate how privacy policies and the sensitivity of questions in voice-based mental health chatbots impacts user self-disclosure. Through a controlled user study, we explore whether the presence of a privacy policy leads to increased self-disclosure, whether question sensitivity influences self-disclosure willingness and whether there is any interaction effect between these two factors. Preliminary findings indicate that while providing a privacy policy did not significantly impact users' privacy understanding or willingness to self-disclose, question sensitivity notably influenced disclosure. Specifically, participants were more willing to disclose to low and medium sensitivity questions compared to high sensitivity. No interaction effect between the privacy policy and the question sensitivity was observed. Future research should expand participant pools, investigate self-disclosure in free-form interactions, and explore alternative methods of communicating privacy information for deeper insights into user perceptions regarding privacy, sensitivity and disclosure.

AI-powered mental health chatbots offer scalable and accessible support, but their effectiveness hinges on users’ willingness to self-disclose—an outcome shaped by chatbot communication style and the sensitivity of the topic. While prior work has explored empathy and rapport, the role of conversational anthropomorphism remains underexamined, particularly in relation to question sensitivity as a potential moderator. This study addresses that gap through a mixed-design experiment (n = 30) in which participants interacted with either an anthropomorphic or a neutral chatbot and rated their willingness to respond to questions varying in sensitivity. Although no effects reached statistical significance, descriptive trends suggest that anthropomorphic cues—such as informal tone, emojis, and adaptive responses—may increase willingness to disclose, while higher question sensitivity slightly reduces it. No significant interaction effect was found, but anthropomorphic language appeared to promote disclosure regardless of sensitivity level. These findings offer tentative support for the use of calibrated human-like design in mental health chatbots. Future work should incorporate open-ended interactions, behavioral measures, and longitudinal designs to better capture disclosure dynamics and trust formation.

This study investigates whether empathetic language in chatbot interactions influences users’ willingness to disclose mental health-related information. Using a two-by-two mixed factorial design, 114 participants were assigned to either an empathetic or neutral chatbot condition and responded to both emotional and behavioural health questions. While prior research suggests that empathy can pormote trust and openness, results from this study revealed no significant difference in disclosure willingness across chatbot styles or question types, although the manipulation check showed a well perceived empathy. The study highlights the importance of individual predispositions, such as prior readiness to disclose, in shaping interactions with digital mental health tools. Future work should explore longer-term interactions and real disclosure behaviour to better understand the role of empathy in chatbot design.

Explainable Artificial Intelligence (XAI) has the potential to enhance user understanding and trust in AI systems, especially in domains where interpretability is crucial, such as cricket training. This study investigates the impact of different explanation formats on user experience within a cricket-specific context. Two prototypes were developed, each including four explanation formats: textual, visual, rule-based, and mixed. The second prototype introduced interactive features to examine their influence on user experience and explanation effectiveness. A small-scale user study evaluated the explanations based on satisfaction and trust. Results show that rule-based explanations were significantly less preferred in terms of satisfaction than the other explanation formats. Furthermore, the addition of interactive features led to a significant increase in user trust, though they did not enhance satisfaction levels. These findings highlight the importance of selecting appropriate explanation formats and the potential of interactive features to enhance trust in AI-generated explanations in a cricket-specific context.

Recent advancements in pose estimation, activity classification, and explainable artificial intelligence (XAI) have opened new opportunities in sports analytics. However, their combined application within the domain of cricket remains largely unexplored. This paper investigates the integration of XAI methods to interpret black-box models trained to classify cricket shot techniques using pose keypoints extracted from video data. I implement and compare several techniques, including SHAP, Grouped SHAP, Feature Importance, Permutation Importance, LIME, Grad-CAM, and Accumulated Local Effects (ALE) plots based on their ability to generate meaningful and interpretable explanations. Through a structured experimental pipeline involving MediaPipe-based keypoint extraction, random forest (RF), convolutional neural networks (CNNs), and multiple explanation methods, I evaluate which techniques most effectively highlight the body keypoints critical to accurate shot classification. The findings indicate that SHAP outperforms other methods due to its ability to generate both local and global explanations, along with intuitive visualizations. This work contributes to the development of transparent sports AI systems and lays the foundation for future applications of interpretable machine learning in athletic coaching and skill assessment.

Classification of cricket shots is a recent field of study that has seen some growth. The addition of Human Pose Estimation (HPE) has the potential to advance the study of cricket shot classification. This paper investigates which of the available and widely use HPE frameworks are most suitable for for the domain of cricket and introduces a hand annotated verification dataset in order to test this. The findings indicate that any increase in precision will come at a steep computational cost but a middle ground can be found depending on the use-case.

As technology advances, people are increasingly exposed to vast amounts of information. When they browse through the information, their perspectives on certain topics—particularly controversial ones—can gradually shift, ultimately influencing their life decisions. These shifts can have profound societal implications, making it essential to examine the factors that shape people's opinions. In recent years, some research has explored how users form opinions on debated topics through web searches. However, these studies have been limited by their focus on traditional information presentation formats (such as Search Engine Results Pages) and the ignorance of other presentation modes (like podcasts). This paper aims to investigate an innovative presentation format: AI-generated podcasts. Using the Audio Overview feature of NotebookLM, we created podcasts based on SERP sources and generated summaries using the same AI model as a control. To comprehensively analyze how different presentation formats affect opinion changes on controversial topics, we recruited participants from Prolific and conducted a 3×3 between-subjects study. In this study, participants were exposed to content under varying conditions (different presentation medium and viewpoint bias). The results show that presentation modes have no significant differences in influencing attitude change, with all modes aligning in direction with the viewpoint bias. While attitude change varies significantly between different kinds of debated topics, the degree of controversiality does not affect how presentation modes influence these changes.

While LLMs have absorbed unprecedented computational investment in 2025, our interactions remain trapped in the text box—a sequential, linear dialogue that mirrors decades-old chat paradigms. To break free from these constraints, I worked alongside Decathlon's AI Innovation & Trust Team, deploying human-centred design methodologies and creative coding to prototype radical new interaction models. These interfaces transform overwhelming product catalogs into navigable, intuitive experiences that feel more like discovery than search. To validate these new paradigms, I conducted controlled comparative studies between non-linear LLM interfaces and traditional chat interfaces, collecting both behavioural metrics and nuanced qualitative insights. The data reveals critical tensions: between human reliance on spatial memory and the need to express ideas in natural language; between chat as an intuitive affordance and its misalignment with underlying AI functionality; between unclear system boundaries and usability demands for clear interaction limits. Building on these findings, the project culminates in a design framework with detailed guidelines for the post-conversational era of human-AI interaction.

Reasoning over large-scale knowledge graphs has long been dominated by embedding-based methods, which focus on representing entities and relationships in vector spaces to perform inference tasks. Despite advancements in knowledge graph completion (KGC), challenges such as data sparsity and the lack of interpretability persist. These issues are critical in biomedical areas, where prediction accuracy and explainability directly impact decision-making.

In light of these limitations, this paper proposes a KGC approach that allows it to leverage both globally structured patterns and locally analogical information. Aiming to enhance the reasoning capability and completeness of knowledge graphs, we develop a better joint inference approach based on the rules mining and knowledge graph embedding. Specifically, our approach employs knowledge graph embedding techniques to discover analogically similar triples, which are then used to construct more relevant and accurate inference paths. Subsequently, rule mining is integrated to extract structured knowledge patterns through analogical reasoning.

Experimental results demonstrate that our approach performs better than the traditional Knowledge Graph Embedding Model in some specified KGC tasks, which depend highly on the inner construction of the dataset and its relation types. The multi-hop nature of DRKG aligns well with rule-based approaches, where learned rules generalize over multiple instances, aiding in missing link prediction. Beyond improved performance, our method enhances explainability, facilitating transparent inference and enabling backtracking of key prediction results—an important feature for biomedical and high-stakes AI applications.

Crowdwork offers flexibility but often lacks the structure needed to sustain motivation and goal progress. This thesis evaluates a guided progress monitoring protocol layered on a shared SMART onboarding for self-set goals in an autonomous crowdwork setting. We implemented a chatbot that supports SMART goal-setting and twice-weekly check-ins over two weeks, combining progress monitoring, brief reflection, and plan updates.

In a two-arm randomized experiment with 44 participants, guided progress monitoring produced higher short-term self-reported goal attainment than onboarding alone. The advantage appeared across the goal domains represented in this sample. Secondary outcomes such as commitment, perceived performance, and intrinsic motivation remained largely unchanged over two weeks, consistent with a performance-phase effect focused on execution rather than immediate motivational shifts.

The study contributes experimental evidence and a concrete chatbot design for guided progress monitoring in crowdwork. Findings suggest next steps focused on durability over longer windows using behavioral metrics, isolating the active components, and type-specific variations to customize support for habits in contrast to project-oriented goals.

The integration of renewable energy sources has fundamentally altered the operating environment of transmission system operators (TSOs). While essential for achieving a sustainable and low-carbon energy system, their volatility introduces significant uncertainty and volatility into the power grid. For operators in control rooms, this has led to more frequent congestion events, narrower safety margins, and rising information demands across multiple fragmented systems. In this context, timely and effective decision-making becomes increasingly challenging.

The first AI-based decision support tools (DSTs) have been deployed in TSO control rooms, for example, the GridOptions tool at TenneT TSO. These DSTs remain in the assistance mode of decision support by providing context and recommendations to the human operator in a one-directional fashion. However, timely and effective decision-making under uncertainty requires bi-directional human-AI communication, feedback, and co-learning. Consequently, this study investigates how AI-based DSTs can move from an assistance mode to joint AI-human decision making.

By employing novel concepts like the Supportive AI Framework and the Joint Control Framework, this study examines how human-AI teaming can evolve across different decision-making contexts and how interfaces can dynamically adapt to situational demands in both time-critical and less urgent scenarios. In particular, human cognitive needs figure prominently in how adaptable AI-powered interfaces can support operators in maintaining grid stability under uncertainty.

Drawing from observation, collaborative interaction patterns were developed to describe how human-AI teamwork can evolve across congestion operation timeframes. These patterns reveal opportunities and challenges in dynamically allocating initiative between humans and AI, maintaining situational awareness, and safeguarding human agency in safety-critical contexts.

Ultimately, this research seeks to contribute to the design of adaptive and supportive DSTs that unify data, reduce cognitive load, and facilitate reflection and learning. By addressing the dual challenge of information overload and uncertainty, the work aims to enhance the resilience of grid control strategies, stimulating effective human-AI collaboration, and enable the continued integration of renewable energy sources into the power system.

As AI technologies gain widespread acceptance across society, human-AI collaboration has emerged as a promising avenue to enhance the accountability and reliability of task outcomes where AI is used in task completion. Although AI systems are advancing rapidly, most people in society – particularly laypeople – still lack sufficient understanding and experience in collaborating with them. This gap becomes a barrier when interacting with deep learning-based AI systems, where users often struggle to assess the trustworthiness of AI advice. Consequently, individuals may develop uncalibrated trust or misperceptions about AI capabilities, hindering appropriate reliance and degrading overall team performance. Empirical studies have shown that human-AI teams often underperform compared to AI systems operating alone, highlighting that current human-AI collaboration remains suboptimal. These observations underscore a substantial need to advance our understanding of fostering effective human-AI collaboration.

This dissertation contributes to the growing literature on human-AI collaboration by analyzing potential approaches to promoting appropriate reliance. Specifically, to ensure effective human-AI collaboration, we aim to achieve both reliable task outcomes and a positive, engaging user experience. Through a series of empirical studies, we explored promoting appropriate reliance by calibrating user perception of competence (Part I), improving user understanding of AI systems with human-centered explainable AI (Part II), and enhancing user control with collaborative workflows (Part III). Our findings confirm that an uncalibrated perception of AI competence and self-competence can be a cause to trigger over-reliance and under-reliance, respectively. Additionally, we observed that both XAI methods (e.g., analogy-based explanation) and interactive XAI interfaces (e.g., conversational XAI interfaces) may induce an illusion of explanatory depth, which can trigger over-reliance. Finally, our analysis of fine-grained reliance patterns within multi-step decision workflows, as well as user involvement in plan-then-execute LLM agents, offer valuable insights for designing effective collaborations with agentic AI systems.

Taken together, the findings and implications in this dissertation advance our understanding of how to foster appropriate reliance on AI systems. By examining human and contextual factors that shape user reliance and perception, and by proposing novel methods for explanation and interaction, this work contributes both theoretical insights and quantitative evidence to the design of human-centered AI systems. We hope the key findings and implications reported in this dissertation will inspire further research on promoting appropriate reliance and facilitating effective human-AI collaboration.

User feedback plays a significant role in helping recommendation systems to make personalized and accurate predictions. Despite the fact that many methods of collecting user feedback have been proposed, little research exists that addresses both the breadth and depth of data collected. In this study, we incorporate personal reflection into traditional crowdsourcing tasks and investigate how it facilitates people's reflection on their usage of YouTube. We present a novel crowdsourcing approach with personal reflection integration based on several design principles, which allows participants to reflect on contextual factors and personal values of using YouTube through guided context recall and evaluations, therefore gathering deeper insights into people's preferences and behaviors on a large scale. We conducted a user study involving 20 participants and explored the insights generated from their textual answers and the role of design principles in the reflection process. This approach successfully enables multiple participants to conduct the study simultaneously, thereby reflecting on their watching behaviors and preferences. Based on the quantitative and qualitative analysis of the findings, we sum up the implications of this approach to provide guidance for the YouTube recommendation system and point to the directions for the design of similar studies in the future.

In the realm of software development, commit messages are vital for understanding code changes, enhancing maintainability, and improving collaboration. Despite their importance, generating high-quality commit messages remains a challenging task, with existing methods often facing issues such as limited flexibility and high training costs. This paper addresses the research gaps in automated commit message generation (CMG) by exploring the capabilities of large language models (LLMs) in this domain. We specifically investigate the potential of the prompt engineering method to enhance LLM performance compared to state-of-the-art (SOTA) techniques such as RACE.

Our research begins with a comprehensive literature review of CMG methodologies, focusing on the effectiveness of various message formats and the limitations of existing approaches. To fill the gap, we introduce a unified commit message formats dataset and evaluate the previous LLM-based method on the dataset, utilizing the GPT model as a representative example of LLMs. By comparing the LLM zero-shot method with previous retrieval-based and hybrid methods, we provide a detailed analysis of the strengths and weaknesses of LLM-based approaches.

We further explore the impact of different retrieval augmented generation (RAG) configurations on CMG performance and investigate what constitutes a good demonstration of the LLM RAG method for the CMG task. That is followed by proposing a new prompt engineering method called Adaptive Retrieval Augmented Generation With Commit Type Classification And Partitioned Retrieval (ARC-PR), which incorporates a classification module and a database partitioning module to the LLM RAG system. Validating through comprehensive testing on the unified message format dataset, our experiments demonstrate that the proposed method shows significant improvements in message effectiveness compared to the previous LLM-based methods and in the aspects of informativeness, message format consistency and the balance between precision and recall, our method surpasses the state-of-the-art methods in the field. Further generalizability study illustrates the robustness of our proposed method. With the introduction of human evaluation, we further confirmed the superiority of our proposed method over the state-of-the-art methods in terms of informativeness and expressiveness.

In summary, this study makes several key contributions: it provides a thorough comparison of previous LLM-based methods with existing techniques, proposes an enhanced LLM prompt engineering approach specifically tailored for commit message generation (CMG) tasks that address the issue of low informativeness and expressiveness seen in past state-of-the-art methods and demonstrates performance that surpasses other LLM-based methods.

The emergence of conversational AI systems like ChatGPT and Microsoft Copilot has impacted how users engage in information retrieval.
Retrieval Augmented Generation (RAG) harnesses the potential of Large Language Models (LLMs) with unstructured data, creating opportunities in science and business.
RAG-based models have gained popularity, but their effectiveness and user reliance in organizational settings call for exploration. This thesis involves a user study with policy experts in the financial domain.
They were tasked with text aggregation using a basic RAG model. The study delves into the model’s performance and the temporal development of user reliance among the experts over four weeks.
Our key findings reveal that outputs assisted by RAG do not match the quality produced by human experts.
The RAG model, however, excels in specific aspects such as structure, spelling, and grammar.
Additionally, the experts express satisfaction with the efficiency of RAG. Our findings suggest that user reliance on RAG increases with experience.
This underscores the need for interventions and policies to support responsible human-AI collaboration.
This work represents an effort to measure the temporal aspects of user reliance within an RAG system.
Simultaneously, it assesses the system’s efficacy in a field study with policy experts in the financial domain.