Understanding mental resilience-how individuals recover from stressors-is a critical focus area for mental health practitioners. To support this, the Experience Sampling Method (ESM) can be utilized, which offers rich, real-time data for tracking emotional dynamics. However, visualizing these data in an informative and actionable way remains a challenge. This study presents a set of visualizations designed to depict emotional recovery trajectories by focusing on mood, emotions, types of stressors and the influence of coping activities. To assess whether the developed visuals are interpretable and useful to mental health professionals, a survey-based evaluation was conducted. Afterwards, thematic analysis was applied in order to analyze and present the results. What was highlighted in the results is that easy-to-interpret visuals, such as line graphs and heatmaps, were more likely to be considered intuitive and clinically relevant. Moreover, visualizations that incorporated contextual details, like specific emotional responses or coping strategies, were regarded as more insightful and valuable for guiding therapy.

Smoking cessation remains a persistent public health challenge, with digital interventions increasingly adopted to support behavior change. This study explores how smokers plan and engage with preparatory activities suggested by conversational agents in online cessation programs. It addresses three main questions: the factors that influence engagement with these activities, the ability of large language models to identify smokers’ articulated plans, and the role of conditional "if-then" formulations in expressing coping strategies. A thematic analysis was conducted on qualitative user responses using both manual coding and automated labeling by local large language models. The findings show that smokers create context-sensitive plans shaped by emotional states, routines, and perceived usefulness of the suggestions, often expressed through conditional intentions. While manual analysis produced consistent and detailed themes, the large language model showed low agreement, highlighting limitations in current AI driven qualitative analysis. These results inform the design of more adaptive digital cessation tools and contribute to the understanding of AI’s role in supporting thematic research.

The use of Artificial Social Agents (ASAs) is rapidly expanding across society. As these agents become more integrated into our interactions, understanding the user experience of them becomes increasingly necessary to ensure their design aligns with user needs, promotes trust, and supports meaningful engagement. This study aims to investigate how users experience interactions with ASAs, focusing on using thematic analysis to identify recurring themes in user-reported experiences with ASAs. In addition, it also explores the reliability of locally hosted Large Language Models (LLMs) in identifying those experiences. We conducted a manual -peer validated- thematic analysis, resulting in a total of 31 themes. Afterwards, we conducted two experiments with LLMs, namely giving hem an unguided prompt (i.e. the LLM discovers and groups themes independently) and a guided prompt (i.e. the LLM matches predefined themes to responses) and measured their agreements with the manual analysis both intuitively and analytically. From our findings, it became clear that users experience ASAs through a balance of practical utility and emotional engagement. Themes covering the agent's helpfulness, sociability, enjoyability and perceived intelligence played a central role in shaping user experience. Most users responded positively to ASAs that felt intuitive, responsive, and human-like, though perceptions of human-likeness varied, sometimes enhancing the experience and other times creating discomfort. Our evaluation of LLMs showed that while they are capable of uncovering broad thematic patterns through unguided analysis, they fall short when tasked with consistently identifying and labeling predefined themes at the individual response level. This suggests that current LLMs, while useful as supplementary tools, are not yet reliable replacements for human-led thematic analysis in capturing the full nuance of user experiences at a detailed level. The conclusions reinforce the continued value and need of human-led thematic analysis, particularly when aiming to capture subtle, context-dependent insights that automated models may overlook.

Smoking remains a leading cause of preventable death, making effective cessation support a global health priority. While conversational agents (chatbots) offer a scalable solution, their success depends on understanding the user’s experience. This study addresses two interconnected challenges: first, understanding the subjective experience of smokers with preparatory activities proposed by a chatbot, and second, evaluating the efficacy of Large Language Models (LLMs) in analyzing this qualitative feedback. This research employs a comparative design. A manual thematic analysis of smokers’ written reflections first established a baseline coding scheme. This scheme was then compared against the outputs of three LLMs, which were tasked with both generating themes independently and applying the predefined manual scheme. The accuracy of the LLMs’ application was measured against the human baseline using Cohen’s Kappa. The manual analysis revealed that smokers’ experiences were predominantly positive, showing strong motivation and a sense that the activities helped reinforce their quitting goals. This was concurrently challenged by expressions of skepticism about the activities’ effectiveness and mentions of personal barriers to quitting. The comparative analysis demonstrated that while LLMs could identify these broad positive and negative topics, they failed to capture more subtle, attitude-based concepts, such as a user’s willingness to engage with an activity despite their personal doubts. Furthermore, the models’ accuracy in applying a predefined coding scheme was substantially lower than the human baseline. This work makes two primary contributions. For digital health, the findings show that cessation aids must be designed to personalize activities to address specific user barriers and skepticism. Methodologically, the study provides a clear verdict on the current role of LLMs in this context: while LLMs show potential as an exploratory aid in theme generation, they are not yet a viable tool for applying a predefined coding scheme, making human analytical oversight essential for ensuring the depth and validity of qualitative research.

Smoking cessation interventions sometimes involve the use of both technology and human support to increase effectiveness. Nevertheless, little is known about user preferences for allocating human support and whether large language models (LLMs) can support qualitative analysis to better understand these preferences. This study analyzes how smokers’ ethical perspectives shape their preferences for time allocation mechanisms in online smoking cessation programs, while also evaluating how can LLMs support this analysis.

We conducted a deductive thematic analysis of open-ended responses from users who completed a questionnaire after participating in a smoking cessation program with a virtual coach. Additionally, we employed the LLaMA large language model to identify patterns in the responses and to assign the ethical themes discovered during the analysis.

The findings indicate that some users valued fairness and preferred scheduled, randomized interventions or no feedback. Others emphasized autonomy, wanting users to request feedback themselves. Some suggested prioritizing motivated or advanced users. A common view was that interventions should focus on those in need, users at risk of disengagement or health problems, those making little progress, experiencing emotional difficulties, or lacking clarity. The large language model was successful in identifying themes but not in accurately allocating themes, reflected by a low Cohen’s Kappa of 0.05.

The results presenting user preferences can guide the design of interventions that need to be effective and ethically sound. The findings suggest that while large language models can identify themes, they are not yet suitable for allocating those themes.

Digital smoking cessation tools increasingly use chatbots to recommend preparatory activities intended to support behavior change. However, little is known about how users actually experience these activities and whether large language models (LLMs) can support the qualitative analysis of such experiences. This study explores how smokers and vapers described their experiences with chatbot-suggested activities in an online intervention and investigates whether an LLM can assist with analyzing their responses.

We conducted a reflexive thematic analysis on 650 filtered user responses and examined whether a local version of LLaMA 3.3 (8B) could support the generation of common patterns and themes from qualitative data, and the categorization of such data into predefined themes. Our findings show that participants had mixed experiences. Some felt encouraged and supported by the activities, while others found them confusing, irrelevant to smoking cessation, or difficult to complete due to contextual or environmental barriers.

While the model was able to generate reasonable themes that overlapped with human interpretations, it was unable to reliably label data points using a predefined coding schema, achieving a Cohen’s Kappa of just 0.003. These findings suggest that LLMs may be useful in the early, exploratory stages of qualitative analysis, but currently lack the accuracy needed for theme application in complex, nuanced data.

Child helplines provide a platform for children facing serious challenges, allowing them to share their stories and receive emotional support and guidance through counselling sessions. To ensure that volunteers are well prepared for these interactions, effective training programmes are essential. Recently, a BDI-based conversational agent was developed to train counsellors through role-play simulations, with the agent taking on the role of the child. However, this simulation training does not provide guidance to the trainees that caused a decrease in the trainees’ self-efficacy. In this research, we aim to enhance the effectiveness of counselling training through role-play simulation by integrating a pedagogical agent. We integrated an adaptive pedagogical agent by applying the scaffolding technique, where we taught a set of skills divided into three modules. The pedagogical agent also provided feedback and hints as additional guidance methods. We evaluated this design through a mixed study involving 22 participants, comparing an intervention group trained with the pedagogical agent to a control group using a standard training approach. We measured the participants’ performance, self-efficacy, and perceived usefulness of the system. While the intervention group showed higher mean scores across all measures compared to the control group, the differences were not statistically significant, indicating a possible underpowered experiment. This study contributes to the integration of pedagogical agents in simulation training systems for child helplines by proposing a framework that combines scaffolding, adaptivity, and structured learning.

Child helpline counsellors require various skills and strategies to achieve lasting change in children who require assistance. Typical training methods such as role-play are resource intensive, leading to the development of computer simulation-based training systems where learners counsel the computer which assumes the role of a child requiring assistance. Such systems are limited in their understanding and responses, causing them to appear unrealistic and repetitive. In this paper, we built upon one such rule-based agent through the integration of Large Language Models (LLMs) to vastly expand both the understanding and responses of the agent. We conducted a within-subject experiment with 37 participants who we recruited online through Prolific, where they interacted with both systems, assuming the role of a counsellor. Our results indicate that participants find the integrated system to be human-like in its behaviour, have a more positive attitude towards it, and have a better impression of their overall experience with it. Our thematic analysis revealed that the integrated system felt more adaptive, and engaging, and allowed them to focus more on applying the conversational strategy, while the rule-based system felt scripted and boring. Our work provides an integrated system for effectively training child helpline counsellors and a method by which LLMs and rule-based systems can be integrated in general.

This thesis investigates how Reinforcement Learning (RL) can increase support effectiveness in virtual coach-based smoking cessation interventions. Such interventions have shown promise in helping people change behaviors such as smoking. However, personalizing the support they provide by accounting for people's current and future states might further increase their effectiveness. States thereby refer to people's relatively stable conditions at certain moments in time, capturing aspects such as motivation, knowledge, or the presence of personal reminders. After deriving general user needs for the support provided by a virtual coach-based smoking cessation intervention from a study with 671 daily smokers, we thus used RL to adapt the support to people's current and future states. Specifically, using data collected from three crowdsourcing studies with each more than 500 participants, we assessed the effectiveness of different RL model components in adapting 1) how people are persuaded, 2) what they are asked to do, and 3) who they are supported by. Our findings suggest that considering current and future states increases the effort smokers spend on smoking cessation activities and helps them build quitting-related competencies over time. Given that model components were derived from behavior change theories, this shows the potential of using psychology-informed RL to create smoking cessation support that is effective in the long run.

This study analyzed the self-introductions of participants in a smoking and vaping cessation program to understand their motivations, barriers, and support needs. Through thematic analysis of 787 participant introductions, six main themes were identified: Motivations for Quitting, Previous Attempts to Quit, Barriers to Quitting, Desired Support, Usage Patterns, and Identity. Health concerns emerged as the most common motivation for quitting, while psychological and social challenges were the primary barriers. The study found no significant correlation between the length of introductions and participant engagement or satisfaction, suggesting that while introductions provide valuable qualitative insights, they do not relate to other factors. The findings describe the contents of introductions to human coaches, although the direct impact on outcomes requires further exploration. Future research should employ a controlled design to evaluate the effectiveness of detailed participant introductions in enhancing eHealth support and consider the complexity of dual usage in smoking and vaping cessation efforts.

Background. Quitting smoking is a challenge nowadays. Virtual coaches offer autonomous, personalized guidance for smoking cessation. However, such systems cannot replace human coaches completely. In situations, when human coaches cannot provide help to everyone - a virtual coach could follow a set of ethical principles to decide on who should get the feedback from a human.
Objective. Our study aims to identify users’ preferences on ethical principles that a virtual coach should follow to decide when to allocate human feedback to individuals preparing to quit smoking.
Methods. Our research was based on pre-gathered data, that included participants’ responses to open and closed questions regarding feedback allocation principles. Thematic analysis was conducted on these responses. Triangulation was performed using a qualitative literature review and quantitative data analysis.
Results. Four main themes were identified: (1) Struggling the Most (63.75%), (2) Increasing Chances of Success the Most (13.75%), (3) Equal Treatment (11.25%), and (4) Appreciating the Most (11.25%). Participants prioritized support for those experiencing the greatest difficulty in smoking cessation. The triangulation supported the validity of these themes.
Conclusions. Our study highlights the importance of integrating user-preferred ethical principles in virtual coaching systems for smoking cessation. Prioritization of users who struggle the most can increase the effectiveness and fairness of such systems, potentially increasing success rates. Future research should explore additional ethical principles, combining several principles into systems, and real-world application of these findings to further refine virtual coaching in healthcare.

Smoking and vaping cessation remains a significant public health challenge despite the availability of numerous aids and eHealth applications. This study explores the reasons behind users' preference for human feedback when preparing to quit smoking or vaping, aiming to address a gap in existing literature on the integration of human elements in eHealth platforms. The research involved 479 participants interacting with a virtual coach, with some receiving human feedback. We conducted a thematic analysis of two open-text questions with 265 responses each from post-questionnaires, and key themes such as emotional connection, personalized advice, effectiveness, motivation, and accountability were identified. Through using quantitative data and previously published research, these findings were further explained. The results from the quantitative analysis show that incorporating human elements in eHealth applications can enhance smoking cessation support. This research provides insights into the main reasons how the human support in eHealth applications should be designed. Key recommendations include designing human feedback to offer empathy and validation, tailoring feedback to individual needs, incorporating interactive elements to maintain engagement, providing constant encouragement, and establishing accountability mechanisms.

Smoking remains one of the largest health concerns worldwide, which is why eHealth applications with virtual coaches have been developed to assist smokers with quitting. Providing additional feedback from human coaches during such smoking cessation programs can further improve the effectiveness of the intervention. However, due to budgetary constraints and the limited availability of human coaches, it is important to make informed decisions about when someone gets human support to optimize the effectiveness. This research investigates the use of reinforcement learning (RL) to determine when to provide human feedback in quitting smoking with a virtual coach. Using data from a longitudinal study, we implemented an RL model that decides when to involve a human coach based on users' appreciation for human support and their self-efficacy, optimizing the effort that people spend on preparatory activities and their likelihood of returning to the program. Results show that the model is effective in allocating human support, increasing users' effort and return likelihood while considering the cost of human coaches. These findings support using RL to help with determining when to provide human support in smoking cessation programs.

Smoking has been one of the great threats to health in recent years, being strongly correlated with multiple negative health consequences, including lung cancer. Recent research suggests that artificial intelligence chatbots can be effective in persuading healthy behavior change. However, these chatbots usually rely on persuasive techniques to achieve their goal. Such techniques depend on identifying and meeting users' needs to be effective. To help improve understanding of the domain of healthy behavior change, we proposed a study which analysed the needs of daily smokers as they emerged from their interactions with a chatbot specifically designed to help them quit. The study performed a thematic analysis on users' free-text feedback, from which a set of 8 themes that directly correspond to 8 different needs were observed. The user needs were correlated with their genders, ages and highest completed education levels. While most of the results indicate that there is no significant correlation between the needs and user characteristics, which suggests that user needs are evenly distributed, certain correlations were highlighted for further analysis.

To assist smokers in potentially quitting their habit, this paper investigates digital eHealth applications. Based on a dataset provided by research into an eHealth application, it aims to determine if persuasive activities can convince users of the usefulness of competencies determined to assist in quitting smoking. A thematic analysis was used on the participant's qualitative responses to the activities. Using this and the quantitative data derived, insights into the efficacy of persuasion were determined.
It was determined that engagement with the optional qualitative aspect of the data produced similar utility perspectives on the competencies to those who did not comment. It was noted that the general perspective of the competencies rose after completing the activity, however not to a significant degree. Additionally, no notable correlations between age, gender or educational level and increased perception of the competency arose. Several interesting remarks from participants were analysed to offer considerations for any future research in this field.

Unhealthy behaviors such as smoking is the major cause for premature deaths and changing behaviors by oneself can be difficult. That is where eHealth applications come into rescue. One of the recent research explored the possibility of using a Reinforcement Learning model to choose persuasive types for a virtual coach to adopt to persuade people to prepare for smoke-quitting and it has shown advantages. However, there are still more aspects to investigate in this context except the persuasive types of the messages, and this paper intended to further look into using reinforcement learning to choose activities for preparing the users to quit smoking, To be more specific, we implemented and evaluated a reinforcement learning model to choose activities to optimize both the effort spent by the users and also the likelihood of them staying for the next session. The result suggests that reinforcement learning is a promising approach to choose activities for people to prepare for quitting smoking and it can move the users to states that they are more likely to spend a good effort on the activities and are more likely to come back to the next session.

Child helplines play a crucial role in delivering expert assistance to young clients facing challenges and seeking support. While counselling is instrumental in enhancing children’s mental well-being, the limited number of experienced counsellors is inadequate given the substantial workload. At this point, effective training to volunteer counsellors becomes essential. This thesis aims to expand upon a Belief-Desire-Intention (BDI) based chatbot, the simulated child, with the specific objective of enhancing volunteer training at child helpline organizations through the integration of emotional capabilities. Through the collaboration with professionals from a Dutch child helpline, the Kindertelefoon, we identified the predominant emotion of frustration along with its underlying triggers. Armed with this insight, we proposed an Emotion-BDI model capable of dynamically adjusting the chatbot’s displayed emotion in response to its environment. The evaluation of a prototype constructed based on this model supported our hypotheses that a chatbot equipped to express frustration possesses greater believability and emotional presence compared to the original version. Moreover, it also indicated heightened user enjoyability, engagement, and perceived usefulness. Within this thesis, we highlighted the enhanced feasibility of the simulated child brought about by our proposed Emotion-BDI model.

The aim of this research is to evaluate the believability of Lilobot, a conversational agent meant to act as a virtual child for training helpline workers. Numerous aspects of believability are explored by means of a user study involving a questionnaire and interview with 10 participants. Questionnaire results indicate that improvement to the chatbot's believability is likely necessary.
The findings from the interviews are that the use of emoticons and acknowledging the context of the application raise believability, while unresponsiveness and repeated utterances lower it. While Lilobot did express valid and real emotions, study participants suggested improving the appropriateness of its reactions and expanding its vocabulary.

Agent-based social skills training systems have been gaining attention for their potential to improve social skills development in various contexts. Through a rapid review methodology, data was collected from diverse sources, including company websites and research papers. This study then uses the collected data to categorize 8 commercial systems based on their agent model and feedback approaches, into two categorization tables. The findings reveal notable trends in the use of choice-based input, scenario-defined decision-making, and post-interaction feedback. Additionally, the paper discusses the limitations of these findings, highlights characteristics of commercial systems and compares them to research systems, as well as suggesting areas for future research. This study contributes to the understanding and advancement of agent-based social skills training systems, offering guidance to researchers in this field.

Helpline counselors can be trained faster using agent-based social skills training systems. These systems utilize conversational agents that simulate interactions with users and provide feedback. This research evaluates the noticeability of behavior changes in such a conversational agent. By examining participants' ability to detect and interpret these changes, valuable insights can be gained to enhance the agent's representation of real-life scenarios and improve its effectiveness as a training tool.

A mixed-methods approach was employed, combining quantitative data using descriptive statistics and qualitative data through content analysis. The quantitative analysis revealed a precision score of 93% and a recall score of 36%, indicating a higher accuracy in correctly identifying behavior changes compared to capturing all instances of change. The qualitative analysis revealed that participants' perceptions fell within four distinct categories: communication style, positive emotion, negative emotion, and tone & attitude. Participants demonstrated heightened attentiveness to changes in positive behavior, particularly instances of happiness expressed by the agent, and displayed less attention towards changes in negative behavior.

In conclusion, this research demonstrates that behavior changes in the conversational agent are noticeable, albeit with varying degrees of attention across different types of behavior. Further exploration with a larger sample size is warranted to ascertain the extent of noticeability. The diverse range of behavior changes indicates the agent's capability to adapt and exhibit visible shifts in behavior. These insights can inform the refinement and development of agent-based training systems, ultimately enhancing the training experiences of chat-based helpline counselors.