Integrating human support with chatbot-based behavior change interventions raises three challenges: (1) attuning the support to an individual’s state (e.g., motivation) for enhanced engagement, (2) limiting the use of the concerning human resources for enhanced efficiency, and (3) optimizing outcomes on ethical aspects (e.g., fairness). Therefore, we conducted a study in which 679 smokers and vapers had a 20% chance of receiving human feedback between five chatbot sessions. We find that having received feedback increases retention and effort spent on preparatory activities. However, analyzing a reinforcement learning (RL) model fit on the data shows there are also states where not providing feedback is better. Even this “standard” benefit-maximizing RL model is value-laden. It not only prioritizes people who would benefit most, but also those who are already doing well and want feedback. We show how four other ethical principles can be incorporated to favor other smoker subgroups, yet, interdependencies exist.

This thesis investigates how Reinforcement Learning (RL) can increase support effectiveness in virtual coach-based smoking cessation interventions. Such interventions have shown promise in helping people change behaviors such as smoking. However, personalizing the support they provide by accounting for people's current and future states might further increase their effectiveness. States thereby refer to people's relatively stable conditions at certain moments in time, capturing aspects such as motivation, knowledge, or the presence of personal reminders. After deriving general user needs for the support provided by a virtual coach-based smoking cessation intervention from a study with 671 daily smokers, we thus used RL to adapt the support to people's current and future states. Specifically, using data collected from three crowdsourcing studies with each more than 500 participants, we assessed the effectiveness of different RL model components in adapting 1) how people are persuaded, 2) what they are asked to do, and 3) who they are supported by. Our findings suggest that considering current and future states increases the effort smokers spend on smoking cessation activities and helps them build quitting-related competencies over time. Given that model components were derived from behavior change theories, this shows the potential of using psychology-informed RL to create smoking cessation support that is effective in the long run.

Health care is under pressure due to an aging population with an increasing prevalence of chronic diseases, including cardiovascular disease. Smoking and physical inactivity are 2 key preventable risk factors for cardiovascular disease. Yet, as with most health behaviors, they are difficult to change. In the interdisciplinary Perfect Fit project, scientists from different fields join forces to develop an evidence-based virtual coach (VC) that supports smokers in quitting smoking and increasing their physical activity. In this Viewpoint paper, intervention content, design, and implementation, as well as lessons learned, are presented to support other research groups working on similar projects. A total of 6 different approaches were used and combined to support the development of the Perfect Fit VC. The approaches used are (1) literature reviews, (2) empirical studies, (3) collaboration with end users, (4) content and technical development sprints, (5) interdisciplinary collaboration, and (6) iterative proof-of-concept implementation. The Perfect Fit intervention integrates evidence-based behavior change techniques with new techniques focused on identity change, big data science, sensor technology, and personalized real-time coaching. Intervention content of the virtual coaching matches the individual needs of the end users. Lessons learned include ways to optimally implement and tailor interactions with the VC (eg, clearly explain why the user is asked for input and tailor the timing and frequency of the intervention components). Concerning the development process, lessons learned include strategies for effective interdisciplinary collaboration and technical development (eg, finding a good balance between end users’ wishes and legal possibilities). The Perfect Fit development process was collaborative, iterative, and challenging at times. Our experiences and lessons learned can inspire and benefit others. Advanced, evidence-based digital interventions, such as Perfect Fit, can contribute to a healthy society while alleviating health care burden.

To increase the effectiveness of behavior change applications, a large variety of algorithms has been developed to adapt what the applications offer, when, how, and with whom. Given the multitude of challenges related to the concept of algorithmic behavior change support, its development, evaluation, and impact on behavior change, this workshop aims to strengthen the community of people with diverse backgrounds (e.g., computer science, psychology, human-computer interaction) and roles in behavior change support (e.g., researcher, designer, practitioner). Combining keynotes of leading researchers with sessions in which individual workshop participants present their work and discuss problems with the audience, the workshop encouraged a lively exchange of ideas that benefits current and future research on algorithmic behavior change support.

Goal-setting is commonly used in behavior change applications for physical activity. However, for goals to be effective, they need to be tailored to a user’s situation (e.g., motivation, progress). One way to obtain such goals is a collaborative process in which a healthcare professional and client set a goal together, thus making use of the professional’s expertise and the client’s knowledge about their own situation. As healthcare professionals are not always available, we created a dialog with the virtual coach Steph to collaboratively set daily step goals. Since judgments in human decision-making processes are adjusted based on the starting point or anchor, the first step goal proposal Steph makes is likely to influence the user’s final goal and self-efficacy. Situational factors impacting physical activity (e.g., motivation, self-efficacy, available time) or how users process information (e.g., mood) may determine which initial proposals are most effective in getting users to reach their underlying previous activity-based recommended step goals. Using data from 117 people interacting with Steph for up to five days, we designed a reinforcement learning algorithm that considers users’ current and future situations when choosing an initial step goal proposal. Our simulations show that initial step goal proposals matter: choosing optimal ones based on this algorithm could make it more likely that people move to a situation with high motivation, high self-efficacy, and a favorable daily context. Then, they are more likely to achieve, but also to overachieve, their underlying recommended step goals. Our dataset is publicly available.

Objective
Adopting healthy behavior is vital for preventing chronic diseases. Mobile health (mHealth) interventions utilizing virtual coaches (i.e., artificial intelligence conversational agents) can offer scalable and cost-effective solutions. Additionally, targeting multiple unhealthy behaviors, like low physical activity and smoking, simultaneously seems beneficial. We developed Perfect Fit, an mHealth intervention with a virtual coach providing personalized feedback to simultaneously promote smoking cessation and physical activity. Through innovative methods (e.g., sensor technology) and iterative development involving end-users, we strive to overcome challenges encountered by mHealth interventions, such as shortage of evidence-based interventions and insufficient personalization. This paper outlines the content of Perfect Fit and the protocol for evaluating its feasibility, acceptability, and preliminary effectiveness, the role of participant characteristics, and the study's feasibility.
Methods
A single-arm, mixed-method, real-world evaluation study will be conducted in the Netherlands. We aim to recruit 100 adult daily smokers intending to quit within 6 weeks. The personalized intervention will last approximately 16 weeks. Primary outcomes include Perfect Fit's feasibility and acceptability. Secondary outcomes are preliminary effectiveness and study feasibility, and we will measure participant characteristics. Quantitative data will be collected through questionnaires administered at baseline, post-intervention and 2, 6, and 12 months post-intervention. Qualitative data will be gathered via semi-structured interviews post-intervention. Data analysis will involve descriptive analyses, generalized linear mixed models (quantitative) and the Framework Approach (qualitative), integrating quantitative and qualitative data during interpretation.
Conclusions
This study will provide novel insight into the potential of interventions like Perfect Fit, as a multiple health behavior change strategy. Findings will inform further intervention development and help identify methods to foster feasibility and acceptability. Successful mHealth interventions with virtual coaches will prevent chronic diseases and promote public health.

Enabling the widespread utilization of the Artificial-Social-Agent (ASA) Questionnaire, a research instrument to comprehensively assess diverse ASA qualities while ensuring comparability, necessitates translations beyond the original English source language questionnaire. We thus present Dutch and German translations of the long and short versions of the ASA Questionnaire and describe the translation challenges we encountered. Summative assessments with 240 English-Dutch and 240 English-German bilingual participants show, on average, excellent correlations (Dutch ICC M = 0.82, SD = 0.07, range [0.58, 0.93]; German ICC M = 0.81, SD = 0.09, range [0.58, 0.94]) with the original long version on the construct and dimension level. Results for the short version show, on average, good correlations (Dutch ICC M = 0.65, SD = 0.12, range [0.39, 0.82]; German ICC M = 0.67, SD = 0.14, range [0.30, 0.91]). We hope these validated translations allow the Dutch and German-speaking populations to evaluate ASAs in their own language.

Despite their prevalence in eHealth applications for behavior change, persuasive messages tend to have small effects on behavior. Conditions or states (e.g., confidence, knowledge, motivation) and characteristics (e.g., gender, age, personality) of persuadees are two promising components for more effective algorithms for choosing persuasive messages. However, it is not yet sufficiently clear how well considering these components allows one to predict behavior after persuasive attempts, especially in the long run. Since collecting data for many algorithm components is costly and places a burden on users, a better understanding of the impact of individual components in practice is welcome. This can help to make an informed decision on which components to use. We thus conducted a longitudinal study in which a virtual coach persuaded 671 daily smokers to do preparatory activities for quitting smoking and becoming more physically active, such as envisioning one’s desired future self. Based on the collected data, we designed a Reinforcement Learning (RL)-approach that considers current and future states to maximize the effort people spend on their activities. Using this RL-approach, we found, based on leave-one-out cross-validation, that considering states helps to predict both behavior and future states. User characteristics and especially involvement in the activities, on the other hand, only help to predict behavior if used in combination with states rather than alone. We see these results as supporting the use of states and involvement in persuasion algorithms. Our dataset is available online.

This document is an encore abstract of the paper “Persuading to Prepare for Quitting Smoking with a Virtual Coach: Using States and User Characteristics to Predict Behavior” presented at AAMAS 2023.

Virtual coaches have the potential to address the low adherence common to eHealth applications for behavior change by, for example, providing motivational support. However, given the multitude of factors affecting users’ attitudes toward virtual coaches, more insights are needed on how such virtual coaches can be designed to affect these attitudes in a specific use context positively. Especially valuable are insights that are based on users interacting with such a virtual coach for longer. We thus conducted a study in which more than 500 smokers interacted with the text-based virtual coach Sam in five sessions. In each session, Sam assigned smokers a new preparatory activity for quitting smoking and provided motivational support for doing the activity. Based on a mixed-methods analysis of users’ willingness to continue working and their relationship with Sam, we obtained eight themes for users’ attitudes toward Sam. These themes relate to whether Sam is seen as human or artificial, specific characteristics of Sam (e.g., caring character), the interaction with Sam, and the relationship with Sam. We used these themes to formulate literature-based recommendations to guide designers of virtual coaches for behavior change. For example, letting the virtual coach get to know users and disclose more information about itself may improve its relationship with users.

Goal-setting is often used in eHealth applications for behavior change as it motivates and helps to stay focused on a desired outcome. However, for goals to be effective, they need to meet criteria such as being specific, measurable, attainable, relevant and time-bound (SMART). Moreover, people need to be confident to reach their goal. We thus created a goal-setting dialog in which the virtual coach Jody guided people in setting SMART goals. Thereby, Jody provided personalized vicarious experiences by showing examples from other people who reached a goal to increase people’s confidence. These experiences were personalized, as it is helpful to observe a relatable other succeed. Data from an online study with a between-subjects with pre-post measurement design (n=39 participants) provide credible support that personalized experiences are seen as more motivating than generic ones. Motivational factors for participants included information about the goal, path to the goal, and the person who accomplished a goal, as well as the mere fact that a goal was reached. Participants also had a positive attitude toward Jody. We see these results as an indication that people are positive toward using a goal-setting dialog with a virtual coach in eHealth applications for behavior change. Moreover, contrary to hypothesized, our observed data give credible support that participants’ self-efficacy was lower after the dialog than before. These results warrant further research on how such dialogs affect self-efficacy, especially whether these lower post-measurements of self-efficacy are associated with people’s more realistic assessment of their abilities.

Tailored motivational messages are helpful to motivate people in eHealth applications for increasing physical activity, but it is not sufficiently clear how such messages can be effectively generated in advance. We, therefore, put forward a theory-driven approach to generating tailored motivational messages for eHealth applications for behavior change, and we examine its feasibility by assessing how motivating the resulting messages are perceived. For this, we designed motivational messages with a specific structure that was based on an adaptation of an existing ontology for tailoring motivational messages in the context of physical activity. To obtain tailored messages, experts in health psychology and coaching successfully wrote messages with this structure for personas in scenarios that differed with regard to the persona’s mood, self-efficacy, and progress. Based on an experiment in which 60 participants each rated the perceived motivational impact of six generic and six tailored messages based on scenarios, we found credible support for our hypothesis that messages tailored to mood, self-efficacy, and progress are perceived as more motivating. A thematic analysis of people’s free-text responses about what they found motivating and demotivating about motivational messages further supports the use of tailored messages, as well as messages that are encouraging and empathetic, give feedback about people’s progress, and mention the benefits of physical activity. To aid future work on motivational messages, we make our motivational messages and corresponding scenarios publicly available.

Background: Smoking and physical inactivity are two key preventable risk factors of cardiovascular disease. Yet, as with most health behaviors, they are difficult to change. In the interdisciplinary Perfect Fit project, scientists from different fields join forces to develop an evidence-based virtual coach that supports smokers in quitting smoking and increasing their physical activity. Intervention content, design and implementation as well as lessons learnt are presented in the hopes of guiding other research groups working on similar projects. Methods: Numerous approaches were used and combined to support the development of the Perfect Fit virtual coach. Approaches include literature reviews, empirical studies, collaboration with end-users, content and technical development sprints, interdisciplinary collaboration and iterative proof-of-concept implementation. Findings: The Perfect Fit intervention integrates evidence-based behavioral change techniques as well as new techniques focused on identity change, big data science, sensor technology and personalized real-time coaching. Intervention content of the virtual coaching matches communication preferences and individual needs of end users. Lessons learnt include ways to optimally implement and tailor interactions from the virtual coach (e.g., ‘explain why user is asked for input’, ‘tailor timing and frequency of intervention components’). With regards to the development process, lessons learnt include strategies for effective interdisciplinary collaboration and technical development (e.g., ‘Find a good balance between wishes of end-users and legal possibilities’). Discussion: The Perfect Fit development process was interactive, iterative and challenging at times. We hope that our experiences and lessons learnt can inspire and benefit others.

Behavior change applications often assign their users activities such as tracking the number of smoked cigarettes or planning a running route. To help a user complete these activities, an application can persuade them in many ways. For example, it may help the user create a plan or mention the experience of peers. Intuitively, the application should thereby pick the message that is most likely to be motivating. In the simplest case, this could be the message that has been most effective in the past. However, one could consider several other elements in an algorithm to choose a message. Possible elements include the user’s current state (e.g., self-efficacy), the user’s future state after reading a message, and the user’s similarity to the users on which data has been gathered. To test the added value of subsequently incorporating these elements into an algorithm that selects persuasive messages, we conducted an experiment in which more than 500 people in four conditions interacted with a text-based virtual coach. The experiment consisted of five sessions, in each of which participants were suggested a preparatory activity for quitting smoking or increasing physical activity together with a persuasive message. Our findings suggest that adding more elements to the algorithm is effective, especially in later sessions and for people who thought the activities were useful. Moreover, while we found some support for transferring knowledge between the two activity types, there was rather low agreement between the optimal policies computed separately for the two activity types. This suggests limited policy generalizability between activities for quitting smoking and those for increasing physical activity. We see our results as supporting the idea of constructing more complex persuasion algorithms. Our dataset on 2,366 persuasive messages sent to 671 people is published together with this article for researchers to build on our algorithm.

Background
Despite their increasing prevalence and potential, eHealth applications for behavior change suffer from a lack of adherence and from dropout. Advances in virtual coach technology provide new opportunities to improve this. However, these applications still do not always offer what people need. We, therefore, need a better understanding of people’s needs and how to address these, based on both actual experiences of users and their reflections on envisioned scenarios.

Methods
We conducted a longitudinal study in which 671 smokers interacted with a virtual coach in five sessions. The virtual coach assigned them a new preparatory activity for quitting smoking or increasing physical activity in each session. Participants provided feedback on the activity in the next session. After the five sessions, participants were asked to describe barriers and motivators for doing their activities. In addition, they provided their views on videos of scenarios such as receiving motivational messages. To understand users’ needs, we took a mixed-methods approach. This approach triangulated findings from qualitative data, quantitative data, and the literature.

Results
We identified 14 main themes that describe people’s views of their current and future behaviors concerning an eHealth application. These themes relate to the behaviors themselves, the users, other parties involved in a behavior, and the environment. The most prevalent theme was the perceived usefulness of behaviors, especially whether they were informative, helpful, motivating, or encouraging. The timing and intensity of behaviors also mattered. With regards to the users, their perceived importance of and motivation to change, autonomy, and personal characteristics were major themes. Another important role was played by other parties that may be involved in a behavior, such as general practitioners or virtual coaches. Here, the themes of companionableness, accountability, and nature of the other party (i.e., human vs AI) were relevant. The last set of main themes was related to the environment in which a behavior is performed. Prevalent themes were the availability of sufficient time, the presence of prompts and triggers, support from one’s social environment, and the diversity of other environmental factors. We provide recommendations for addressing each theme.

Conclusions
The integrated method of experience-based and envisioning-based needs acquisition with a triangulate analysis provided a comprehensive needs classification (empirically and theoretically grounded). We expect that our themes and recommendations for addressing them will be helpful for designing applications for health behavior change that meet people’s needs. Designers should especially focus on the perceived usefulness of application components. To aid future work, we publish our dataset with user characteristics and 5,074 free-text responses from 671 people.