Integrating human support with chatbot-based behavior change interventions raises three challenges: (1) attuning the support to an individual’s state (e.g., motivation) for enhanced engagement, (2) limiting the use of the concerning human resources for enhanced efficiency, and (3) optimizing outcomes on ethical aspects (e.g., fairness). Therefore, we conducted a study in which 679 smokers and vapers had a 20% chance of receiving human feedback between five chatbot sessions. We find that having received feedback increases retention and effort spent on preparatory activities. However, analyzing a reinforcement learning (RL) model fit on the data shows there are also states where not providing feedback is better. Even this “standard” benefit-maximizing RL model is value-laden. It not only prioritizes people who would benefit most, but also those who are already doing well and want feedback. We show how four other ethical principles can be incorporated to favor other smoker subgroups, yet, interdependencies exist. ... Read more

Background: Smoking and physical inactivity compromise health, especially in combination. Interventions to promote smoking cessation and increased physical activity (PA) often lack impact, especially in the long term. Digital future-self interventions (FSIs), which prompt individuals to imagine who they do and do not want to become (ie, their desired and undesired future selves), show promise in encouraging sustainable changes in both behaviors. However, knowledge of user experiences with digital FSIs is limited. A deeper understanding of these experiences could help optimize FSIs, enhancing their efficacy in supporting smoking cessation and increased PA sustainably. Objective: This study examined behavioral, cognitive, and affective experiences with digital FSIs focused on smoking, PA, or both. Potential differences in user experiences based on behavior (smoking vs PA), polarity (desired vs undesired future self), and modality (verbal vs visual description of future selves) were explored. Methods: Secondary analyses of quantitative and qualitative survey data from 3 studies using digital FSIs as a means to encourage smoking cessation or increase PA were conducted. In study 1, participants (N=144) thought about how it would be to complete the FSI. In studies 2 (N=447) and 3 (N=87), they completed an FSI. Each study highlighted different aspects of user experiences with FSIs, namely, behavioral (eg, time spent), cognitive (eg, mental effort exerted), or affective (eg, emotions) experiences. Quantitative and qualitative findings were integrated for a comprehensive interpretation. Results: Regarding behavioral experiences, participants completed future-self tasks promptly (mean 6.64, SD 8.30 minutes), spent less time completing the desired- versus undesired-future-self (P<.001; η _p ²=0.227) and verbal versus visual (P=.03; η _p ²=0.060; quantitative) tasks, and integrated the tasks into their lives (qualitative). Despite tasks being preparatory and not actively encouraging behavior change, multiple participants reported implementing changes in their smoking or PA (qualitative). Regarding cognitive experiences, moderate effort (mean 5.85/10, SD 2.56) was exerted on the tasks regardless of behavior (P=.69; η _p ²=0.002), modality (P=.45; η _p ²=0.004), or polarity (P=.69; η _p ²=0.002; quantitative). Experiences of task difficulty were inconsistent across studies, individuals, and tasks, although mental visualization and describing one’s future self using images were consistently reported as challenging (quantitative and qualitative). Future-self tasks were reported to prompt cognitive processes such as contemplating consequences of smoking and PA behavior (qualitative). Regarding affective experiences, desired- and undesired-future-self tasks elicited different emotions (P<.001; η _p ²=0.630; quantitative). Desired-future-self tasks were perceived as enjoyable and happiness inducing, whereas undesired-future-self tasks were perceived as confronting and unpleasant, evoking feelings of sadness, fear, and anger (quantitative and qualitative). Conclusions: Digital FSIs appeared to be a time-efficient, feasible, and acceptable way of strengthening identities as a means to encourage smoking cessation and PA. Findings support continued implementation of digital FSIs, although further research is required to optimize their operationalization. Avenues in that regard are proposed and discussed. ... Read more

This thesis investigates how Reinforcement Learning (RL) can increase support effectiveness in virtual coach-based smoking cessation interventions. Such interventions have shown promise in helping people change behaviors such as smoking. However, personalizing the support they provide by accounting for people's current and future states might further increase their effectiveness. States thereby refer to people's relatively stable conditions at certain moments in time, capturing aspects such as motivation, knowledge, or the presence of personal reminders. After deriving general user needs for the support provided by a virtual coach-based smoking cessation intervention from a study with 671 daily smokers, we thus used RL to adapt the support to people's current and future states. Specifically, using data collected from three crowdsourcing studies with each more than 500 participants, we assessed the effectiveness of different RL model components in adapting 1) how people are persuaded, 2) what they are asked to do, and 3) who they are supported by. Our findings suggest that considering current and future states increases the effort smokers spend on smoking cessation activities and helps them build quitting-related competencies over time. Given that model components were derived from behavior change theories, this shows the potential of using psychology-informed RL to create smoking cessation support that is effective in the long run. ... Read more

Background
Reaching personal goals typically requires building competencies (e.g., insights into personal strengths), but expert health professionals and non-expert clients often think differently about which competencies are needed. Just having a virtual coach advise activities for "expert-devised" competencies may not motivate clients to carry them out, while advising only "non-expert devised" activities may not result in all required competencies being built.

Methods
We integrated the client and health expert worldviews in our modeling method for informing the activity selection by a virtual coach: We created a pipeline to build a reinforcement learning model for proposing activities in the context of preparing for quitting smoking. This model considers smokers’ current and future levels for expert-devised competencies as well as their beliefs about the usefulness of different competencies when choosing activities. To train the model, we conducted a micro-randomized trial in which 542 smokers interacted with a virtual coach in five sessions spread over at least nine days and received a randomly chosen activity in each session. Using data from this study, we performed simulations to systematically assess the impact of the different model components on the competencies built by smokers. Moreover, we performed paired Bayesian t-tests to determine the effect of persuasive activities on smokers’ usefulness beliefs.

Results
Our simulations show that smokers’ current levels for the expert competencies and their usefulness beliefs are important to consider when building expert competencies. In fact, we saw improvements of up to 22% when considering current competencies, and an additional 13% when also accounting for usefulness beliefs. Furthermore, although we found credible evidence that persuasive activities changed smokers’ usefulness beliefs, the effects might be too small to contribute in an optimal strategy for building competencies.

Conclusion
The worldviews of both health experts and smokers are important to consider when proposing activities for preparing for quitting smoking. We have presented a reinforcement learning model that combines these worldviews, and we hope that our work can be an example of incorporating different worldviews in a reinforcement learning model for building competencies. Our code and dataset are publicly available. ... Read more

Goal-setting is commonly used in behavior change applications for physical activity. However, for goals to be effective, they need to be tailored to a user’s situation (e.g., motivation, progress). One way to obtain such goals is a collaborative process in which a healthcare professional and client set a goal together, thus making use of the professional’s expertise and the client’s knowledge about their own situation. As healthcare professionals are not always available, we created a dialog with the virtual coach Steph to collaboratively set daily step goals. Since judgments in human decision-making processes are adjusted based on the starting point or anchor, the first step goal proposal Steph makes is likely to influence the user’s final goal and self-efficacy. Situational factors impacting physical activity (e.g., motivation, self-efficacy, available time) or how users process information (e.g., mood) may determine which initial proposals are most effective in getting users to reach their underlying previous activity-based recommended step goals. Using data from 117 people interacting with Steph for up to five days, we designed a reinforcement learning algorithm that considers users’ current and future situations when choosing an initial step goal proposal. Our simulations show that initial step goal proposals matter: choosing optimal ones based on this algorithm could make it more likely that people move to a situation with high motivation, high self-efficacy, and a favorable daily context. Then, they are more likely to achieve, but also to overachieve, their underlying recommended step goals. Our dataset is publicly available. ... Read more

Health care is under pressure due to an aging population with an increasing prevalence of chronic diseases, including cardiovascular disease. Smoking and physical inactivity are 2 key preventable risk factors for cardiovascular disease. Yet, as with most health behaviors, they are difficult to change. In the interdisciplinary Perfect Fit project, scientists from different fields join forces to develop an evidence-based virtual coach (VC) that supports smokers in quitting smoking and increasing their physical activity. In this Viewpoint paper, intervention content, design, and implementation, as well as lessons learned, are presented to support other research groups working on similar projects. A total of 6 different approaches were used and combined to support the development of the Perfect Fit VC. The approaches used are (1) literature reviews, (2) empirical studies, (3) collaboration with end users, (4) content and technical development sprints, (5) interdisciplinary collaboration, and (6) iterative proof-of-concept implementation. The Perfect Fit intervention integrates evidence-based behavior change techniques with new techniques focused on identity change, big data science, sensor technology, and personalized real-time coaching. Intervention content of the virtual coaching matches the individual needs of the end users. Lessons learned include ways to optimally implement and tailor interactions with the VC (eg, clearly explain why the user is asked for input and tailor the timing and frequency of the intervention components). Concerning the development process, lessons learned include strategies for effective interdisciplinary collaboration and technical development (eg, finding a good balance between end users’ wishes and legal possibilities). The Perfect Fit development process was collaborative, iterative, and challenging at times. Our experiences and lessons learned can inspire and benefit others. Advanced, evidence-based digital interventions, such as Perfect Fit, can contribute to a healthy society while alleviating health care burden. ... Read more

Objective
Adopting healthy behavior is vital for preventing chronic diseases. Mobile health (mHealth) interventions utilizing virtual coaches (i.e., artificial intelligence conversational agents) can offer scalable and cost-effective solutions. Additionally, targeting multiple unhealthy behaviors, like low physical activity and smoking, simultaneously seems beneficial. We developed Perfect Fit, an mHealth intervention with a virtual coach providing personalized feedback to simultaneously promote smoking cessation and physical activity. Through innovative methods (e.g., sensor technology) and iterative development involving end-users, we strive to overcome challenges encountered by mHealth interventions, such as shortage of evidence-based interventions and insufficient personalization. This paper outlines the content of Perfect Fit and the protocol for evaluating its feasibility, acceptability, and preliminary effectiveness, the role of participant characteristics, and the study's feasibility.
Methods
A single-arm, mixed-method, real-world evaluation study will be conducted in the Netherlands. We aim to recruit 100 adult daily smokers intending to quit within 6 weeks. The personalized intervention will last approximately 16 weeks. Primary outcomes include Perfect Fit's feasibility and acceptability. Secondary outcomes are preliminary effectiveness and study feasibility, and we will measure participant characteristics. Quantitative data will be collected through questionnaires administered at baseline, post-intervention and 2, 6, and 12 months post-intervention. Qualitative data will be gathered via semi-structured interviews post-intervention. Data analysis will involve descriptive analyses, generalized linear mixed models (quantitative) and the Framework Approach (qualitative), integrating quantitative and qualitative data during interpretation.
Conclusions
This study will provide novel insight into the potential of interventions like Perfect Fit, as a multiple health behavior change strategy. Findings will inform further intervention development and help identify methods to foster feasibility and acceptability. Successful mHealth interventions with virtual coaches will prevent chronic diseases and promote public health. ... Read more

Enabling the widespread utilization of the Artificial-Social-Agent (ASA) Questionnaire, a research instrument to comprehensively assess diverse ASA qualities while ensuring comparability, necessitates translations beyond the original English source language questionnaire. We thus present Dutch and German translations of the long and short versions of the ASA Questionnaire and describe the translation challenges we encountered. Summative assessments with 240 English-Dutch and 240 English-German bilingual participants show, on average, excellent correlations (Dutch ICC M = 0.82, SD = 0.07, range [0.58, 0.93]; German ICC M = 0.81, SD = 0.09, range [0.58, 0.94]) with the original long version on the construct and dimension level. Results for the short version show, on average, good correlations (Dutch ICC M = 0.65, SD = 0.12, range [0.39, 0.82]; German ICC M = 0.67, SD = 0.14, range [0.30, 0.91]). We hope these validated translations allow the Dutch and German-speaking populations to evaluate ASAs in their own language. ... Read more

To increase the effectiveness of behavior change applications, a large variety of algorithms has been developed to adapt what the applications offer, when, how, and with whom. Given the multitude of challenges related to the concept of algorithmic behavior change support, its development, evaluation, and impact on behavior change, this workshop aims to strengthen the community of people with diverse backgrounds (e.g., computer science, psychology, human-computer interaction) and roles in behavior change support (e.g., researcher, designer, practitioner). Combining keynotes of leading researchers with sessions in which individual workshop participants present their work and discuss problems with the audience, the workshop encouraged a lively exchange of ideas that benefits current and future research on algorithmic behavior change support. ... Read more

Despite their prevalence in eHealth applications for behavior change, persuasive messages tend to have small effects on behavior. Conditions or states (e.g., confidence, knowledge, motivation) and characteristics (e.g., gender, age, personality) of persuadees are two promising components for more effective algorithms for choosing persuasive messages. However, it is not yet sufficiently clear how well considering these components allows one to predict behavior after persuasive attempts, especially in the long run. Since collecting data for many algorithm components is costly and places a burden on users, a better understanding of the impact of individual components in practice is welcome. This can help to make an informed decision on which components to use. We thus conducted a longitudinal study in which a virtual coach persuaded 671 daily smokers to do preparatory activities for quitting smoking and becoming more physically active, such as envisioning one’s desired future self. Based on the collected data, we designed a Reinforcement Learning (RL)-approach that considers current and future states to maximize the effort people spend on their activities. Using this RL-approach, we found, based on leave-one-out cross-validation, that considering states helps to predict both behavior and future states. User characteristics and especially involvement in the activities, on the other hand, only help to predict behavior if used in combination with states rather than alone. We see these results as supporting the use of states and involvement in persuasion algorithms. Our dataset is available online. ... Read more

Background: Smoking and physical inactivity are two key preventable risk factors of cardiovascular disease. Yet, as with most health behaviors, they are difficult to change. In the interdisciplinary Perfect Fit project, scientists from different fields join forces to develop an evidence-based virtual coach that supports smokers in quitting smoking and increasing their physical activity. Intervention content, design and implementation as well as lessons learnt are presented in the hopes of guiding other research groups working on similar projects. Methods: Numerous approaches were used and combined to support the development of the Perfect Fit virtual coach. Approaches include literature reviews, empirical studies, collaboration with end-users, content and technical development sprints, interdisciplinary collaboration and iterative proof-of-concept implementation. Findings: The Perfect Fit intervention integrates evidence-based behavioral change techniques as well as new techniques focused on identity change, big data science, sensor technology and personalized real-time coaching. Intervention content of the virtual coaching matches communication preferences and individual needs of end users. Lessons learnt include ways to optimally implement and tailor interactions from the virtual coach (e.g., ‘explain why user is asked for input’, ‘tailor timing and frequency of intervention components’). With regards to the development process, lessons learnt include strategies for effective interdisciplinary collaboration and technical development (e.g., ‘Find a good balance between wishes of end-users and legal possibilities’). Discussion: The Perfect Fit development process was interactive, iterative and challenging at times. We hope that our experiences and lessons learnt can inspire and benefit others. ... Read more

Virtual coaches have the potential to address the low adherence common to eHealth applications for behavior change by, for example, providing motivational support. However, given the multitude of factors affecting users’ attitudes toward virtual coaches, more insights are needed on how such virtual coaches can be designed to affect these attitudes in a specific use context positively. Especially valuable are insights that are based on users interacting with such a virtual coach for longer. We thus conducted a study in which more than 500 smokers interacted with the text-based virtual coach Sam in five sessions. In each session, Sam assigned smokers a new preparatory activity for quitting smoking and provided motivational support for doing the activity. Based on a mixed-methods analysis of users’ willingness to continue working and their relationship with Sam, we obtained eight themes for users’ attitudes toward Sam. These themes relate to whether Sam is seen as human or artificial, specific characteristics of Sam (e.g., caring character), the interaction with Sam, and the relationship with Sam. We used these themes to formulate literature-based recommendations to guide designers of virtual coaches for behavior change. For example, letting the virtual coach get to know users and disclose more information about itself may improve its relationship with users. ... Read more

This document is an encore abstract of the paper “Persuading to Prepare for Quitting Smoking with a Virtual Coach: Using States and User Characteristics to Predict Behavior” presented at AAMAS 2023. ... Read more

Tailored motivational messages are helpful to motivate people in eHealth applications for increasing physical activity, but it is not sufficiently clear how such messages can be effectively generated in advance. We, therefore, put forward a theory-driven approach to generating tailored motivational messages for eHealth applications for behavior change, and we examine its feasibility by assessing how motivating the resulting messages are perceived. For this, we designed motivational messages with a specific structure that was based on an adaptation of an existing ontology for tailoring motivational messages in the context of physical activity. To obtain tailored messages, experts in health psychology and coaching successfully wrote messages with this structure for personas in scenarios that differed with regard to the persona’s mood, self-efficacy, and progress. Based on an experiment in which 60 participants each rated the perceived motivational impact of six generic and six tailored messages based on scenarios, we found credible support for our hypothesis that messages tailored to mood, self-efficacy, and progress are perceived as more motivating. A thematic analysis of people’s free-text responses about what they found motivating and demotivating about motivational messages further supports the use of tailored messages, as well as messages that are encouraging and empathetic, give feedback about people’s progress, and mention the benefits of physical activity. To aid future work on motivational messages, we make our motivational messages and corresponding scenarios publicly available. ... Read more

Goal-setting is often used in eHealth applications for behavior change as it motivates and helps to stay focused on a desired outcome. However, for goals to be effective, they need to meet criteria such as being specific, measurable, attainable, relevant and time-bound (SMART). Moreover, people need to be confident to reach their goal. We thus created a goal-setting dialog in which the virtual coach Jody guided people in setting SMART goals. Thereby, Jody provided personalized vicarious experiences by showing examples from other people who reached a goal to increase people’s confidence. These experiences were personalized, as it is helpful to observe a relatable other succeed. Data from an online study with a between-subjects with pre-post measurement design (n=39 participants) provide credible support that personalized experiences are seen as more motivating than generic ones. Motivational factors for participants included information about the goal, path to the goal, and the person who accomplished a goal, as well as the mere fact that a goal was reached. Participants also had a positive attitude toward Jody. We see these results as an indication that people are positive toward using a goal-setting dialog with a virtual coach in eHealth applications for behavior change. Moreover, contrary to hypothesized, our observed data give credible support that participants’ self-efficacy was lower after the dialog than before. These results warrant further research on how such dialogs affect self-efficacy, especially whether these lower post-measurements of self-efficacy are associated with people’s more realistic assessment of their abilities. ... Read more

Smoking tobacco and physical inactivity are key preventable behavioural risk factors of cardiovascular disease (CVD). Computerised coaching systems can help individuals to modify risky behaviours, thereby preventing CVD. However, most reported eHealth or computerized coaching systems are hard to reuse in slightly different settings. To provide an open-source, reusable computer coaching system, we developed Perfect Fit. The reusability is manifested by building around the open-source text- and voice-based contextual assistant framework Rasa. Rasa provides a simple, standard interface to many popular messaging and voice channels, and custom connectors are easily implemented. A set of algorithms have been developed and connected to Rasa to drive and personalize the conversation flow and the coaching process. Such algorithms make use of data stored in a devoted database. Furthermore, Perfect Fit adheres to best practices and standards in software engineering. The modular design of Perfect Fit will allow researchers to connect the virtual coach to any messaging or voice channel with only modest modification. Perfect Fit is available under open-source license in GitHub and is currently in prototype-phase. Concluding, Perfect Fit will deliver a virtual coach that can easily be adapted and reused in different settings. The coach helps individuals to achieve and maintain abstinence from smoking and sufficient physical activity (PA). ... Read more