To equip new counsellors at a Dutch child helpline with the needed counselling skills, the helpline uses role-playing, a form of learning through simulation in which one counsellor-in-training portrays a child seeking help and the other portrays a counsellor. However, this process is time-intensive and logistically challenging-issues that a conversational agent could help address. In this paper, we propose an initial design for a computer agent that acts as a child help-seeker to be used in a role-play setting. Our agent, Lilobot, is based on a Belief-Desire-Intention (BDI) model to simulate the reasoning process of a child who is being bullied at school. Through interaction with Lilobot, counsellors-in-training can practise the Five Phase Model, a conversation strategy that underpins the helpline’s counselling principle of keeping conversations child-centred. We compared a training session with Lilobot to a text-based training, inviting experienced counsellors from the Dutch child helpline to participate in both sessions. We conducted pre- and post-measurement comparisons for both training sessions. Contrary to our expectations, the results show a decrease in counselling self-efficacy at post-measurement, particularly in Lilobot’s condition. Still, the counsellors’ qualitative feedback indicated that, with further development and refinements, they believed Lilobot could potentially serve as a useful supplementary tool for training new helpline counsellors. Our work also highlights three future research directions for training simulators in this domain: integrating emotions into the model, providing guided feedback to the counsellor, and incorporating Large Language Models (LLMs) into the conversations.

Validating claims and replicating findings on the impact of artificial social agents (ASA), such as virtual agents, conversational agents, and social robots, requires a standardised measurement instrument that researchers can employ in different settings and for various agents. Such an instrument would allow researchers to evaluate their agents and establish insights beyond their specific study context. Therefore, we present the long and short versions of the ASA questionnaire (ASAQ) for evaluating human-ASA interaction on 19 constructs, such as the agent's believability, sociability, and coherence. It has been developed by an international workgroup with more than 100 ASA-researchers over multiple years who identified community-relevant constructs and associated questionnaire items and examined the questionnaire's reliability, validity, and interpretability. The result is a questionnaire that can capture more than 80% of the constructs that studies in the intelligent virtual agent community investigate, with acceptable levels of reliability, content validity, construct validity, and cross-validity. We suggest that ASA-researchers use the ASAQ short version to report their agent's psychographic information and the ASAQ long version to analyse any constructs in-depth that are specifically relevant to their agent or study. Finally, this paper gives instructions for practical use, such as sample size estimations, and how to interpret and present results.

Integrating human support with chatbot-based behavior change interventions raises three challenges: (1) attuning the support to an individual’s state (e.g., motivation) for enhanced engagement, (2) limiting the use of the concerning human resources for enhanced efficiency, and (3) optimizing outcomes on ethical aspects (e.g., fairness). Therefore, we conducted a study in which 679 smokers and vapers had a 20% chance of receiving human feedback between five chatbot sessions. We find that having received feedback increases retention and effort spent on preparatory activities. However, analyzing a reinforcement learning (RL) model fit on the data shows there are also states where not providing feedback is better. Even this “standard” benefit-maximizing RL model is value-laden. It not only prioritizes people who would benefit most, but also those who are already doing well and want feedback. We show how four other ethical principles can be incorporated to favor other smoker subgroups, yet, interdependencies exist.

Child helplines offer a safe and private space for children to share their thoughts and feelings with volunteers. However, training these volunteers to help can be both expensive and time-consuming. In this demo, we present Lilobot, a conversational agent designed to train volunteers for child helplines. Lilobot’s reasoning is based on the Belief-Desire-Intention (BDI) model, which simulates, for example, a bullied child who contacts the helpline through text. Users engage with Lilobot in a role-play format, taking on the volunteer’s role. Through this system, volunteers can practice applying the Five Phase Model, a conversational strategy helplines use. The training tool includes a trainer interface for monitoring and modifying Lilobot’s interactions. Trainers can also create new conversational scenarios through an authoring tool. An initial evaluation led to enhancements in Lilobot’s knowledge base and intent recognition, addressing the main issues encountered by participants. The components used to implement the system were Java Spring for the BDI model and the authoring tool, Rasa for Natural Language Understanding, PostgreSQL for the database, and Vue.js for the front-end. This tool aims to provide volunteers with consistent, interactive training, enhancing their counselling skills in a controlled environment.

Background: Every month, around 3800 people complete an anonymous self-test for suicidal thoughts on the website of the Dutch suicide prevention helpline. Although 70% score high on the severity of suicidal thoughts, <10% navigate to the web page about contacting the helpline. Objective: This study aimed to test the effectiveness of a brief barrier reduction intervention (BRI) in motivating people with severe suicidal thoughts to contact the suicide prevention helpline, specifically in high-risk groups such as men and middle-aged people. Methods: We conducted a fully automated, web-based, randomized controlled trial. Respondents with severe suicidal thoughts and little motivation to contact the helpline were randomly allocated either to a brief BRI, in which they received a short, tailored message based on their self-reported barrier to the helpline (n=610), or a general advisory text (care as usual as the control group: n=612). Effectiveness was evaluated using both behavioral and attitudinal measurements. The primary outcome measure was the use of a direct link to contact the helpline after completing the intervention or control condition. Secondary outcomes were the self-reported likelihood of contacting the helpline and satisfaction with the received self-test. Results: In total, 2124 website visitors completed the Suicidal Ideation Attributes Scale and the demographic questions in the entry screening questionnaire. Among them, 1222 were randomized into the intervention or control group. Eventually, 772 respondents completed the randomized controlled trial (intervention group: n=369; control group: n=403). The most selected barrier in both groups was “I don’t think that my problems are serious enough.” At the end of the trial, 33.1% (n=122) of the respondents in the intervention group used the direct link to the helpline. This was not significantly different from the respondents in the control group (144/403, 35.7%; odds ratio 0.87, 95% CI 0.64‐1.18, P=.38). However, the respondents who received the BRI did score higher on their self-reported likelihood of contacting the helpline at a later point in time (B=0.22, 95% CI 0.12‐0.32, P≤.001) and on satisfaction with the self-test (B=0.27, 95% CI 0.01‐0.53, P=.04). For male and middle-aged respondents specifically, the results were comparable to that of the whole group. Conclusions: This trial was the first time the helpline was able to connect with high-risk website visitors who were hesitant to contact the helpline. Although the BRI could not ensure that those respondents immediately used the direct link to the helpline at the end of the trial, it is encouraging that respondents indicated that they were more likely to contact the helpline at a later point in time. In addition, this low-cost intervention provided greater insight into the perceived barriers to service. Follow-up research should be focused on identifying the added value of other components (eg, video or photo material) in the BRI and increasing its effectiveness, especially for men and middle-aged people.

Enabling the widespread utilization of the Artificial-Social-Agent (ASA) Questionnaire, a research instrument to comprehensively assess diverse ASA qualities while ensuring comparability, necessitates translations beyond the original English source language questionnaire. We thus present Dutch and German translations of the long and short versions of the ASA Questionnaire and describe the translation challenges we encountered. Summative assessments with 240 English-Dutch and 240 English-German bilingual participants show, on average, excellent correlations (Dutch ICC M = 0.82, SD = 0.07, range [0.58, 0.93]; German ICC M = 0.81, SD = 0.09, range [0.58, 0.94]) with the original long version on the construct and dimension level. Results for the short version show, on average, good correlations (Dutch ICC M = 0.65, SD = 0.12, range [0.39, 0.82]; German ICC M = 0.67, SD = 0.14, range [0.30, 0.91]). We hope these validated translations allow the Dutch and German-speaking populations to evaluate ASAs in their own language.

Goal-setting is commonly used in behavior change applications for physical activity. However, for goals to be effective, they need to be tailored to a user’s situation (e.g., motivation, progress). One way to obtain such goals is a collaborative process in which a healthcare professional and client set a goal together, thus making use of the professional’s expertise and the client’s knowledge about their own situation. As healthcare professionals are not always available, we created a dialog with the virtual coach Steph to collaboratively set daily step goals. Since judgments in human decision-making processes are adjusted based on the starting point or anchor, the first step goal proposal Steph makes is likely to influence the user’s final goal and self-efficacy. Situational factors impacting physical activity (e.g., motivation, self-efficacy, available time) or how users process information (e.g., mood) may determine which initial proposals are most effective in getting users to reach their underlying previous activity-based recommended step goals. Using data from 117 people interacting with Steph for up to five days, we designed a reinforcement learning algorithm that considers users’ current and future situations when choosing an initial step goal proposal. Our simulations show that initial step goal proposals matter: choosing optimal ones based on this algorithm could make it more likely that people move to a situation with high motivation, high self-efficacy, and a favorable daily context. Then, they are more likely to achieve, but also to overachieve, their underlying recommended step goals. Our dataset is publicly available.

To increase the effectiveness of behavior change applications, a large variety of algorithms has been developed to adapt what the applications offer, when, how, and with whom. Given the multitude of challenges related to the concept of algorithmic behavior change support, its development, evaluation, and impact on behavior change, this workshop aims to strengthen the community of people with diverse backgrounds (e.g., computer science, psychology, human-computer interaction) and roles in behavior change support (e.g., researcher, designer, practitioner). Combining keynotes of leading researchers with sessions in which individual workshop participants present their work and discuss problems with the audience, the workshop encouraged a lively exchange of ideas that benefits current and future research on algorithmic behavior change support.

In the published article, there was an error in Table 5. For each second construct/dimension, the means are swapped between Chinese and English data, which is caused by an error in the underlying R script. Consequently, the plus and minus signs for the delta and CI values are also wrong. The corrected Table 5 and its caption appear below. Construct/dimension rating difference between mixed-international English-speaking and Chinese mother-tongue groups. Δ Score are pairwise differences between Chinese and mother-tongue cultural background and mixed-international cultural background taken from the posterior distribution. M, mean; SD, standard deviation; CI, credible interval. The authors apologize for this error and state that this does not change the scientific conclusions of the article in any way. The original article has been updated.

Agent-based training systems can enhance people's social skills. The effective development of these systems needs a comprehensive architecture that outlines their components and relationships. Such an architecture can pinpoint improvement areas and future outlooks. This paper presents ARTES: a general architecture illustrating how components of agent-based social training systems work together. We studied existing systems and architectures for training and tutoring to design ARTES and identify its essential components and interaction characteristics. ARTES comprises two core components: the agent simulation of social situations, and educational elements to provide guided learning. We link ARTES's crucial components to four primary learning theories (behaviourism, cognitivism, social cognitive theory, and constructivism) to illustrate the role of agent simulation and tutoring elements in establishing desired learning outcomes. Furthermore, we map ARTES's components against eight architectures, 43 systems and three tools to indicate the components' relevance, completeness, generalisation, and deployment potential across contexts. In addition to ARTES, the paper also contributes by identifying future improvements and research directions, such as the agent's thinking, tutoring methods, knowledge transfer, and ethical implications. We believe ARTES can help bridge the gap between virtual human simulations and impactful educational learning, offering training system developers desirable features like understandability and adaptability.

Social interactions form an essential aspect of people’s life, however, it is quite challenging for individuals to handle a wide range of social situations. Therefore, a variety of training systems have been developed to improve their skills. This literature review seeks to give an overview of the state of the art of technology-supported systems for social skills training. The studies eligible for inclusion described a technology-supported system with the purpose of training social skills and included an experimental or observational study to evaluate the efficacy of the system. 225 studies (224 publications) with 216 systems were identified, characterized, and analyzed in this literature review. Using the taxonomy as put forward in this study, the analysis shows that the majority of these systems were screen-based applications, with virtual reality technology being the most frequently observed. The systems most often targeted communication skills that focus on transferring information to produce greater understanding, i.e. mending general communication impairments in children with autism. In terms of functions, support for learning-by-doing was the most observed function, while focusing on job interviews provided the largest number of functions. Finally, the studies reported overwhelmingly positively regarding the systems’ impact, including 76 studies with a randomized controlled trial design. Still, most studies only used a quasi-experimental design based on self-report measures. We anticipate the proposed taxonomy to be a starting point for researchers to position their work and that the review will help them with gaining inspiration for the design and evaluation of social skills training systems.

Counsellors at the child helpline offer a confidential environment for children to be heard and empowered. However, training counsellors on handling children’s conversations in text-based chat can be costly and time-consuming. This paper introduces Lilobot, a conversational agent designed for training counsellors of child helplines. The agent’s dialogue is built on the Belief-Desire-Intention (BDI) model, which, in this case, simulates a child victim of school bullying in a text based interaction. Trainees engage with Lilobot in a role-play format, taking on the counsellor’s role. This interactive system helps trainees learn the Five Phase Model, a conversation protocol child’s helplines use. The system also has a trainer interface, where a trainer can oversee and control Lilobot’s interactions, and see a suggested optimal conversational path. The system was built with three main components - a natural language processing model (using Rasa) and the BDI reasoning model and optimal path generation (using Java Spring).

Background: Globally, suicide is among the leading causes of death, with men being more at risk to die from suicide than women. Research suggests that people with suicidal ideation often struggle to find adequate help. Every month, around 4000 people fill in the anonymous self-test for suicidal thoughts on the website of the Dutch suicide prevention helpline. This self-test includes the Suicidal Ideation Attributes Scale (SIDAS), which educates users about the severity of their suicidal thoughts. The vast majority (70%) of people who complete the self-test score higher than the cutoff point (≥21) for severe suicidal thoughts. Unfortunately, despite this, less than 10% of test-takers navigate to the web page about contacting the helpline. Objective: This protocol presents the design of a web-based randomized controlled trial that aims to reduce barriers to contacting the suicide prevention helpline. The aim of this study is 2-fold: (1) to measure the effectiveness of a brief barrier reduction intervention (BRI) provided in the self-test motivating people with severe suicidal thoughts to contact the Dutch suicide prevention helpline and (2) to specifically evaluate the effectiveness of the BRI in increasing service use by high-risk groups for suicide such as men and middle-aged people. Methods: People visiting the self-test for suicidal thoughts on the website of the suicide prevention helpline will be asked to participate in a study to improve the self-test. Individuals with severe suicidal thoughts and little motivation to contact the helpline will be randomly allocated either to a brief BRI, in which they will receive a short tailored message based on their self-reported barrier to the helpline (n=388) or care as usual (general advisory text, n=388). The primary outcome measure is the use of a direct link to contact the helpline after receiving the intervention or control condition. Secondary outcomes are the self-reported likelihood of contacting the helpline (on a 5-point scale) and satisfaction with the self-test. In the BRI, participants receive tailored information to address underlying concerns and misconceptions of barriers to the helpline. A pilot study was conducted among current test-takers to identify these specific barriers. Results: The pilot study (N=1083) revealed multiple barriers to contacting the helpline. The most prominent were the belief that a conversation with a counselor would not be effective, fear of the conversation itself, and emotional concerns about talking about suicidal thoughts. Conclusions: Our study will provide insight into the effectiveness of a brief BRI designed to increase the use of a suicide prevention helpline provided in a self-test on suicidal thoughts. If successful, this intervention has the potential to be a low-cost, easily scalable, and feasible method to increase service use for helplines across the world.

Goal-setting is often used in eHealth applications for behavior change as it motivates and helps to stay focused on a desired outcome. However, for goals to be effective, they need to meet criteria such as being specific, measurable, attainable, relevant and time-bound (SMART). Moreover, people need to be confident to reach their goal. We thus created a goal-setting dialog in which the virtual coach Jody guided people in setting SMART goals. Thereby, Jody provided personalized vicarious experiences by showing examples from other people who reached a goal to increase people’s confidence. These experiences were personalized, as it is helpful to observe a relatable other succeed. Data from an online study with a between-subjects with pre-post measurement design (n=39 participants) provide credible support that personalized experiences are seen as more motivating than generic ones. Motivational factors for participants included information about the goal, path to the goal, and the person who accomplished a goal, as well as the mere fact that a goal was reached. Participants also had a positive attitude toward Jody. We see these results as an indication that people are positive toward using a goal-setting dialog with a virtual coach in eHealth applications for behavior change. Moreover, contrary to hypothesized, our observed data give credible support that participants’ self-efficacy was lower after the dialog than before. These results warrant further research on how such dialogs affect self-efficacy, especially whether these lower post-measurements of self-efficacy are associated with people’s more realistic assessment of their abilities.

Virtual coaches have the potential to address the low adherence common to eHealth applications for behavior change by, for example, providing motivational support. However, given the multitude of factors affecting users’ attitudes toward virtual coaches, more insights are needed on how such virtual coaches can be designed to affect these attitudes in a specific use context positively. Especially valuable are insights that are based on users interacting with such a virtual coach for longer. We thus conducted a study in which more than 500 smokers interacted with the text-based virtual coach Sam in five sessions. In each session, Sam assigned smokers a new preparatory activity for quitting smoking and provided motivational support for doing the activity. Based on a mixed-methods analysis of users’ willingness to continue working and their relationship with Sam, we obtained eight themes for users’ attitudes toward Sam. These themes relate to whether Sam is seen as human or artificial, specific characteristics of Sam (e.g., caring character), the interaction with Sam, and the relationship with Sam. We used these themes to formulate literature-based recommendations to guide designers of virtual coaches for behavior change. For example, letting the virtual coach get to know users and disclose more information about itself may improve its relationship with users.

Tailored motivational messages are helpful to motivate people in eHealth applications for increasing physical activity, but it is not sufficiently clear how such messages can be effectively generated in advance. We, therefore, put forward a theory-driven approach to generating tailored motivational messages for eHealth applications for behavior change, and we examine its feasibility by assessing how motivating the resulting messages are perceived. For this, we designed motivational messages with a specific structure that was based on an adaptation of an existing ontology for tailoring motivational messages in the context of physical activity. To obtain tailored messages, experts in health psychology and coaching successfully wrote messages with this structure for personas in scenarios that differed with regard to the persona’s mood, self-efficacy, and progress. Based on an experiment in which 60 participants each rated the perceived motivational impact of six generic and six tailored messages based on scenarios, we found credible support for our hypothesis that messages tailored to mood, self-efficacy, and progress are perceived as more motivating. A thematic analysis of people’s free-text responses about what they found motivating and demotivating about motivational messages further supports the use of tailored messages, as well as messages that are encouraging and empathetic, give feedback about people’s progress, and mention the benefits of physical activity. To aid future work on motivational messages, we make our motivational messages and corresponding scenarios publicly available.

The Artificial-Social-Agent (ASA) questionnaire is an instrument for evaluating human-ASA interaction. It consists of 19 constructs and related dimensions measured by either 24 questionnaire items (short version) or 90 questionnaire items (long version). The questionnaire was built and validated by a research community effort to make evaluation results more comparable between agents and findings more generalizable. The current questionnaire is in English, which limits its use to only a population with an adequate command of the English language. Translating the questionnaire into more languages allows for the inclusion of other populations and the possibility of comparing them. Therefore, this paper presents a Mandarin Chinese translation of the questionnaire. After three construction cycles that included forward and backward translation, we gave both the final version of the translated and original English questionnaire to 242 bilingual crowd-workers to evaluate 14 ASAs. Results show on average a good level of correlation on the construct/dimension level (ICC M = 0.79, SD = 0.09, range [0.61, 0.95]) and on the item level (ICC M = 0.62, SD = 0.14, range [0.19, 0.92]) between the two languages for the long version, and for the short version (ICC M = 0.66, SD = 0.12, range [0.41, 0.92]). The analysis also established correction values for converting questionnaire item scores between Chinese and English questionnaires. Moreover, we also found systematic differences in English questionnaire scores between the bilingual sample and a previously collected mixed-international English-speaking sample. We hope this and the Chinese questionnaire translation will motivate researchers to study human-ASA interaction among a Chinese literate population and to study cultural similarities and differences in this area.