Child helplines provide a platform for children facing serious challenges, allowing them to share their stories and receive emotional support and guidance through counselling sessions. To ensure that volunteers are well prepared for these interactions, effective training programmes are essential. Recently, a BDI-based conversational agent was developed to train counsellors through role-play simulations, with the agent taking on the role of the child. However, this simulation training does not provide guidance to the trainees that caused a decrease in the trainees’ self-efficacy. In this research, we aim to enhance the effectiveness of counselling training through role-play simulation by integrating a pedagogical agent. We integrated an adaptive pedagogical agent by applying the scaffolding technique, where we taught a set of skills divided into three modules. The pedagogical agent also provided feedback and hints as additional guidance methods. We evaluated this design through a mixed study involving 22 participants, comparing an intervention group trained with the pedagogical agent to a control group using a standard training approach. We measured the participants’ performance, self-efficacy, and perceived usefulness of the system. While the intervention group showed higher mean scores across all measures compared to the control group, the differences were not statistically significant, indicating a possible underpowered experiment. This study contributes to the integration of pedagogical agents in simulation training systems for child helplines by proposing a framework that combines scaffolding, adaptivity, and structured learning. ... Read more

Child helpline counsellors require various skills and strategies to achieve lasting change in children who require assistance. Typical training methods such as role-play are resource intensive, leading to the development of computer simulation-based training systems where learners counsel the computer which assumes the role of a child requiring assistance. Such systems are limited in their understanding and responses, causing them to appear unrealistic and repetitive. In this paper, we built upon one such rule-based agent through the integration of Large Language Models (LLMs) to vastly expand both the understanding and responses of the agent. We conducted a within-subject experiment with 37 participants who we recruited online through Prolific, where they interacted with both systems, assuming the role of a counsellor. Our results indicate that participants find the integrated system to be human-like in its behaviour, have a more positive attitude towards it, and have a better impression of their overall experience with it. Our thematic analysis revealed that the integrated system felt more adaptive, and engaging, and allowed them to focus more on applying the conversational strategy, while the rule-based system felt scripted and boring. Our work provides an integrated system for effectively training child helpline counsellors and a method by which LLMs and rule-based systems can be integrated in general. ... Read more

Scenario-based learning uses interactive scenarios to present the user with situations that need user input to be resolved in order to teach the user the correct behaviour. Traditionally these scenarios would be presented in a rigid order that is linearly increasing in difficulty. Every student however has a different learning rate. Studies have shown that students can lose motivation when challenged too much or too little. This project aims to improve learning efficiency and user satisfaction by adapting the scenario content to the user's skill and knowledge level.
A driving instructor application was developed where users are presented with short interactive scenarios in a 3D environment where they take control of the vehicle using a steering wheel and pedals and learn to make correct decisions when driving. For the research experiment the sessions were divided into two different modes, linear or adaptive. In the linear mode there was a strict gradual increase in difficulty. In the adaptive mode, the user's performance in the previous scenarios determined which scenario is presented next. The collected on user performance and self-efficacy showed no significant difference in user performance. There was however a significant increasement in self-efficacy when users played the adaptive session. This validates the positive value that personalization could bring to training applications. ... Read more

Helpline counselors can be trained faster using agent-based social skills training systems. These systems utilize conversational agents that simulate interactions with users and provide feedback. This research evaluates the noticeability of behavior changes in such a conversational agent. By examining participants' ability to detect and interpret these changes, valuable insights can be gained to enhance the agent's representation of real-life scenarios and improve its effectiveness as a training tool.

A mixed-methods approach was employed, combining quantitative data using descriptive statistics and qualitative data through content analysis. The quantitative analysis revealed a precision score of 93% and a recall score of 36%, indicating a higher accuracy in correctly identifying behavior changes compared to capturing all instances of change. The qualitative analysis revealed that participants' perceptions fell within four distinct categories: communication style, positive emotion, negative emotion, and tone & attitude. Participants demonstrated heightened attentiveness to changes in positive behavior, particularly instances of happiness expressed by the agent, and displayed less attention towards changes in negative behavior.

In conclusion, this research demonstrates that behavior changes in the conversational agent are noticeable, albeit with varying degrees of attention across different types of behavior. Further exploration with a larger sample size is warranted to ascertain the extent of noticeability. The diverse range of behavior changes indicates the agent's capability to adapt and exhibit visible shifts in behavior. These insights can inform the refinement and development of agent-based training systems, ultimately enhancing the training experiences of chat-based helpline counselors. ... Read more

The aim of this research is to evaluate the believability of Lilobot, a conversational agent meant to act as a virtual child for training helpline workers. Numerous aspects of believability are explored by means of a user study involving a questionnaire and interview with 10 participants. Questionnaire results indicate that improvement to the chatbot's believability is likely necessary.
The findings from the interviews are that the use of emoticons and acknowledging the context of the application raise believability, while unresponsiveness and repeated utterances lower it. While Lilobot did express valid and real emotions, study participants suggested improving the appropriateness of its reactions and expanding its vocabulary. ... Read more

Chatbots are tools that can potentially be utilized in chat-based child helpline training. In this type of training, the quality of the feedback received is of vital importance. This paper aims to analyze the automated feedback generated by such a chatbot. The domains analyzed include user comprehension, usefulness, and potential improvement points. In a user study, a formative assessment and two interviews were conducted for each domain, respectively.
For comprehensibility, all participants could easily understand the feedback report. They found that the bot could be easier to work around after reading the feedback, with the table being of much guidance. They found the transcript to be a welcome addition, but missing constructive feedback. Regarding improvement points, two of them were tightly related to the limitations of the chatbot, rather than the report itself. Extra guidance and instructions were deemed necessary by the participants, alongside an easier-to-read transcript interface. ... Read more

Agent-based social skills training systems have been gaining attention for their potential to improve social skills development in various contexts. Through a rapid review methodology, data was collected from diverse sources, including company websites and research papers. This study then uses the collected data to categorize 8 commercial systems based on their agent model and feedback approaches, into two categorization tables. The findings reveal notable trends in the use of choice-based input, scenario-defined decision-making, and post-interaction feedback. Additionally, the paper discusses the limitations of these findings, highlights characteristics of commercial systems and compares them to research systems, as well as suggesting areas for future research. This study contributes to the understanding and advancement of agent-based social skills training systems, offering guidance to researchers in this field. ... Read more

De Kindertelefoon is a children's helpline aimed at providing (pre-)adolescents with a person to talk to for a variety of subjects such as bullying, sex, and abuse. These people who talk for De Kindertelefoon need proper training and guidance. Among the tools that De Kindertelefoon provides is the 5-phase model, a conversational model. Researchers set up a simulation of a virtual child and De Kindertelefoon to help teach the 5-phase model. However, simulation alone was not enough, and we wanted to see if a feedback system could improve the results, with immediate feedback being the focus.
Immediate feedback is when the user fills in an answer and the system immediately provides information to strengthen knowledge.
When applied to an academic environment, both immediate feedback and student engagement have been proven to be important for completing a task.

Using a literature review, we found that immediate feedback can be directly linked to educational techniques such as self-improvement, self-efficacy, and the Self-Determination Theory. Through a focus group, we also found that constructive immediate feedback is an important pillar of De Kindertelefoon.

The design was achieved by looking at the existing and limited 5-phase model as a graph and trying to find an optimal path through that graph.

After conducting a within-subjects study experiment with 34 participants, the results were inconclusive, with neither condition appearing to be more useful for the group of participants, nor either condition being better at teaching them the 5-phase model.
As the results were inconclusive, the data was explored more by looking at it as a between-subject study, which showed that the explanation sheet might perform better for knowledge.

The research shows the possible strength of feedback in a practical manner. From the results, the immediate feedback is neither more nor less resonant than a classical approach to teaching the 5-phase model. ... Read more