Controlled Yet Natural: A Hybrid BDI-LLM Conversational Agent for Child Helpline Training Conference: Preprint version. Accepted at IVA ’25; Preprint version. Accepted at IVA ’25; CCS: Applied computing Interactive learning environments CCS: Human-centered computing Empirical studies in HCI CCS: Com

Mohammed Al Owayyed, Adarsh Denga, Willem-Paul Brinkman · 2025

Read Controlled Yet Natural: A Hybrid BDI-LLM Conversational Agent for Child Helpline Training Conference: Preprint version. Accepted at IVA ’25; Preprint version. Accepted at IVA ’25; CCS: Applied computing Interactive learning environments CCS: Human-centered computing Empirical studies in HCI CCS: Com by Mohammed Al Owayyed, Adarsh Denga, Willem-Paul Brinkman (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2509-16784.txt.

Introduction

Mohammed Al Owayyed, Adarsh Denga, and Willem-Paul Brinkman build a training agent for child helpline staff in a 2025 paper. New counselors often train through roleplay with people, which costs time and staff. Rule-based agents give structure but can sound stiff. The team blend rules with a language model for more natural chat.

The blend keeps talks on track while letting replies flow. In tests, trainees rated the agent as more controlled yet natural. Some support for higher engagement also appeared, though it was weaker. Such agents could let helplines train more staff at lower cost. Control matters here, since the talks mimic sensitive calls.

Catalog record

CREATORMohammed Al Owayyed, Adarsh Denga, Willem-Paul Brinkman
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE42 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2509-16784

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Controlled Yet Natural: A Hybrid BDI-LLM Conversational Agent for Child Helpline Training Conference: Preprint version. Accepted at IVA ’25; Preprint version. Accepted at IVA ’25; CCS: Applied computing Interactive learning environments CCS: Human-centered computing Empirical studies in HCI CCS: Computing methodologies Artificial intelligence Mohammed Al Owayyed, Adarsh Denga, Willem-Paul Brinkman Abstract . Child helpline training often relies on human-led roleplay, which is both time- and resource-consuming. To address this, rule-based interactive agent simulations have been proposed to provide a structured training experience for new counsellors. However, these agents might suffer from limited language understanding and response variety. To overcome these limitations, we present a hybrid interactive agent that integrates Large Language Models (LLMs) into a rule-based Belief-Desire-Intention (BDI) framework, simulating more realistic virtual child chat conversations. This hybrid solution incorporates LLMs into three components: intent recognition, response generation, and a bypass mechanism. We evaluated the system through two studies: a script-based assessment comparing LLM-generated responses to human-crafted responses, and a within-subject experiment ( N = 37 N=37 ) comparing the LLM-integrated agent with a rule-based version. The first study provided evidence that the three LLM components were non-inferior to human-crafted responses. In the second study, we found credible support for two hypotheses: participants perceived the LLM-integrated agent as more believable and reported more positive attitudes toward it than the rule-based agent. Additionally, although weaker, there was some support for increased engagement (posterior probability = 0.845 0.845 , 95% HDI [–0.149, 0.465]). Our findings demonstrate the potential of integrating LLMs into rule-based systems, offering a promising direction for more flexible but controlled training systems. 1. Introduction Child helplines offer a safe space for children in need to seek help. This requires extensive counselling training efforts. Children can contact helplines such as De Kindertelefoon through online chat interfaces or by phone. As of 2023, the Dutch Children’s Helpline, De Kindertelefoon, has over 700 volunteer counsellors, with more than 340 new volunteers trained that year—an increase of 20% compared to 2022 ( De Kindertelefoon 2023 ) . To train new counsellors, a common practice is role-playing, where one counsellor acts as a child contacting the helpline while the trainee engages in conversation. However, this method can be time-consuming and resource-intensive. A potential alternative is using interactive social agents to simulate child-helpline interactions, which were helpful in similar training scenarios, such as virtual patients in medical education ( Guetterman et al. 2019 ; Yao et al. 2022 ) . A crucial aspect of a simulation system for learning is being as faithful to the actual situation ( Saus et al. 2010 ) . In this case, simulating realistic, open-ended chat conversations of a virtual child. However, training counsellors using virtual agents can be sensitive, especially when using open-ended unrestricted interaction ( Bickmore 2022 ) . While closed-answer options provide a safer environment, they do not fully replicate real-world interactions where counsellors formulate responses independently to develop their chat conversational skills. One way to enable open-ended conversations is through rule-based intent recognition, where the system interprets user input and selects appropriate responses. An example of such a system is Lilobot, a BDI-based (Belief-Desire-Intention) training system designed for child helplines ( Grundmann et al. 2025 ; Al Owayyed et al. 2024 ) . The system focuses on bullying scenarios, one of the main topics children discuss with De Kindertelefoon ( Kindertelefoon 2023 ) . Lilobot simulates a virtual child who has been bullied at school and is seeking support from a helpline. The system follows a five-phase model of counselling conversations ( Sindahl 2011 ) , which trainees should apply correctly for the interaction to progress. The phases are building rapport, clarifying the child’s story, setting the conversation goal, working towards that goal and wrapping up the conversation. If a counsellor fails to follow these phases, Lilobot may leave the conversation. However, traditional intent recognition has limitations, as it can only respond within the constraints of its predefined rules and training data ( Rahman et al. 2017 ) . This was also reported for Lilobot. Although positively received for its training potential, participants highlighted certain limitations. Specifically, the system struggled to understand complex sentences outside its knowledge base, indicating a need for improved comprehension. Participants also expressed a need for more response variations, as variety and realism are important for effective learning ( Atkinson et al. 2000 ) . These challenges align with the broader limitations of rule-based conversational agents ( Rahman et al. 2017 ) . Recent advancements with large language models (LLMs) offer promising solutions to these limitations. LLMs can generate human-like text with contextual relevance and variability, making conversations appear more natural and engaging compared to natural language understanding and generation ( Freire et al. 2024 ; Karanikolas et al. 2023 ; Gürcan 2024 ) . However, using pure LLMs in training scenarios presents a challenge; while they enhance realism, they should operate within a controlled learning environment to ensure specific educational objectives are met. Without appropriate constraints, LLMs may generate unintended or extraneous content, which could interfere with the learning process, adding extraneous load to the trainee. To balance these considerations, we propose integrating LLMs with Lilobot’s existing BDI framework. This paper presents an integration design that preserves the structured learning environment required for counsellor training and evaluates its effects. Our evaluation has two parts: (1) a non-inferiority analysis, conducted through human coders, comparing LLM-generated and BDI system responses in terms of both understanding and response quality, and (2) an experimental study where participants used both LLM-integrated and BDI-only systems in a within-subject design. Based on this evaluation, we formulate and test the following hypotheses: H1: Individuals perceive the integrated system to be more believable than the rule-based system. H2: Individuals perceive the integrated system to be more engaging than the rule-based system. H3: Individuals have a more positive attitude towards the LLM-integrated system than the rule-based one. 2. Related Work 2.1. Social Agents for Helpline Training Researchers explored interactive social agents as tools for training helpline counsellors. Demasi et al. ( Demasi et al. 2020 ) proposed a chatbot simulation to train counsellors in suicide prevention hotlines. This simulation uses annotated transcripts and a multi-task framework to generate responses that mimic various crisis conversation scenarios. Evaluations of the proposed model showed improvements in response diversity and specificity compared to simpler models. For child helpline training specifically, one example is a serious game designed to simulate high-risk interactions in a safe environment (e.g., trafficking and sexual exploitation) ( Veldhuizen-Ochodničanová et al. 2023 ) . This game uses choice-based interactions to train counsellors in handling sensitive situations while developing essential skills. However, it does not support open-ended interactions, which helps avoid language processing and generating limitations. 2.2. LLMs for Counselling LLMs have been used to support counselling in various ways. To simulate a counsellor providing help, Steenstra et al. ( Steenstra et al. 2024 ) explored LLMs’ potential in delivering motivational interviewing for alcohol counselling. They found that LLM-powered virtual agent responses were perceived similarly to human-generated ones, addressing the limitations of rule-based approaches in understanding nuanced therapy conversations. Recently, Heinz et al. ( Heinz et al. 2025 ) reported a reduction of symptoms in depression, anxiety, feeding and eating disorders after people

Read the complete text