Structure Matters: Evaluating Multi-Agents Orchestration in Generative Therapeutic Chatbots CCS: Human-centered computing Empirical studies in HCI CCS: Human-centered computing Natural language interfaces CCS: Applied computing Health care information systems

Sina Elahimanesh, Mohammadali Mohammadkhani, Sara Zahedi Movahed, Mohammadmahdi Abootorabi, Shayan Salehi, Abbas Edalat · 2026

Read Structure Matters: Evaluating Multi-Agents Orchestration in Generative Therapeutic Chatbots CCS: Human-centered computing Empirical studies in HCI CCS: Human-centered computing Natural language interfaces CCS: Applied computing Health care information systems by Sina Elahimanesh, Mohammadali Mohammadkhani, Sara Zahedi Movahed, Mohammadmahdi Abootorabi, Shayan Salehi, Abbas Edalat (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2603-00774.txt.

Introduction

Elahimanesh and five colleagues test whether chatbot architecture shapes therapeutic dialogue across three designs grounded in Self-Attachment Technique psychotherapy. A multi agent system with a stage matched finite state machine and shared long term memory faces a single agent with identical knowledge and prompts, plus an unguided model. In an eight day trial of 66 Farsi speaking participants across three chatbots, the multi agent system feels significantly more natural and human like and leads on most other ratings. Structure matters as much as prompt wording, the findings suggest.

Catalog record

CREATORSina Elahimanesh, Mohammadali Mohammadkhani, Sara Zahedi Movahed, Mohammadmahdi Abootorabi, Shayan Salehi, Abbas Edalat
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE36757
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2603.00774

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Structure Matters: Evaluating Multi-Agents Orchestration in Generative Therapeutic Chatbots CCS: Human-centered computing Empirical studies in HCI CCS: Human-centered computing Natural language interfaces CCS: Applied computing Health care information systems Sina Elahimanesh, Mohammadali Mohammadkhani, Sara Zahedi Movahed, Mohammadmahdi Abootorabi, Shayan Salehi, Abbas Edalat Abstract . While large language models (LLMs) excel at open-ended dialogue, effective psychotherapy requires structured progression and adherence to clinical protocols, making the design of psychotherapist chatbots challenging. We investigate how different LLM-based designs shape perceived therapeutic dialogue in a chatbot grounded in the Self-Attachment Technique (SAT), a novel self-administered psychotherapy rooted in attachment theory. We compare three architectural variants: (1) a multi-agent system utilizing finite state machine aligned with therapeutic stages and a shared long-term memory, (2) a single-agent using identical knowledge-base and the same prompts, and (3) an unguided LLM. In an eight-day randomized controlled trial (RCT) with N = 66 N=66 Farsi-speaking participants, balanced across the three chatbots, the multi-agent system is perceived as significantly more natural and human-like than the other variants and achieves higher ratings across most other metrics. These findings demonstrate that for therapeutic AI, architectural orchestration is as critical as prompt engineering in fostering natural, engaging dialogue. Figure 1. Overview of the user study comprising three phases: (1) recruitment and blinded RCT group assignment; (2) an eight-day study period during which participants interacted with one of three therapeutic chatbot versions, multi-agent FSM-based, single-agent with therapy knowledge, or unguided single-agent; and (3) anonymized post-study survey completion data analysis. 1. Introduction Conversational agents are increasingly explored as scalable mental health support tools, offering on-demand access, anonymity, and reduced cost compared to traditional care ( 22 ; 23 ; 44 ; 19 ) . Advances in large language models (LLMs), such as ChatGPT, now enable open-ended, context-sensitive conversations, expanding the expressive capacity of mental health chatbots ( 27 ; 30 ) . At the same time, structured therapeutic frameworks can be embedded into conversational systems. The Self-Attachment Technique (SAT) ( 10 ) is a self-administered, attachment-based approach that supports emotion regulation and self-compassion by cultivating an imaginative affectional bond between the adult self and the childhood self of an individual ( 3 ; 12 ) . Prior SAT chatbots using rule-based or classification-driven dialogue demonstrated promising empathy and engagement, but were limited in flexibility and scalability ( 3 ; 12 ) . Despite these advances, most mental health chatbots emphasize feasibility and acceptability while relying on shallow interaction patterns and short-term evaluations, leaving open questions about how system architecture shapes perceived therapeutic quality ( 23 ; 44 ; 19 ) . Research highlights the importance of personalization, empathic responses, and sustained engagement ( 19 ; 22 ) , yet existing work rarely compares architectural paradigms under controlled conditions ( 47 ) . Recent proposals such as multi-agent orchestration and finite state machines offer ways to structure complex LLM behavior ( 47 ) , but their impact on therapeutic naturalness, authenticity, and engagement, particularly for structured interventions like SAT, remains underexplored. To address these gaps, we conducted a between-subjects randomized controlled trial (RCT) examining how LLM-based architectural design influences perceived therapeutic quality in a SAT-informed chatbot. Using GPT-4o ( 31 ) as the base model across all conditions, we compared three architectures. The Intervention employed a multi-agent finite state machine aligned with SAT stages, incorporating explicit state transitions, a SAT knowledge base, and shared long-term memory among the agents. The Active Control used a single agent with equivalent prompts and SAT access. The Placebo Control used GPT-4o without SAT knowledge or structure, resembling a generic conversational companion. System architecture and prompts were authored in English, while deployment targeted a Farsi-speaking user population. We deployed the system for eight days with N=66 participants (Intervention: 22, Control: 23, Placebo: 21), who completed demographic measures, interacted with their assigned chatbot, and completed post-study surveys. The intervention was perceived as significantly more natural and human-like than both control conditions (F=7.017, p=0.0018, \eta^{2} =0.187), with mostly higher but non-significant scores on other therapeutic metrics, such as trust, memory coherence, user satisfaction, fewer off-topic responses, and ease of use. Chat logs showed the intervention produced more but shorter messages (459 messages, \sim 230 characters) than the active control (336 messages, \sim 409 characters) and placebo control (206 messages, \sim 635 characters), indicating a more interactive dialogue style. Findings demonstrate that architectural choices, beyond prompt design alone, meaningfully shape user experience in LLM-based therapeutic chatbots. 2. Related Work The Self-Attachment Technique ( 10 ; 11 ) , grounded in Bowlby’s attachment theory ( 5 ; 6 ) , helps users regulate emotional distress by alternating between care-seeking and care-giving roles. Early SAT chatbots operationalize this with rule-based dialogue and emotion classification to provide empathetic responses ( 3 ; 26 ; 9 ) , enforcing structure but remaining limited in generative flexibility, contextual adaptation, long-term personalization, and multilingual support. Similarly, other therapeutic chatbots, including CBT systems ( 22 ; 19 ) , Evebot ( 46 ) , GPT-2–based therapy bots ( 45 ) , and student distress-detection frameworks ( 32 ) , combine emotion recognition with neural generation but generally lack explicit therapeutic state modeling, memory, and robustness. The emergence of large language models has enabled richer human–AI collaboration and motivated hybrid neuro-symbolic architectures. Human-in-the-loop systems support cognitive restructuring ( 38 ) , yet CAPE-II evaluations report failures in crisis handling and boundary maintenance ( 39 ) , and excessive trust can amplify harm ( 40 ; 7 ) . FSM-based systems like MindfulDiary ( 24 ) and ChaCha ( 37 ) enforce structured progression while allowing free expression, multi-agent frameworks improve robustness and empathy ( 47 ; 21 ) , EmoAgent monitors user state to reduce deterioration ( 33 ) , and LLMs serve as scalable front-ends for structured intake and assessment ( 36 ) . Compared to early SAT bots ( 3 ; 12 ) , these architectures provide stronger guarantees of consistency, safety, and flexibility. Emotion modeling and personalization are central to therapeutic effectiveness, optimizing empathy, fluency, and novelty ( 4 ) , adapting interventions via real-time mood detection (EMMA; ( 17 ) ), and strengthening therapeutic bonds ( 25 ) . RCTs validate their impact: Woebot ( 14 ) reduced symptoms compared to self-help, while modality and interaction style influence loneliness and emotional dependence ( 13 ) ; other studies highlight emotional resilience ( 28 ) , humor ( 42 ) , and just-in-time interventions ( 17 ) . Farsi-language systems remain scarce: early Persian SAT chatbots report positive engagement ( 12 ) , while HamRaz and HamRazEval support culturally grounded therapy ( 2 ) and PersianLLaMA provides a foundation for generative modeling ( 1 ) , though personalization and fluency remain limited. Our work synthesizes these strands by embedding a context-aware LLM within a multi-agent, FSM-driven SAT architecture, advancing prior SAT and Persian systems with structured progression, safety-aware generation, long-term engagement, and one of the first RCT-based evaluations of a Farsi SAT chatbot. 3. Method System Design and Architecture. To investigate how conversational structure and adaptivity influence therapeutic engagement, we developed a three-tier conversational AI system that delivers the Self-Attachment Technique within a Randomized Controlled Trial framework. The study included three experimental conditions: (1) a finite state machine

Read the complete text