A Practical Approach for Building Production-Grade Conversational Agents with Workflow Graphs
Chiwan Park Wonjun Jang Daeryong Kim Aelim Ahn Kichang Yang · 2025
Read A Practical Approach for Building Production-Grade Conversational Agents with Workflow Graphs by Chiwan Park Wonjun Jang Daeryong Kim Aelim Ahn Kichang Yang (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2505-23006.txt.
Introduction
Chiwan Park and co-authors share a practical way to build production grade chat agents with workflow graphs in a 2025 paper. Large language models have lifted search, recommendation, and chatbot services. Yet real services need both flexible talk and strict rule compliance, and the random nature of such models makes those goals clash.
Their approach maps service flows as graphs, so the agent follows set paths while keeping natural talk. A case study in online shopping details the build flow and the tuning steps used. The authors frame the work as a bridge from research to real use, giving a path to agents that are scalable, controllable, and reliable at work.
Catalog record
| CREATOR | Chiwan Park Wonjun Jang Daeryong Kim Aelim Ahn Kichang Yang |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 34 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2505-23006
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
A Practical Approach for Building Production-Grade Conversational Agents with Workflow Graphs Thanks: Corresponding authors Thanks: Contact: {marko.k, don.kang}@kakaocorp.com Chiwan Park Wonjun Jang Daeryong Kim Aelim Ahn Kichang Yang Abstract The advancement of Large Language Models (LLMs) has led to significant improvements in various service domains, including search, recommendation, and chatbot applications. However, applying state-of-the-art (SOTA) research to industrial settings presents challenges, as it requires maintaining flexible conversational abilities while also strictly complying with service-specific constraints. This can be seen as two conflicting requirements due to the probabilistic nature of LLMs. In this paper, we propose our approach to addressing this challenge and detail the strategies we employed to overcome their inherent limitations in real-world applications. We conduct a practical case study of a conversational agent designed for the e-commerce domain, detailing our implementation workflow and optimizations. Our findings provide insights into bridging the gap between academic research and real-world application, introducing a framework for developing scalable, controllable, and reliable AI-driven agents. 1 Introduction Large Language Models ( OpenAI 2022 ; OpenAI 2023 ; Antrophic 2024 ; Touvron et al. 2023 ) have exhibited exceptional performance improvement across various language tasks, making them highly valuable in numerous industries. Beyond their language task performance, several works Schick et al. 2023 ; Yao et al. 2023b ; Qin et al. 2024 demonstrate the model’s ability to effectively utilize external tools to tackle complex tasks in various domains, including coding Zhang et al. 2024a , travel planning Xie et al. 2024 , recommendation Wang et al. 2024 , and scientific research Gottweis et al. 2025 . This ability rapidly led to the advancements of Conversational Agents, which aim to assist users with real-world tasks, such as booking restaurants or purchasing gifts, by interacting with external systems. Despite their excellent performance, many challenges still exist in building real-world agents Sadek et al. 2023 . First, because of the nature of the probabilistic next-token generation of LLMs, the agents randomly fail to comply with business requirements for specific domains. For example, considering a conversational e-commerce agent, the agent should retrieve the exact metadata of products to prohibit recommending cigarettes or alcohol to an underage user. However, occasionally, the agent uses its pre-trained knowledge instead of retrieving the external metadata, resulting in a wrong hallucinated response Zhang et al. 2024b . This drawback becomes particularly apparent in cases where strict compliance with business requirements exists. Second, there is a general demand for response formatting capabilities for the agent. In the case of mobile-targeted agents, due to their small screen size, the model should respond with a specific format, such as a length limit and emoji bullets. Furthermore, for certain products, the e-commerce agent must strictly comply with specific constraints, such as avoiding hype or exaggerated advertisements or ensuring proper attribution and source citation. Last, prompt engineering involves writing detailed descriptions into the system prompt to ensure that LLMs follow these requirements. The more detailed requirements are, the longer the system prompts will be; thus, the comprehensive system prompt degrades the latency and accuracy of response Levy et al. 2024 . Figure 1: A mobile messenger conversation between a user and our e-commerce agent. The first two turns require external tool calls to respond without hallucination. There are also output format constraints to make the responses readable in a mobile environment, such as emoji bullets. In this paper, we present our work on building a conversational e-commerce agent that runs on a mobile messenger. Figure 1 shows an example conversation between a user and the agent about purchasing a birthday gift for a friend. The agent helps the user explore products through search and recommendation, obtain detailed information about a product, and purchase the product. Furthermore, using social relationships in the messenger platform, the user can check the birthdays of his/her friends and send gifts. We adopt a hybrid approach that leverages a directed acyclic graph (DAG) workflow to guide the agent’s behavior, instead of relying on end-to-end generation from LLMs. This design enables flexible interactions while ensuring strict compliance with scenario-specific requirements. While the DAG framework efficiently handles the complex business requirements, fine-tuning becomes nontrivial since each message of the chat history comes from different states. To tackle this problem, we present a dataset construction and training approach that enables effective fine-tuning despite state-dependent chat histories. We begin with converting our requirements into a workflow graph. Then, we implement the workflow as a prototype agent with LLMs and several system prompts. After gathering annotated conversations between human annotators and the prototype agent, we used the conversations to train our agent models carefully. We repeat this process iteratively to achieve the required response quality. Thanks to the hybrid approach and training, the agent shows a 52% improvement in task accuracy and a 50% improvement in format adherence compared to the baseline, outperforming GPT-4o performance. Our main contributions are as follows: • Multi-State DAG Framework: Real-world agents must comply with many scenario-dependent constraints. We present a graph-based framework, each state with distinct prompts, tools and execution rules adhering to the specific constraints of the state. Traversing the graph seamlessly represents the wide range of expected scenarios, while efficiently distributing constraint handling across appropriate states. • Training Strategy on DAG Framework: We introduce a dataset construction and training strategy specifically designed to overcome the challenges posed by state-dependent message histories in our DAG framework. This further enhances the precision of the agent to meet even the stringent demands of sensitive domains such as e-commerce. • Real-World Example: We show a real-world working example using the two methods above. Our empirical results clearly demonstrate that even state-of-the-art LLMs fall short in achieving satisfactory performance in the e-commerce domain, underscoring the necessity of our proposed hybrid approach for practical deployment. 2 Background 2.1 Conversational Agents Traditional dialog-based frameworks such as Rasa Rasa Technologies 2019 and Talkamatic Larsson and Berman 2016 manage conversations using rule-based state tracking, offering reliability and interpretability. However, they often lack the flexibility and reasoning capabilities of modern LLM-based agents. Recent advances in LLMs such as GPT-4 OpenAI 2023 , Claude Antrophic 2024 , Mixtral Jiang et al. 2024 , Qwen Yang et al. 2024 , and Deepseek DeepSeek-AI 2024 have driven rapid progress and shifted expectations regarding the fluency and capabilities of conversational agents. LLMs can invoke external tools when provided with natural-language descriptions and instructions. Toolformer Schick et al. 2023 demonstrates how LLMs formulate calls of external tools with appropriate parameters based on a few examples and textual instructions. ToolLLM Qin et al. 2024 shows that LLMs can use multiple external tools to answer user questions. Agents with reasoning capabilities like ReAct Yao et al. 2023b , Chain-of-Thoughts Wei et al. 2022 , and Tree-of-Thoughts Yao et al. 2023a show significant performance improvements. As LLM-based agents become more capable and widely adopted, much effort has also been devoted to evaluating their performance across various domains, such as general agents Liu et al. 2024 ; Ma et al. 2024 , travel planning Xie et al. 2024 , games Costarelli et al. 2024 , coding Zhang et al. 2024a , and scientific research Gottweis et al. 2025 . These evaluation methods vary slightly in detail, but they all essentially