ROAD: Reflective Optimization via Automated Debugging for Zero-Shot Agent Alignment

Natchaya Temyingyong, Daman Jain, Neeraj Kumarsahu, Prabhat Kumar, Rachata Phondi, Wachiravit Modecrua, Krittanon Kaewtawee, Krittin Pachtrachai, Touchapon Kraisingkorn · 2025

Read ROAD: Reflective Optimization via Automated Debugging for Zero-Shot Agent Alignment by Natchaya Temyingyong, Daman Jain, Neeraj Kumarsahu, Prabhat Kumar, Rachata Phondi, Wachiravit Modecrua, Krittanon Kaewtawee, Krittin Pachtrachai, Touchapon Kraisingkorn (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2512-24040.txt.

Introduction

Natchaya Temyingyong and co-authors address a cold-start problem. Tuning a language model agent usually demands a large, labeled test set. New projects rarely have one. Engineers instead face messy production logs and shifting failure modes. The team's framework, ROAD, treats prompt improvement as debugging.

ROAD uses three cooperating agents. One finds the root cause of each failure. A second groups recurring patterns across failures. A third folds the lessons into decision tree protocols. These protocols then guide the main agent. The team tested ROAD on an academic benchmark. They also tried it on a live knowledge management system. Success rates rose within just three rounds of revision. On hard retail reasoning tasks, performance improved by about a fifth. The authors argue this debugging loop can rival far heavier training methods.

Catalog record

CREATORNatchaya Temyingyong, Daman Jain, Neeraj Kumarsahu, Prabhat Kumar, Rachata Phondi, Wachiravit Modecrua, Krittanon Kaewtawee, Krittin Pachtrachai, Touchapon Kraisingkorn
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE62 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2512-24040

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

ROAD: Reflective Optimization via Automated Debugging for Zero-Shot Agent Alignment Natchaya Temyingyong, Daman Jain, Neeraj Kumarsahu, Prabhat Kumar, Rachata Phondi, Wachiravit Modecrua, Krittanon Kaewtawee, Krittin Pachtrachai, Touchapon Kraisingkorn Abstract Automatic Prompt Optimization (APO) has emerged as a critical technique for enhancing Large Language Model (LLM) performance, yet current state-of-the-art methods typically rely on large, labeled gold-standard development sets to compute fitness scores for evolutionary or Reinforcement Learning (RL) approaches. In real-world software engineering, however, such curated datasets are rarely available during the initial cold start of agent development, where engineers instead face messy production logs and evolving failure modes. We present ROAD (Reflective Optimization via Automated Debugging) , a novel framework that bypasses the need for refined datasets by treating optimization as a dynamic debugging investigation rather than a stochastic search. Unlike traditional mutation strategies, ROAD utilizes a specialized multi-agent architecture, comprising an Analyzer for root-cause analysis, an Optimizer for pattern aggregation, and a Coach for strategy integration to convert unstructured failure logs into robust, structured Decision Tree Protocols . We evaluated ROAD across both a standardized academic benchmark and a live production Knowledge Management engine. Experimental results demonstrate that ROAD is highly sample-efficient, achieving a 5.6% increase in Success Rate (73.6% to 79.2%) and a 3.8% increase in Search Accuracy within just three automated iterations. Furthermore, on complex reasoning tasks in the Retail domain, ROAD improved agent performance by approximately 19% relative to the baseline. These findings suggest that mimicking the human engineering loop of the fail analysis patch offers a viable, data-efficient alternative to resource-intensive RL training for deploying reliable LLM agents. 1 1 footnotetext: Co-first authors 1 1 footnotetext: Amity AI Research and Application Center, {natchaya, rachata.pho, wachiravit}@amitysolutions.com 2 2 footnotetext: Tollring, {daman.jain, neeraj.kumarsahu, prabhat.kumar}@tollring.com 1 Introduction The integration of Large Language Models (LLMs) into complex software systems has necessitated a shift from traditional coding to prompt engineering [ 1 , 2 ] . However, optimizing these LLM systems is rarely a straightforward path. While recent academic research has introduced powerful algorithms for Automatic Prompt Optimization (APO)—utilizing evolutionary strategies or Reinforcement Learning (RL) to refine agent behaviors [ 8 , 4 , 9 , 3 ] —applying these methods within the constraints of an active engineering organization reveals a critical bottleneck: the data itself. Standard optimization methodologies, such as those relying on gradient-like feedback or declarative compilation [ 5 , 6 ] , typically operate on the assumption that a “gold-standard” dataset exists. These approaches require a curated training set ( D_{train} ) consisting of high-quality inputs paired with verifiable evaluation metadata, such as correct answers or unit tests, to drive the evolutionary loop. In academic settings, such datasets are often readily available [ 17 ] . However, in the rapid “fail-fix-deploy” cycles of real-world software development, engineers rarely have the luxury of curating a refined dataset upfront. Instead, development teams are frequently confronted with the “Cold Start” problem, working with messy production logs, evolving edge cases, and complex failure modes that do not fit neatly into a static Question-and-Answer format [ 19 , 18 ] . The cost of pausing development to manually label thousands of examples creates a friction that renders traditional data-heavy optimization algorithms impractical for many production environments. To bridge the gap between academic optimization algorithms and the iterative needs of real-world software development, we introduce ROAD (Reflective Optimization via Automated Debugging) . ROAD is a novel framework designed to automate the failure analysis loop without requiring a perfect dataset from day one. Unlike traditional approaches that rely on static training data or stochastic text mutations, ROAD treats optimization as a dynamic debugging investigation. By analyzing the “messy” logs of failed interactions, the framework utilizes a multi-agent architecture to identify root causes and systematically evolve the agent’s core instructions, drawing inspiration from automated fault localization and self-correction techniques [ 21 , 20 , 11 , 10 , 12 ] . The ROAD framework diverges from conventional prompt engineering by shifting the output format from unstructured natural language to structured Decision Tree Protocols . By mapping out the logic gaps where models fail, ROAD generates rigorous operational frameworks that enforce strict sequencing and safety guardrails, effectively reducing hallucination and logic errors [ 23 , 32 ] . We validated the ROAD workflow across two distinct environments: the standardized academic benchmark and a live production Knowledge Management (KM) engine. Our experiments demonstrate that ROAD achieves significant performance gains with high sample efficiency. In a real-world deployment, the framework delivered a 5.6% increase in Success Rate and a 3.8% boost in Search Accuracy within just three automated iterations. Furthermore, on complex retail domain tasks using the Qwen3-4B model [ 29 ] , ROAD improved performance by approximately 19% relative to the baseline . These results confirm that by “trading tokens for time”—utilizing LLMs to automate the role of a lead engineer [ 30 ] —ROAD offers a scalable solution for building robust, self-correcting agents in data-scarce environments. 2 Related Work Prior work on aligning and optimizing LLM-based agents has largely relied on large, curated datasets and heavy training. Early approaches framed agent behavior as a reinforcement learning (RL) problem, tuning policies based on scalar rewards or human feedback. For example, Ziegler et al. (2020) demonstrated RL from human preferences to align language models, but it required tens of thousands of labeled comparisons to achieve good performance. Similarly, automated prompt engineering methods have emerged: soft prompt tuning uses gradient descent on embedding vectors, but these prompts lack interpretability and portability across models. Discrete prompt optimization is more challenging. Shin et al. (2020) introduced AutoPrompt , a gradient-guided search to generate fill-in-the-blank prompts, but this method suffered from training instability and limited gains in practice. More recently, RLPROMPT (Deng et al., 2022) used reinforcement learning to train a policy network that directly generates optimized discrete prompts, achieving state-of-the-art results on few-shot classification and style-transfer tasks. Notably, these methods still assume a fixed reward signal or labeled evaluation set for training. Other work has shown that prompting strategies which elicit reasoning or self-analysis can improve LLM performance. Chain-of-Thought prompting (Wei et al., 2022) demonstrated that providing a sequence of intermediate reasoning steps dramatically boosts reasoning accuracy on arithmetic and symbolic reasoning tasks. Independent of reward tuning, instructing a model to reflect on its outputs has been shown to improve problem solving. Renze and Guven (2024) found that LLM agents significantly improve when asked to analyze and correct their own mistakes. In the code generation domain, Song et al. (2024) proposed a best-first tree search (BESTER) framework in which an LLM generates multiple self-reflection comments on its buggy code and then repairs each variant; this iterative debugging yielded state-of-the-art results on code benchmarks. The Genetic-Pareto (GEPA) approach is another recent development: it treats prompts as “genetic” candidates that are iteratively mutated and tested, learning high-level rules via natural language reflection. GEPA showed that optimizing prompts through language-based mutation and selection can outperform RL methods by \sim 10% on average while using orders of magnitude fewer evaluations. However, like other methods, GEPA still relies on a gold-standard training set to evaluate prompt fitness. In contrast, our ROAD framework automates a human-like debugging loop, extracting corrections from raw failure logs. To our

Read the complete text