Generator-Assistant Stepwise Rollback Framework for Large Language Model Agent
Xingzuo Li, Kehai Chen, Yunfei Long, Xuefeng Bai, Yong Xu, Min Zhang · 2025
Read Generator-Assistant Stepwise Rollback Framework for Large Language Model Agent by Xingzuo Li, Kehai Chen, Yunfei Long, Xuefeng Bai, Yong Xu, Min Zhang (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2503-02519.txt.
Introduction
Xingzuo Li and colleagues gave language model agents a way to take back bad steps in 2025. Agents that think and act in turns usually commit every intermediate thought to their trajectory, right or wrong, so one early mistake poisons everything after it. Their framework, GA-Rollback, splits the labor between two roles to break that one-pass habit.
A generator agent interacts with the environment and proposes each action, while an assistant agent inspects every action before it sticks. When the assistant spots an error, it triggers a rollback, undoing the step so the generator can try again, and two extra strategies tailored to the rollback situation sharpen the recovery. Across three widely used benchmarks the method improved significantly over strong baselines, and the authors show it can bolt onto existing agents as a plug-and-play module.
Catalog record
| CREATOR | Xingzuo Li, Kehai Chen, Yunfei Long, Xuefeng Bai, Yong Xu, Min Zhang |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 45 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2503-02519
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Generator-Assistant Stepwise Rollback Framework for Large Language Model Agent Xingzuo Li, Kehai Chen, Yunfei Long, Xuefeng Bai, Yong Xu, Min Zhang Abstract Large language model (LLM) agents typically adopt a step-by-step reasoning framework, in which they interleave the processes of thinking and acting to accomplish the given task. However, this paradigm faces a deep-rooted one-pass issue whereby each generated intermediate thought is plugged into the trajectory regardless of its correctness, which can cause irreversible error propagation. To address the issue, this paper proposes a novel framework called Generator-Assistant Stepwise Rollback (GA-Rollback) to induce better decision-making for LLM agents. Particularly, GA-Rollback utilizes a generator to interact with the environment and an assistant to examine each action produced by the generator, where the assistant triggers a rollback operation upon detection of incorrect actions. Moreover, we introduce two additional strategies tailored for the rollback scenario to further improve its effectiveness. Extensive experiments show that GA-Rollback achieves significant improvements over several strong baselines on three widely used benchmarks. Our analysis further reveals that GA-Rollback can function as a robust plug-and-play module, integrating seamlessly with other methods. 1 1 1 Our code is available at https://github.com/wisper12933/GA-Rollback . 1 Introduction Figure 1: An example of one-pass paradigm. The trajectory is generated by LLaMA3.1-8B-Instruct in ReAct-style. Our approach blocks error propagation through rollback operations and ensures the quality of thinking processes through evaluation. Developing Large Language Model (LLM) agents capable of helping humans tackle real-world challenges has become a central focus in current artificial intelligence research Xi et al. 2023 ; Zhang et al. 2023 . Recently, researchers have leveraged the inherent self-planning abilities of LLMs to integrate thoughts with actions, enabling step-by-step reasoning processes during agent tasks Song et al. 2022 ; Yao et al. 2023b ; Qiao et al. 2024 . This enhancement empowers LLM agents to achieve remarkable improvements on diverse applications, including mathematical reasoning Hendrycks et al. 2021 ; Cobbe et al. 2021 , web browsing Yao et al. 2022 ; Deng et al. 2023 ; Zhou et al. 2024b , and embodied tasks Shridhar et al. 2020 ; Wang et al. 2022 . However, this paradigm is inherently constrained by an one-pass limitation, where each generated thought is inserted directly into the trajectory regardless of whether it is correct or not. As a result, incorrect thoughts may persist in the context, influencing subsequent actions and ultimately compromising the outcome. Take an example in Figure 1 generated by LLaMA3.1-8B-Instruct in ReAct-style. In this scenario, the environment responds to all thoughts with a simple “ OK ”, which implies that the system assumes the correctness of these thoughts by default. Consequently, a low-quality thought that mistakes “ butterknife ” for “ knife ” leads to the invalid action “ take knife 1 from countertop 2 ”. To address this issue, many works have been devoted to improving the precision and clarity of reasoning processes. Typically, self-correction methods Madaan et al. 2023 ; Shinn et al. 2023 ; Gou et al. 2024 summarize refined plans using feedback from previous executions and improve the quality of the solution through multiple trials. Despite these achievements, the one-pass reasoning pattern within each trial remains prone to error propagation due to its dependence on potentially flawed intermediate steps. In this paper, we propose a novel framework called Generator-Assistant Stepwise Rollback (GA-Rollback) to induce better decision-making for goal-driven agents. Specifically, the generator interacts with the environment and supplies the assistant with essential contextual information. Meanwhile, the assistant meticulously examines each action produced by the generator along with the corresponding observation. When errors or suboptimal actions are detected, the assistant will provide detailed feedback to guide the generator in performing rollback operations, which revise previous incorrect actions to prevent error propagation. To further enhance our framework, we introduce two key strategies: probability-based feedback evaluation (§ 3.2 ) to ensure feedback credibility, and Wait-Info strategy (§ 3.3 ) to enrich contextual information in embodied tasks. Experiments on three representative tasks reveal that our method outperforms several strong baselines across various models. Moreover, our analysis demonstrates that GA-Rollback can function as a robust plug-and-play module, integrating seamlessly with other methods. In summary, our main contributions are as follows: • We propose the GA-Rollback framework for llm-based agents, which separates action and thinking processes to ensure more precise and credible reasoning trajectory. • We introduce the probability-based feedback evaluation along with Wait-Info strategy designed for embodied environments. • Experiments on three tasks reveal that our method achieves notable improvement and exhibits stronger robustness, indicating its extensibility as a plug-and-play module. 2 Related Work Our work is related to LLM agents and self-correction mechanism. In this section, we first review recent advances in LLM agents, followed by an analysis of self-correction mechanism. 2.1 LLM Agents The reasoning and instruction-following capabilities that have emerged in LLMs Huang and Chang 2022 ; Wei et al. 2022a make them capable enough to serve as intelligent agents to complete various tasks, such as web navigation Deng et al. 2023 ; Zhou et al. 2024b and machine translation Chen et al. 2024a ; Chen et al. 2024b . Standard LLM-agent methods employ structured reasoning frameworks to strengthen their analytical abilities Yao et al. 2023a ; Besta et al. 2024 , or apply specialized decoding strategies to improve accuracy Wang et al. 2024 . Recent works have made significant breakthroughs by leveraging the inherent self-planning and reflection abilities of LLMs Yao et al. 2023b ; Paul et al. 2024 ; Zhou et al. 2024a . These methods achieve more effective interaction with external environments through task decomposition and planning within or between trials. While these methods have shown great progress, they still face a limitation in the lack of necessary evaluation of the reasoning process. This shortcoming can lead to the generation of low-quality reasoning steps, which may mislead subsequent actions and cause error propagation. 2.2 Self-correction Mechanism Self-correction represents a feasible approach to improve responses from LLMs by enabling them to refine their outputs during inference Bai et al. 2022 ; Madaan et al. 2023 . The simplest implementation of self-correction prompts LLMs to provide feedback on their own responses and revise the responses based on the feedback Huang et al. 2023 . Recent studies have been dedicated to improving feedback by using additional resources, including external tools such as code executors Chen et al. 2024c ; Stengel-Eskin et al. 2024 ; Gou et al. 2024 , knowledge accessed through web browsing Jiang et al. 2023 ; Zhao et al. 2023 ; Peng et al. 2023 , and observation gathered from simulation environments Shinn et al. 2023 . Notably, Kamoi et al. 2024 identified feedback generation as the bottleneck in self-correction process, emphasizing that high-quality, reliable feedback is essential for LLMs to successfully complete assigned tasks. Building upon these insights, our framework leverages LLMs as independent assistants in interactive environments and incorporates appropriate evaluations to enhance the credibility of generated feedback. Figure 2: An illustrative overview of GA-Rollback framework. The dark green arrows represent the current workflow. Each action and observation will be reviewed by the assistant, and the feedback provided by the assistant will also be evaluated before being applied to the trajectory. 3 Methodology The overall architecture of