SPIRAL: Symbolic LLM Planning via Grounded and Reflective Search
Yifan Zhang, Giridhar Ganapavarapu, Srideepika Jayaraman, Bhavna Agrawal, Dhaval Patel, Achille Fokoue · 2025
Read SPIRAL: Symbolic LLM Planning via Grounded and Reflective Search by Yifan Zhang, Giridhar Ganapavarapu, Srideepika Jayaraman, Bhavna Agrawal, Dhaval Patel, Achille Fokoue (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2512-23167.txt.
Introduction
Yifan Zhang and co-authors built SPIRAL, a planning framework that wraps three specialized language model agents inside a Monte Carlo tree search. Language models plan poorly over long horizons because their linear reasoning cannot backtrack from early mistakes, while plain tree search flounders when rewards are sparse and cannot use the model's semantic sense. SPIRAL's agents divide the cognitive labor: a Planner proposes creative next steps, a Simulator predicts realistic outcomes to ground the search, and a Critic scores positions through reflection.
That division turns brute-force search into a guided, self-correcting process with dense feedback. On the DailyLifeAPIs and HuggingFace datasets, SPIRAL beat chain-of-thought planning and other leading agent frameworks. Its accuracy on DailyLifeAPIs reached the mid-eighties percent, more than sixteen points above the next-best search framework, using tokens more efficiently. Code, appendices, and experimental data are released for reproducibility.
Catalog record
| CREATOR | Yifan Zhang, Giridhar Ganapavarapu, Srideepika Jayaraman, Bhavna Agrawal, Dhaval Patel, Achille Fokoue |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 85 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2512-23167
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
SPIRAL: Symbolic LLM Planning via Grounded and Reflective Search Yifan Zhang, Giridhar Ganapavarapu, Srideepika Jayaraman, Bhavna Agrawal, Dhaval Patel, Achille Fokoue Abstract Large Language Models (LLMs) often falter at complex planning tasks that require exploration and self-correction, as their linear reasoning process struggles to recover from early mistakes. While search algorithms like Monte Carlo Tree Search (MCTS) can explore alternatives, they are often ineffective when guided by sparse rewards and fail to leverage the rich semantic capabilities of LLMs. We introduce SPIRAL (Symbolic LLM Planning via Grounded and Reflective Search), a novel framework that embeds a cognitive architecture of three specialized LLM agents into an MCTS loop. SPIRAL’s key contribution is its integrated planning pipeline where a Planner proposes creative next steps, a Simulator grounds the search by predicting realistic outcomes, and a Critic provides dense reward signals through reflection. This synergy transforms MCTS from a brute-force search into a guided, self-correcting reasoning process. On the DailyLifeAPIs and HuggingFace datasets, SPIRAL consistently outperforms the default Chain-of-Thought planning method and other state-of-the-art agents. More importantly, it substantially surpasses other state-of-the-art agents; for example, SPIRAL achieves 83.6% overall accuracy on DailyLifeAPIs, an improvement of over 16 percentage points against the next-best search framework, while also demonstrating superior token efficiency. Our work demonstrates that structuring LLM reasoning as a guided, reflective, and grounded search process yields more robust and efficient autonomous planners. The source code, full appendices, and all experimental data are available for reproducibility at the official project repository. Code — https://github.com/IBM/SPIRAL Introduction The recent evolution of Large Language Models (LLMs) has marked a paradigm shift from pure text generation towards the development of autonomous agents capable of complex, goal-oriented behavior ( 44 ) . A key capability of these agents is their ability to formulate symbolic plans and interact with external tools to accomplish tasks ( 22 ) . Foundational approaches for this planning process often rely on prompting the LLM to interleave reasoning ”thoughts” with ”actions” in a linear sequence ( 34 ; 38 ) . However, the reliability of this auto-regressive generation is a critical bottleneck; the plans produced are often brittle, where a single logical error can derail the entire process without a mechanism for structured deliberation or backtracking. Consequently, enhancing the robustness of LLM agents necessitates a move beyond simple linear prompting towards more sophisticated planning paradigms. To address this challenge, integrating tree search frameworks has shown potential for adding structured exploration to the planning process ( 37 ; 45 ) . While MCTS ( 25 ) is a particularly compelling choice, applying it effectively presents significant challenges. Standard implementations often treat the LLM as a black-box policy with sparse rewards, leading to inefficient exploration ( 40 ) . Furthermore, without a grounding world model, agents may explore syntactically valid but practically nonsensical paths ( 5 ) . This highlights a critical gap for a framework that synergizes MCTS’s exploratory power with LLM semantics, guided by dense, semantic-aware feedback. Figure 1: A comparison of a Vanilla Agent and the SPIRAL Agent on a conditional planning task. The Vanilla Agent follows a linear but flawed plan, while SPIRAL uses its grounded and reflective search to correctly handle the weather-based constraint before acting. To overcome these limitations, we propose SPIRAL (Symbolic LLM Planning via Grounded and Reflective Search), a framework designed to address the strategic failures of linear planners. The motivation for our approach is illustrated in Figure 1 . Faced with a simple conditional planning task, a standard ”vanilla” agent follows a plausible but incorrect plan, failing to respect the user’s constraints. In contrast, SPIRAL successfully navigates the task by employing a more deliberative process. It reframes LLM-driven MCTS by introducing a cognitive architecture of three specialized LLM agents that work in concert: a Planner to propose actions, a Critic to reflect on their strategic soundness, and a Simulator to ground the plan in plausible outcomes. The synergy between these agents transforms the search process, allowing SPIRAL to intelligently prune flawed reasoning paths and converge on robust solutions. SPIRAL is built on the key insight that LLM-based planning can be made more robust by structuring it as a grounded and reflective search process. We achieve this by embedding three core principles within the MMCTS framework: 1) Decomposition, treating planning as a cognitive task fulfilled by specialized agentic roles; 2) Grounding, where a learned world model grounds exploration in plausible consequences; and 3) Reflection, using a dense reward signal that evaluates strategic merit to replace sparse terminal rewards. To our knowledge, SPIRAL is the first framework to holistically integrate these principles within a formal search algorithm for LLM agents. Our approach substantially outperforms other state-of-the-art agents; on the DailyLifeAPIs of TaskBench, SPIRAL achieves 83.6% accuracy, surpassing the next-best search framework by over 16 percentage points. Our main contributions are fourfold: 1) We propose SPIRAL , a novel framework that integrates a synergistic tri-agent cognitive architecture into Monte Carlo Tree Search to enable structured and dynamic planning. 2) We introduce its core mechanisms: a Planner to generate actions, a Critic to provide dense reflective feedback, and a Simulator to ground the search. Together, these components address the common MCTS challenges of sparse rewards and ungrounded exploration. 3) We evaluate SPIRAL on complex tool-use benchmarks, showing that it substantially outperforms both default planners and state-of-the-art frameworks like ReAct and LATS. 4) Finally, we demonstrate SPIRAL’s superior resource efficiency and validate our architecture through ablation studies that confirm the critical role of each component. Preliminaries SPIRAL’s core idea is to frame the complex task of LLM-based planning as a search process that is both grounded and reflective. To achieve this, the paper defines tool-use planning as a sequential decision-making problem within a Markov Decision Process (MDP) framework. The approach then uses the Monte Carlo Tree Search (MCTS) algorithm as its foundational search method. LLM-based Planning as a Search Problem We model the task of generating a multi-step tool-use plan as a search problem through a state space, formally defined as a sequential decision-making process by the tuple (\mathcal{S},\mathcal{A},T,R) . The state space \mathcal{S} consists of states s_{t}\in\mathcal{S} , where each state is the history of actions and observations generated so far, often called a chain: s_{t}=(a_{0},o_{1},\dots,a_{t-1},o_{t}) . The action space \mathcal{A} is the union of available tool calls, \mathcal{A}_{\text{tool}} , and a terminal action, \mathcal{A}_{\text{term}}=\{\texttt{finish}\} . The transition function T:\mathcal{S}\times\mathcal{A}\rightarrow\mathcal{S} maps a state-action pair to a new state. Given a state s_{t} and an action a_{t} from a policy \pi(a_{t}|s_{t}) , we approximate this transition with a world model, \mathcal{W}(o_{t+1}|s_{t},a_{t}) , which generates a plausible observation o_{t+1} . The subsequent state is then s_{t+1}=s_{t}\oplus(a_{t},o_{t+1}) . Finally, the reward function R:\mathcal{S}\times\mathcal{A}\rightarrow\mathbb{R} provides feedback. A key challenge in this domain is that the reward signal is sparse; it is zero for all intermediate steps and non-zero only upon termination, where R(s_{T},a_{T}) evaluates the final plan’s success. The overall objective is to find an optimal policy, \pi^{*} , which generates a sequence of actions (a_{0},a_{1},\dots,a_{T}) that maximizes the expected cumulative reward: \pi^{*}=\arg\max_{\pi}\mathbb{E}\left[\sum_{t=0}^{T}R(s_{t},a_{t})\right] (1) Monte Carlo Tree Search MCTS is a heuristic search algorithm particularly well-suited for navigating the large decision spaces inherent in planning problems. The algorithm iteratively builds a search tree to balance the