Limits of Emergent Reasoning of Large Language Models in Agentic Frameworks for Deterministic Games
Chris Su, Harrison Li, Matheus Marques, George Flint, Kevin Zhu, Sunishchal Dev · 2025
Read Limits of Emergent Reasoning of Large Language Models in Agentic Frameworks for Deterministic Games by Chris Su, Harrison Li, Matheus Marques, George Flint, Kevin Zhu, Sunishchal Dev (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2510-15974.txt.
Introduction
Chris Su and five co-authors ask whether giving a language model a live game board cures its collapse on hard puzzles. Recent reports show reasoning models failing sharply once puzzle complexity passes a threshold. A fair objection is that tracking the board state alone might be the true burden, not reasoning itself.
Their test uses Tower of Hanoi with an environment interface: the model calls a tool to move, writes its justification, sees the new state, and prompts itself for the next move. Collapse still arrives at the same complexity, neither delayed nor removed. Policy analysis shows the model's choices drift away from both optimal and random play as levels rise, a mode-like failure at each level. Success hinges on whether the model's favoured mode happens to be the right solution, the team suggests.
Catalog record
| CREATOR | Chris Su, Harrison Li, Matheus Marques, George Flint, Kevin Zhu, Sunishchal Dev |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 51 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2510-15974
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Limits of Emergent Reasoning of Large Language Models in Agentic Frameworks for Deterministic Games Chris Su, Harrison Li, Matheus Marques, George Flint, Kevin Zhu, Sunishchal Dev Abstract Recent work reports that Large Reasoning Models (LRMs) undergo a collapse in performance on solving puzzles beyond certain perplexity thresholds. In subsequent discourse, questions have arisen as to whether the nature of the task muddles an evaluation of true reasoning. One potential confound is the requirement that the model keep track of the state space on its own. We provide a large language model (LLM) with an environment interface for Tower of Hanoi problems, allowing it to make a move with a tool call, provide written justification, observe the resulting state space, and reprompt itself for the next move. We observe that access to an environment interface does not delay or eradicate performance collapse. Furthermore, LLM-parameterized policy analysis reveals increasing divergence from both optimal policies and uniformly random policies, suggesting that the model exhibits mode-like collapse at each level of complexity, and that performance is dependent upon whether the mode reflects the correct solution for the problem. We suggest that a similar phenomena might take place in LRMs. 1 1 1 Code: https://anonymous.4open.science/r/cdz50eodtx/README.md 1 Introduction Recent advances in large language models (LLMs) have enabled them to solve difficult problems by generating “thoughts" involved in the solution ( Wei et al. 2022a ; Chen et al. 2023 ; Yao et al. 2023 ; Besta et al. 2024 ) . However, these models, often called large reasoning models (LRMs), can struggle with reasoning when faced with problems that require multistep logic ( Wei et al. 2022a ) . This limitation is especially pronounced in domains such as game-like scenarios, e.g. Tower of Hanoi. There is ongoing discussion that reasoning capability measured by existing benchmarks may be inflated ( Chollet 2019 ; Wei et al. 2022a ; Chen et al. 2023 ) , where reasoning ability is often conflated with memorization of training data. Furthermore, existing evaluation approaches of puzzle-solving scenarios require models to maintain internal representations of state spaces, introducing potential confounds that obscure whether failures stem from reasoning deficits or architectural limitations in state tracking. To test this, we provide an environmental framework that externalizes state management. This externalization enables us to conduct policy analysis by treating the model as an agent whose decision patterns can be analyzed. If models possess genuine reasoning capabilities, independent of state management limitations, we should observe policies that approach optimal behavior, differ meaningfully from random exploration, and exhibit exploratory patterns when faced with novel states. Conversely, if models exhibit deterministic behavior, this would suggest that apparent reasoning is actually fixed, learned patterns rather than dynamic problem-solving capabilities. We hypothesize that allowing models to interact dynamically with problem states rather than maintaining internal representations will force the model to use emergent heuristics and reasoning capabilities outside of the model’s priors created during the training process. To this end, we analyze criticism of LLM and LRM reasoning capabilities ( Shojaee et al. 2025 ; Lawsen et al. 2025 ) through the lens of an agentic environment framework, where the foundational model must plan and execute a series of optimal steps that will allow it to reach the goal state. We build on an increasing complexity approach introduced by Shojaee et al. 2025 , extending an interactive environment where models can make sequences of singular tool calls. This setup allows us to track the reasoning states of the models and observe their decision making in a structured manner. Our makes two key contributions to understanding emergent reasoning in large language models. First, we establish that providing dynamic environment interfaces does not mitigate performance collapse in reasoning tasks. Second, we demonstrate that models increasingly diverge from optimal policies and uniformly random policies, suggesting that the models are incapable of learning from past mistakes. In particular, we find that the models fall into looping behavior, characterized by sequences which transitions return to previously visited states. For deterministic games with single goal states such as Tower of Hanoi, this is considered un-optimal or problematic behavior, as returning to a previously visited state implies that the agent has made no progress. 2 Related Works Shojaee et al. 2025 proposed a puzzle-based framework featuring controllable problem complexity to evaluate Large Language Model (LLM) and Large Reasoning Model (LRM) reasoning. The paper demonstrated that LLMs and their reasoning counterparts both systematically collapse at high complexity. Critics of this work argue that observed failures stem from experimental limitations, such as token limits or unsolvable problems ( Lawsen et al. 2025 ) . Reasoning performance is noted to be highly task-dependent and nonlinear, and the debate on reasoning in LRMs is nuanced ( Varela et al. 2025 ) . While there is a diverse landscape of reasoning benchmarks, this proliferation shows the difficulty of defining reasoning itself. The concept of emergent reasoning, where qualitatively new abilities appear at certain model scales ( Wei et al. 2022b ; Berti et al. 2025 ) , has gained particular attention. Models trained on structured data like code have shown improved generalization across varied tasks ( Aryabumi et al. 2024 ) . Yet this apparent emergence raises questions: Are models developing actual reasoning capabilities? Or merely becoming more adept at recognizing and reproducing logical patterns from training data? Our approach builds directly on the complexity-based evaluation of Shojaee et al. 2025 while also addressing experimental limitations and providing a framework that separates reasoning assessment from potential confounds and allows for analysis of LLM policy. 3 Methodology We evaluate reasoning as search in the context of the Tower of Hanoi puzzle. This environment requires systematic exploration and move pruning, as well as a controllable search space where search space size scales exponentially with increasing number of disks ( Shojaee et al. 2025 ; Lawsen et al. 2025 ) . 3.1 Tower of Hanoi The Tower of Hanoi puzzle is a recursive puzzle consisting of three pegs and n disks of distinct sizes initially stack in order of size on a single peg (largest on the bottom). The objective is to transfer the entire stack from the first peg to the third peg. The rules are (1) that only the top disk of a peg may be moved, and (2) that a larger disk may never be placed on top of a smaller one. The difficulty of this puzzle grows with the number of disks, where the optimal number of moves required to solve an n-disk instance is 2^{n}-1 ( Frame and Stewart 1941 ) . 3.2 Experimental Setup 3.2.1 Baseline We first establish a baseline in which models are required to generate a complete Tower of Hanoi solution in a single pass with no interaction with the environment. Our experiments are conducted on large language models and their LRM counterparts. Similar to the experimental setup of Shojaee et al. 2025 , we specifically use Claude 3.7 Sonnet (and its reasoning counterpart), DeepSeek V3.1, and DeepSeek R1 because these allow access to reasoning traces ( Anthropic 2025 ; DeepSeek-AI et al. 2024 ) . We restrict evaluation to up to n = 8 disks, which corresponds to the regime in which IOT reported almost complete