Large Language Models Can Take False First Steps at Inference-time Planning
Haijiang Yan ([email protected]), Jian-Qiao Zhu, Adam Sanborn · 2026
Read Large Language Models Can Take False First Steps at Inference-time Planning by Haijiang Yan ([email protected]), Jian-Qiao Zhu, Adam Sanborn (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2602-02991.txt.
Introduction
Language models can plan during training yet seem short-sighted when generating, and Haijiang Yan and 2 co-authors offer a Bayesian explanation in a 2026 paper. Their account centers on context. Subtle differences between natural language and the language the model internalized mean self-generated context gradually shifts planning behavior during inference, creating an appearance of weak planning.
Two controlled experiments test the account. In a random-generation task, planning looked constrained under human prompts but strengthened as self-generated context accumulated. In a Gaussian-sampling task, conditioning on self-generated sequences reduced initial bias. Together, theory and evidence characterize how models plan ahead at inference time. The first step a model takes can be false, the authors show, because of the context it has built.
Catalog record
| CREATOR | Haijiang Yan ([email protected]), Jian-Qiao Zhu, Adam Sanborn |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 28 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2602-02991
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Large Language Models Can Take False First Steps at Inference-time Planning Haijiang Yan ([email protected]), Jian-Qiao Zhu, Adam Sanborn Abstract Large language models (LLMs) have been shown to acquire sequence - level planning abilities during training, yet their planning behavior exhibited at inference time often appears short - sighted and inconsistent with these capabilities. We propose a Bayesian account for this gap by grounding planning behavior in the evolving generative context: given the subtle differences between natural language and the language internalized by LLMs, accumulated self-generated context drives a planning-shift during inference and thereby creates the appearance of compromised planning behavior. We further validate the proposed model through two controlled experiments: a random - generation task demonstrating constrained planning under human prompts and increasing planning strength as self - generated context accumulates, and a Gaussian - sampling task showing reduced initial bias when conditioning on self - generated sequences. These findings provide a theoretical explanation along with empirical evidence for characterizing how LLMs plan ahead during inference. Keywords: Large language models; Inference - time planning; Bayesian inference; Cognitive modeling of LLMs. Introduction Despite the fact that LLMs generate text by predicting the next token auto - regressively, a growing body of evidence indicates that they engage in sequence - level planning ( 2 ; 20 ; 19 ; 23 ; 28 ; 18 ; 5 ; 7 ) , rather than operating in a purely “short - sighted” manner. 2 establish a theoretical equivalence between next - token prediction in LLMs and sequence - level modeling in energy - based models, demonstrating that training LLMs for auto - regressive prediction is, in effect, training them to plan at the sequence level. Under this view, an LLM should maintain awareness of the full set of future continuations it is implicitly planning for before any individual token is generated. Empirically, however, related evidence for such planning remains limited: studies instead report short - term planning patterns during inference ( 20 ; 19 ; 5 ) , suggesting a gap between the systematic planning abilities LLMs acquire through training and the behaviors they actually exhibit at inference time. At present, the mechanism underlying this gap remains poorly understood. In principle, next - token training should equivalently induce sequence - level planning ( 2 ) . Given an prompt sequence I_{\mathcal{C}} sampled from training corpus \mathcal{C} , the pretraining objective that determines the probability assigned to next tokens can be expressed probabilistically as: P(x|I_{\mathcal{C}})=\int_{s}P(x|s)P(s|I_{\mathcal{C}})ds (1) where s ranges over all possible response sequences consistent with I_{\mathcal{C}} . LLMs are thereby encouraged to consider the full set of candidate sequences s before pinning down the distribution over next token x . This perspective aligns with empirical findings showing that future information is more accurately decoded from early - and mid - layer activations, whereas accuracy on the immediate next token peaks only at the final layer ( 7 ; 20 ; 5 ) . Notably, 14 reported a similar “plan-first” pattern in a chess-playing neural network, suggesting that such planning behavior may arise generally in models trained on sequential data. However, inference - time behavior of LLMs appears to be more complex. Empirical work that decodes future tokens from early neural activations suggests that their planning capacities primarily govern short - term future generations rather than extending uniformly across the whole response ( 20 ; 19 ) . This raises a central question we aim to answer in this paper: if LLMs are trained on \mathcal{C} to perform sequence - level planning as expressed in Equation 1 , what constrains their ability to fully plan ahead at inference. A key distinction at inference time lies in the inherent mismatch between the distribution of the training corpus \mathcal{C} (i.e., natural language produced by humans) and LLM-generated data \mathcal{M} (what LLMs actually represent). Prior work has documented such discrepancies both in specific tasks ( 9 ) and in broader text - generation settings ( 1 ; 31 ) . These findings suggest that, even trained extensively on human data, the generative process of LLMs introduces systematic deviations from the training distribution, yielding stylized outputs that enable a high rate of self-recognition ( 21 ) . Consequently, an arbitrary human - written prompt I_{\mathcal{C}} may constitute an out - of - distribution input for the model at inference time. Yet whether—and in what ways—such out - of - distribution prompts compromise planning behavior remains unknown. With this in mind, our aim in this paper is to reconcile LLMs’ theoretical planning abilities with their actually planning behavior at inference time, with particular attention to how this behavior is shaped by the dynamically evolving context. In the remainder of the paper, we first introduce a Bayesian perspective on sequence - level planning in LLMs, clarifying how such planning is deployed at inference in ways that can lead to compromised planning behavior. Building on this framework, we derive previously unreported dynamic patterns of planning and then test these predictions empirically. In Experiment 1, using a random - generation task, we confirm the presence of constrained planning under human - written prompts and validate the prediction that planning shifts and converges as in - distribution self - generated text accumulates during inference. In Experiment 2, we employ a Gaussian - sampling task with ground - truth structure to further verify how prompts dynamically shape the evolution of the model’s planning. Overall, our work provides both theoretical perspective and empirical evidence that deepen our understanding of how LLMs plan at inference. A Bayesian Account of Sequence-level Planning Meta-learned Bayesian Prior LLMs are generally known as excellent few-shot ( 3 ) or even zero-shot learners ( 16 ; 27 ) . That is, they can solve a variety of tasks by simply conditioning on a few task examples or merely instructions of the task. Meta-learning over prior across tasks provides a principled explanation for such quick adaption to sparse or no new data ( 6 ; 8 ; 15 ) . Suppose a domain-specific generation task distribution \phi=\{\tau_{1},\tau_{2},\dots\tau_{n}\} is embedded in the training data. Combined with the training objective in Equation 1 , an LLM trained to capture such a domain - specific planning prior can be described as P(s|\phi)=\int_{\tau}P(s|\tau,\phi)d\tau (2) The priors encoded by LLMs have been shown to align closely with those employed by humans ( 13 ) . Given such priors, Bayesian inference over response sequences becomes possible, enabling LLMs to perform few - shot and even zero - shot learning. At inference time given a task prompt I associated with domain \phi , the model infers a response sequence s by combining the encoded domain prior with the prompt information in a Bayesian manner: P(s|I,\phi)\propto\underbrace{P(I|s,\phi)}_{\text{planning likelihood}}\;\cdot\underbrace{P(s|\phi)}_{\text{domain prior}} (3) The planning term P(I|s,\phi) measures how well the prompt I is explained by a planned sequence s under domain \phi , whereas the domain prior P(s|\phi) reflects the distribution over sequences induced solely by the domain - level structure learned during training. Figure 1: Scheme illustrating the evolving planning behavior of LLMs during inference. Compromised Planning during Inference Inherent differences between natural - language prompts I_{\mathcal{C}} and truly internalized language I_{\mathcal{M}} may underlies LLMs’ compromised planning observed at inference time through Equation 3