Intention Collapse: Intention-Level Metrics for Reasoning in Language Models
P. M. Vera · 2026
Read Intention Collapse: Intention-Level Metrics for Reasoning in Language Models by P. M. Vera (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2601-01011.txt.
Introduction
Patricio Vera studies the moment a language model's inner intent collapses into one stream of words. Generation maps a rich internal state to a single token sequence, a many-to-one projection Vera calls intention collapse. He defines three cheap metrics on the state just before collapse: intention entropy, effective dimensionality, and recoverability, the last measuring how well a probe can predict eventual success from that state.
The metrics were evaluated across three models and three reasoning benchmarks, comparing plain generation, chain-of-thought prompting, and a babble control. Chain-of-thought raised average accuracy substantially, driven by large gains on grade-school math but consistent losses on a science challenge set. The metrics are model-agnostic and meant to diagnose reasoning quality before any answer appears.
Catalog record
| CREATOR | P. M. Vera |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 88 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2601-01011
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Intention Collapse: Intention-Level Metrics for Reasoning in Language Models P. M. Vera Abstract Language generation maps a rich, high-dimensional internal state to a single token sequence. We study this many-to-one mapping through the lens of intention collapse : the projection from an internal intention space I I to an external language space L L . We introduce three cheap, model-agnostic metrics computed on a pre-collapse state I I : (i) intention entropy H int ( I ) H_{\mathrm{int}}(I) , (ii) effective dimensionality dim eff ( I ) \mathrm{dim}_{\mathrm{eff}}(I) , and (iii) recoverability Recov ( I ) \mathrm{Recov}(I) , operationalized as probe AUROC for predicting eventual success. We evaluate these metrics in a 3 × 3 3\times 3 study across models (Mistral-7B, LLaMA-3.1-8B, Qwen-2.5-7B) and benchmarks (GSM8K, ARC-Challenge, AQUA-RAT), comparing baseline, chain-of-thought (CoT), and a babble control ( n = 200 n=200 items per cell). CoT increases average accuracy from 34.2 % 34.2\% to 47.3 % 47.3\% (+ 13.1 13.1 pp), driven by large gains on GSM8K but consistent degradations on ARC-Challenge. Across models, CoT induces distinct entropy regimes relative to baseline, Δ H = H int ( CoT ) − H int ( Base ) \Delta H=H_{\mathrm{int}}(\mathrm{CoT})-H_{\mathrm{int}}(\mathrm{Base}) : Mistral shows Δ H < 0 \Delta H<0 (lower-entropy CoT), whereas LLaMA shows Δ H > 0 \Delta H>0 (higher-entropy CoT), highlighting heterogeneity in CoT-induced internal uncertainty. Finally, probe AUROC is significantly above chance in a subset of settings and can dissociate from behavioral accuracy (e.g., high AUROC alongside lower CoT accuracy on ARC-Challenge for Qwen), suggesting that informative internal signal is not always reliably converted into a final discrete decision under certain response formats. Keywords: language models; chain-of-thought; internal representations; entropy; effective dimensionality; probing; AUROC; reasoning; response format 1 Introduction Every act of human communication begins in a space far richer than language itself. Before a speaker utters a single word, what exists is a high-dimensional, pre-linguistic manifold of memories, emotions, goals, analogies, and half-formed arguments. From this unbounded cloud of potential meanings, only one linear sequence of words is ultimately produced. The same internal state could have been expressed as “I’m exhausted,” “I barely slept,” “My brain is mush today,” or infinitely many other variants—yet exactly one utterance crystallizes at this particular moment. We term this irreversible reduction intention collapse : the process by which a cognitive system—biological or artificial—compresses a vast, implicit configuration of meaning into a single concrete linguistic message. Intention collapse is not a peripheral byproduct of language generation; it is a structural feature of any system that must commit a rich internal state to a discrete, serial surface. Contemporary large language models (LLMs) exhibit a closely analogous pattern. Beneath the familiar formalism of next-token prediction lies a two-stage computation: first, the model integrates the input with parameters and context to form a high-dimensional internal representation; then, this representation is projected into a discrete token sequence. We formalize this as an intention state I —the prompt-conditioned latent representation immediately before the first token is emitted—and a collapse operator \kappa_{\theta} that maps I into language. The mapping is many-to-one and inherently lossy: information present in I may not survive the projection into the output. This framing offers a unifying lens on a diverse landscape of reasoning-enhancing techniques. Chain-of-thought prompting ( 36 ) , self-taught reasoning ( 40 ) , process reward models ( 21 ) , test-time training ( 31 ) , and self-refinement methods ( 24 ) can be viewed as interventions that alter what the model “brings to the brink” of generation—i.e., the informational and geometric properties of I prior to collapse. Making I explicit lets us ask not only whether a method improves benchmark scores, but how it reshapes the pre-collapse state and when such reshaping helps or harms performance. To operationalize this framework, we propose three simple, model-agnostic intention metrics: • Intention entropy H_{\text{int}}(I) : the Shannon entropy of the next-token distribution at the pre-collapse moment, measuring how concentrated or diffuse the model’s immediate continuation is. • Effective dimensionality d_{\text{eff}}(I) : the participation ratio of PCA eigenvalues over pre-collapse hidden states, capturing the geometric richness of the intention representation. • Latent knowledge recoverability \text{Recov}(I;Z) : the performance of linear probes trained on I to predict task outcomes (e.g., answer correctness), quantifying information present pre-collapse that may fail to be expressed in the final output. These metrics are cheap to compute, require no architectural modifications, and can be applied to any transformer-based model. Empirical contributions. We instantiate the intention collapse framework in a systematic empirical study spanning: • Three model families: Mistral-7B-Instruct, Llama-3.1-8B-Instruct, and Qwen-2.5-7B-Instruct, covering diverse instruction-tuned model families. • Three reasoning benchmarks: GSM8K (free-response grade-school math), ARC-Challenge (multiple-choice abstract reasoning), and AQUA-RAT (multiple-choice math word problems), chosen to contrast response formats and problem structures. • Three inference regimes: direct-answer baseline, chain-of-thought (CoT), and a babble control for verbosity confounds. Across this 3\times 3 matrix, we find heterogeneous effects that sharpen the intention-collapse perspective: • CoT is not universally beneficial. CoT yields large gains on GSM8K across all three models, but consistently degrades performance on ARC-Challenge across all three models, and produces mixed outcomes on AQUA-RAT (including strong improvements for some models and substantial drops for others). This motivates separating changes in internal state from the reliability of the final discrete commitment. • CoT induces distinct internal entropy regimes across models. When comparing CoT to baseline via \Delta H=H_{\text{int}}(\text{CoT})-H_{\text{int}}(\text{Base}) , we observe qualitatively different regimes: Mistral operates under lower pre-collapse entropy in CoT ( \Delta H<0 ), while LLaMA operates under higher pre-collapse entropy in CoT ( \Delta H>0 ). These labels describe comparative uncertainty under CoT vs baseline and do not assume token-level temporal dynamics. • Response format is a major axis of variation. Large within-domain gaps between free-response and multiple-choice settings (e.g., math in GSM8K vs math in AQUA-RAT) indicate strong response-format sensitivity, with cross-model reversals suggesting that format preference is not uniform across model families. • Recoverability signal is real but can dissociate from accuracy. Linear probes trained on I achieve AUROC above chance in a subset of settings; strikingly, high AUROC can co-occur with degraded CoT accuracy in multiple-choice regimes, suggesting that an informative internal signal may fail to be reliably converted into a clean final decision under certain formats. • Babble helps isolate verbosity effects, but does not replicate CoT signatures. Despite producing long outputs, babble does not systematically reproduce the same metric–performance relationships observed under CoT, supporting the view that CoT changes more than output length. We also report important limitations. Item-level predictive power of H_{\text{int}}(I) is weak in several settings; probe performance varies across layers, tasks, and models; and the relationship between intention metrics and downstream accuracy is correlational rather than causal. Moreover, CoT substantially increases output length relative to baseline, introducing a compute confound that must be considered when interpreting accuracy gains. Conceptual contributions. Beyond the empirical findings, we argue that intention collapse provides a useful vocabulary for reasoning about LLM behavior: • It reframes diverse inference-time methods as interventions on a common object ( I ), enabling direct comparison via shared metrics. • It connects to parallel literatures on latent knowledge elicitation ( 4 ) , pre-generation