Thinking in Latents: Adaptive Anchor Refinement for Implicit Reasoning in LLMs
Disha Sheshanarayana, Rajat Subhra Pal, Manjira Sinha, Tirthankar Dasgupta, [0.45em] Manipal University Jaipur · 2026
Read Thinking in Latents: Adaptive Anchor Refinement for Implicit Reasoning in LLMs by Disha Sheshanarayana, Rajat Subhra Pal, Manjira Sinha, Tirthankar Dasgupta, [0.45em] Manipal University Jaipur (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2603-15051.txt.
Introduction
Sheshanarayana and co-authors present AdaAnchor, a way for language models to reason silently in latent space. Instead of writing long chains of thought, the model refines a set of latent anchor vectors tied to the input. A halting rule watches the anchors settle and stops early on easy problems. Hard problems get more steps under a shared budget. On three math word-problem benchmarks, accuracy rose by up to 5 percent. Average refinement steps fell by about half, and output tokens by over 90 percent.
Catalog record
| CREATOR | Disha Sheshanarayana, Rajat Subhra Pal, Manjira Sinha, Tirthankar Dasgupta, [0.45em] Manipal University Jaipur |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 28046 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2603.15051
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Thinking in Latents: Adaptive Anchor Refinement for Implicit Reasoning in LLMs Disha Sheshanarayana, Rajat Subhra Pal, Manjira Sinha, Tirthankar Dasgupta, [0.45em] Manipal University Jaipur Abstract Token-level Chain-of-Thought (CoT) prompting has become a standard way to elicit multi-step reasoning in large language models (LLMs), especially for mathematical word problems. However, generating long intermediate traces increases output length and inference cost, and can be inefficient when the model could arrive at the correct answer without extensive verbalization. This has motivated latent-space reasoning approaches that shift computation into hidden representations and only emit a final answer. Yet, many latent reasoning methods depend on a fixed number of latent refinement steps at inference, adding another hyperparameter that must be tuned across models and datasets to balance accuracy and efficiency. We introduce AdaAnchor, a latent reasoning framework that performs silent iterative computation by refining a set of latent anchor vectors attached to the input. AdaAnchor further incorporates an adaptive halting mechanism that monitors anchor stability across iterations and terminates refinement once the anchor dynamics converge, allocating fewer steps to easier instances while reserving additional refinement steps for harder ones under a shared maximum-step budget. Our empirical evaluation across three mathematical word-problem benchmarks shows that AdaAnchor with adaptive halting yields accuracy gains of up to 5% over fixed-step latent refinement while reducing average latent refinement steps by 48–60% under the same maximum-step budget. Compared to standard reasoning baselines, AdaAnchor achieves large reductions in generated tokens (92–93%) by moving computation into silent latent refinement, offering a different accuracy–efficiency trade-off with substantially lower output-token usage. 1 Introduction Large Language Models (LLMs) have demonstrated strong capabilities in mathematical reasoning, particularly when prompted to produce explicit intermediate traces such as Chain-of-Thought (CoT) ( 23 ) . Recent progress has further amplified these gains through improved instruction-tuning and longer reasoning trajectories, reinforcing the view that additional thinking tokens can unlock better problem-solving behavior ( 11 ; 22 ) . Despite these advances, a practical limitation persists: lengthy reasoning traces are computationally expensive, increasing decoding latency and token usage and raising serving cost especially under high-concurrency deployments. This trade-off motivates methods that preserve the benefits of multi-step reasoning while reducing the cost of token-level generation. Several research studies have emerged to address this computational challenge. One line of research focuses on improving efficiency within the token level, for example by encouraging more concise rationales, skipping less informative tokens, or terminating early when the model appears confident in a candidate answer ( 6 ) . While valuable, these methods remain coupled to sparse, discrete token generation and therefore inherit the cost structure of autoregressive decoding. A more promising direction explores reasoning within the dense latent space, where models can perform computation internally and emit only the final answer. Prior work has pursued latent reasoning via distillation and curriculum strategies, by reusing computation through layer skipping or looping ( 18 ) , or by partially replacing token-level traces with latent representations ( 9 ; 25 ; 4 ; 19 ) . However, many latent reasoning methods still depend on a fixed number of latent refinement steps at inference, introducing another hyperparameter that must be tuned across models and datasets to balance accuracy and efficiency ( 9 ; 4 ; 3 ; 19 ) . Figure 1: Comparison of AdaAnchor with explicit Chain-of-Thought (CoT) reasoning. CoT generates long intermediate reasoning tokens, whereas AdaAnchor performs implicit multi-step computation by refining latent anchor vectors and uses stability-based early stopping before answer-only decoding. To overcome these limitations, we introduce AdaAnchor, an implicit reasoning framework that performs silent iterative computation by refining a compact set of latent anchor vectors attached to the input. Instead of generating intermediate reasoning tokens, AdaAnchor updates these anchors using the model’s hidden activations while keeping the output in an answer-only format. Furthermore, we propose an adaptive halting strategy for latent refinement: AdaAnchor tracks anchor stability across iterations and stops updating when the dynamics converge, thereby allocating computation on a per-instance basis. This enables instance-wise compute allocation under a shared maximum-step budget, so easier problems terminate quickly while harder problems receive additional refinement without requiring a fixed latent step count to be tuned per dataset ( 8 ) . We evaluate AdaAnchor on three mathematical word-problem benchmarks, GSM8K, SVAMP, and MultiArith, and compare it against fixed-step latent refinement and standard reasoning baselines under a shared maximum latent budget ( 2 ; 15 ; 17 ) . The results show that convergence-driven adaptive halting improves the efficiency–accuracy trade-off relative to fixed-step refinement by avoiding unnecessary latent iterations, while answer-only decoding reduces generation overhead compared to token-based baselines. Together, these findings indicate that stability-aware termination is a practical mechanism for controlling implicit computation and improving the deployability of latent reasoning methods. 2 Related Work 2.1 Explicit CoT reasoning Large Language Models (LLMs) often exhibit substantially stronger mathematical reasoning when prompted to generate explicit intermediate traces, most notably through Chain-of-Thought (CoT) style prompting 23 . Follow-up work showed that even without exemplars, carefully designed prompts can elicit multi-step reasoning behaviors 11 , and decoding-time strategies such as self-consistency can further improve reliability by aggregating over diverse reasoning paths 22 . At the same time, generating long token-level rationales can be costly in practice, motivating methods that reduce the number of produced tokens while preserving reasoning quality. Representative directions include introducing structured pause mechanisms that encourage internal computation without fully verbalizing every step 6 , as well as prompting frameworks that constrain or compress intermediate traces to be shorter and cheaper to decode 1 ; 24 . Orthogonally, improvements at the training objective level—such as predicting multiple future tokens per position have been explored as a way to increase throughput and reduce inference time without changing the basic autoregressive interface 5 . 2.2 Latent reasoning To avoid the overhead of long textual traces, latent reasoning approaches shift more computation into hidden representations and emit only the final answer. A prominent line of work performs silent reasoning by feeding internal hidden states back into the model as continuous thoughts, enabling multi-step computation without committing to discrete tokens at every step 9 . Related approaches learn latent reasoning trajectories via distillation or self-training so that intermediate computation is represented implicitly rather than as explicit text 19 . More recently, hybrid schemes have explored mixing latent and text tokens during training and inference, aiming to retain some controllability and interpretability while still gaining efficiency from latent computation 20 . In parallel, latent-space compression methods seek to reduce the length of reasoning traces by compressing multiple reasoning tokens into fewer latent steps, improving efficiency while keeping performance competitive 21 . Another complementary direction analyzes or exploits latent dynamics during generation to guide or improve reasoning behavior 25 . 2.3 Adaptive halting and compute allocation A recurring limitation in iterative (token or latent) reasoning is that many methods require fixing the number of refinement steps in advance, which can lead to over-computation on easy instances and under-computation on hard ones. Adaptive computation mechanisms address this by allowing models to halt dynamically based on instance difficulty, with classic formulations such as Adaptive Computation Time (ACT) learning when to stop iterating 8 . Prior work on latent and implicit reasoning often (i) runs a fixed number