The Geometric Reasoner: Manifold-Informed Latent Foresight Search for Long-Context Reasoning
Ren Zhuang, Ben Wang, Shuifa Sun · 2026
Read The Geometric Reasoner: Manifold-Informed Latent Foresight Search for Long-Context Reasoning by Ren Zhuang, Ben Wang, Shuifa Sun (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2601-18832.txt.
Introduction
Ren Zhuang, Ben Wang, and Shuifa Sun build a thought tool that plans its search by the shape of its own hid space. Long chains of thought can crack hard tasks, but each new step costs time and store. Past tools branch their search wide and waste steps on near twin paths. Or they train a guide at great cost to tell good paths from bad.
Their tool, called the Geometric Reasoner, needs no training. It reads the shape of the space where the model's thoughts sit as points. Good new steps are those that run with that shape's grain. The search stays in a tight store cap while it roams far. Such a search could give long thought at a fair cost. It may serve tasks where the store cap is the hard wall.
Catalog record
| CREATOR | Ren Zhuang, Ben Wang, Shuifa Sun |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 34 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2601-18832
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
The Geometric Reasoner: Manifold-Informed Latent Foresight Search for Long-Context Reasoning Ren Zhuang, Ben Wang, Shuifa Sun Abstract Scaling test-time compute enhances long chain-of-thought (CoT) reasoning, yet existing approaches face a fundamental trade-off between computational cost and coverage quality: either incurring high training expense or yielding redundant trajectories. We introduce The Geometric Reasoner (TGR), a training-free framework that performs manifold-informed latent foresight search under strict memory bounds. At each chunk boundary, TGR scores candidate latent anchors via a lightweight look-ahead estimate combined with soft geometric regularizers that encourage smooth trajectories and diverse exploration. Chunk-wise KV cache resets keep memory linear in chunk length. On challenging math and code benchmarks, TGR improves robust trajectory coverage, measured by the area under the Pass@ k k curve (AUC), by up to 13 points on Qwen3-8B, with negligible overhead of about 1.1–1.3 × \times . \icml@noticeprintedtrue † † footnotetext: \Notice@String 1 Introduction Scaling test-time compute reliably enhances large language model (LLM) performance, unlocking reasoning capabilities previously out of reach ( 42 ; 27 ; 48 ; 40 ) . Yet turning extra compute into diverse exploration without incurring prohibitive memory costs or wasting budget on redundant, correlated trajectories remains difficult. The quadratic complexity of attention and the footprint of KV caching ( 35 ; 4 ; 11 ; 12 ) bottleneck practical deployment, rendering naive long-horizon exploration intractable. Existing approaches make progress on this tension, yet critical gaps persist. Reinforcement Learning (RL) and preference optimization amortize long-horizon control into model weights ( 30 ; 10 ; 31 ; 50 ) . While effective for single-sample accuracy, these methods demand substantial training compute and can collapse the trajectory distribution ( 46 ; 33 ) . Sampling-based inference trades compute for robust coverage without retraining ( 37 ; 24 ; 14 ) , yet returns diminish rapidly as additional samples often remain correlated and fail to expand the set of distinct, valid trajectories ( 34 ; 40 ) . These limitations motivate a training-free inference-time search procedure that allocates compute dynamically under strict memory bounds. Figure 1 : TGR-Latent consistently outperforms baselines on Qwen3-8B. Manifold-informed latent foresight search steering converts modest inference-time compute into robust coverage without weight updates. To address this, we introduce The Geometric Reasoner (TGR) , a training-free framework for manifold-informed latent foresight search. TGR formulates reasoning as path search on a latent manifold but avoids rigid constraints that become brittle in high dimensions ( 43 ; 6 ; 17 ) . Instead, it ranks candidate reasoning chunks via a soft geometric score combining lightweight look-ahead utility, a bumpiness penalty for smooth transitions, and a repulsive diversity force that discourages similarity. Chunk-wise KV cache resets keep memory linear in chunk length, enabling scalable long-context exploration. This chunk-boundary state interface relates to markovian control approaches that learn explicit state transitions, like Delethink ( 3 ) . Rather than relying on learned state transitions or external verifiers, TGR enables training-free inference-time search, allocating compute to achieve controllable exploration and robust coverage under matched budgets. Our contributions are: • Manifold-Informed Latent Foresight Search. We propose TGR, a training-free inference-time framework that scores chunk-level latent anchors with a soft geometric objective, enabling controllable exploration without parameter updates. • Budget-Efficient Robustness. TGR improves robust trajectory coverage on challenging math and code benchmarks, achieving up to 13-point AUC (area under the Pass@ k curve) gains on Qwen3-8B with negligible overhead of about 1.1–1.3 \times , as shown in Figure 1 . • Scaling Limits of Hard Geometry. We demonstrate that hard geometric filtering suffers from vanishing feasibility in high-dimensional latent spaces, where the acceptance rate decays exponentially as dimension increases, establishing soft geometric regularization as a scalable alternative. 2 Related Work Figure 2 : Overview of reasoning frameworks. Unlike (a) test-time sampling , exploring trajectories without explicit structure, or (b) reinforcement learning , which internalizes preferences through costly training, (c) TGR introduces a training-free inference-time search over the latent manifold. It selects optimal chunk-level anchors via a soft geometric score combining foresight, bumpiness, and uniformity, then injects them into generation with KV cache resets to maintain linear memory cost. 2.1 Test-time Reasoning Prompt engineering steers model behavior via input formatting and exemplars, spanning in-context learning ( 8 ) , Chain-of-Thought decomposition ( 38 ; 25 ) , and program-aided methods such as PAL ( 16 ) . These approaches improve controllability without additional training but remain sensitive to prompt design and operate via token-level heuristics. Sampling-based reasoning includes local decoding controls (temperature, top- k , top- p ), candidate pooling with Best-of- N selection ( 36 ) , and aggregation methods such as Self-Consistency ( 37 ) . Aggressive test-time sampling shows that additional inference compute can elicit latent reasoning without retraining ( 24 ) . Approaches such as Tree of Thoughts (ToT) ( 45 ) and Graph of Thoughts (GoT) ( 5 ) exemplify structured inference, which introduces explicit search in token space and trades computation for robustness. However, these methods search over discrete continuations and rely on token-space proxies or external scoring, rendering long-horizon search brittle under limited budgets. 2.2 Reinforcement Learning and Preference Optimization Amortizing long-horizon control into model parameters via reinforcement learning and preference optimization has become a dominant paradigm ( 10 ; 47 ; 39 ) . Beyond PPO ( 30 ) and DPO ( 29 ) , recent methods including GRPO ( 31 ) and GSPO ( 50 ) scale reasoning by reducing reliance on a learned critic. Delethink ( 3 ) learns discrete state transitions, demonstrating the utility of explicit Markovian states for long-horizon control. While effective, reward-and-preference-driven training exhibits failure modes at reduced diversity and overconfident generations ( 46 ; 33 ) . SimKO ( 28 ) mitigates mode collapse and length bias, yet these training-centric approaches remain computationally intensive. 2.3 Geometric Perspectives on Latent Modeling Geometric Deep Learning interprets learned representations as structured spaces ( 7 ; 21 ) . Recent works frame attention as a geometric object governing representation transport along a curved manifold, treating layers as discrete geodesic steps shaped by training ( 2 ; 13 ; 51 ) . Complementing this view, LLM decoding is interpreted as tracing trajectories on a low-dimensional manifold, where coherent generations follow smooth paths and hallucinations appear as abrupt jumps ( 49 ) . These insights motivate geometric inductive biases at training time: manifold priors ( 23 ) and architectural constraints such as mHC ( 41 ) , which applies Sinkhorn–Knopp normalization ( 32 ) to confine signal propagation to a stable low-dimensional structure. Soft enforcement of geometry is preferred over hard feasibility filtering, which becomes impractical in high dimensions due to concentration of measure ( 43 ; 6 ; 17 ) . TGR operationalizes this insight at inference time, translating geometric priors into differentiable soft penalties without architectural changes. Figure 2 contrasts TGR with test-time reasoning and RL methods, highlighting its training-free, manifold-informed latent foresight search. 3 Method TGR is a training-free geometric reasoner that operates in latent space. It keeps the backbone LLM frozen, segments reasoning into chunks, and performs search at chunk boundaries within a continuous anchor space. At each boundary, TGR samples candidate anchors, scores them using computationally efficient rollouts and soft geometric penalties, and selects the highest-scoring anchor