Domain-Specialized Tree of Thought through Plug-and-Play Predictors
Xuanqi Gao, Haoyu Wang, Jun Sun, Shiqing Ma, Chao Shen · 2026
Read Domain-Specialized Tree of Thought through Plug-and-Play Predictors by Xuanqi Gao, Haoyu Wang, Jun Sun, Shiqing Ma, Chao Shen (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2603-20267.txt.
Introduction
Gao and co-authors make tree of thought reasoning cheaper with a small plug and play predictor. Tree search explores many branches, but model based scoring of each branch costs a lot. Rigid heuristics are cheaper yet inflexible across tasks. Their DST predictor is a light supervised model that guides the search. It prunes in a context aware way, staying near greedy on easy steps. The beam widens only when the predictor senses uncertainty or complexity. Accuracy matched or beat strong baselines while compute fell by 26 to 75 percent.
Catalog record
| CREATOR | Xuanqi Gao, Haoyu Wang, Jun Sun, Shiqing Ma, Chao Shen |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 55597 |
| LICENSE | http://creativecommons.org/licenses/by-sa/4.0/ |
| RIGHTS | CC BY SA |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2603.20267
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Domain-Specialized Tree of Thought through Plug-and-Play Predictors Xuanqi Gao, Haoyu Wang, Jun Sun, Shiqing Ma, Chao Shen Abstract While Large Language Models (LLMs) have advanced complex reasoning, prominent methods like the Tree of Thoughts (ToT) framework face a critical trade-off between exploration depth and computational efficiency. Existing ToT implementations often rely on heavyweight LLM-based self-evaluation or rigid heuristics for branch pruning, making them prohibitively expensive and inflexible for broad application. To address this, we introduce DST , an adaptable, plug-and-play predictor that serves as a lightweight, supervised heuristic to guide the ToT search process. Our predictor enables dynamic, context-aware pruning, allowing the search to proceed with near-greedy efficiency on simpler reasoning steps while adaptively expanding the search beam only when encountering uncertainty or task complexity. We evaluate our approach on a diverse suite of benchmarks spanning mathematical reasoning, general reasoning, and complex logical reasoning. Experimental results demonstrate that our method achieves accuracy competitive with or superior to strong baselines, including standard ToT, while reducing computational overhead by 26-75%. Our work effectively resolves the accuracy-efficiency trade-off in tree-based reasoning, transforming ToT from a resource-intensive technique into a scalable and practical paradigm for complex problem-solving in LLMs. 1 Introduction Large Language Models (LLMs) have demonstrated remarkable reasoning capabilities across diverse domains, ranging from mathematics and programming to planning and scientific discovery. By using chain-of-thought prompting ( 27 ) , tool use ( 20 ; 6 ) , and multi-agent collaboration ( 28 ) , recent advances have pushed LLMs beyond simple pattern matching toward complex problem solving. Despite this progress, reasoning with LLMs remains imperfect. Models often produce incorrect intermediate steps, pursue unproductive solution paths, or become trapped in lengthy reasoning chains ( 32 ) . Several approaches have been proposed to improve LLM reasoning capability. For post-training methods such as reinforcement learning with human feedback ( 21 ; 18 ; 22 ) , models are optimized to better follow human preferences. While effective, such approaches are computationally costly, requiring expensive fine-tuning runs. On the other hand, test-time methods enhance reasoning without modifying model parameters. For instance, the Tree of Thoughts (ToT) ( 30 ) framework extends stepwise reasoning into a tree search, where each partial reasoning step is assigned a score reflecting its promise toward solving the task. The scores are used to determine which nodes to expand and which branches to prune, allowing the model to concentrate its computation on the most promising reasoning paths. A number of recent works have extended the Tree-of-Thoughts (ToT) paradigm ( 30 ) by incorporating different reasoning guidance. ProbTree ( 2 ) employs probabilistic scoring, while DPTS ( 4 ) leverages confidence estimates and AGoT ( 16 ) adapts task-specific heuristics. Other variants introduce interactive designs, such as iToT ( 1 ) with tool-cost awareness and MA-ToT ( 8 ) using validator agents. Preference-based methods have also emerged, including BPP-Search ( 25 ) and CPO ( 33 ) , as well as retrieval-augmented approaches like RATT ( 31 ) . However, evaluating whether a partial chain is promising at test time is challenging. First, it must be lightweight, since relying on repeated LLM self-evaluation is prohibitively expensive and introduces significant computational overhead ( 14 ) . Second, it should be easily adaptable to new domains; methods that depend on manually crafted rules or rigid, task-specific verifiers lack such flexibility and require extensive engineering efforts ( 5 ) . Finally, it must effectively forecast the potential utility of partial reasoning chains. Prior work such as DPTS ( 4 ) relies on local confidence scores to guide parallel expansion. However, confidence alone does not necessarily predict the future utility of a reasoning path, since confident steps may still lead to hallucinations or unproductive exploration. To address the above challenges, we propose DST , which extends ToT framework by introducing a novel adaptable plug-and-play predictor that enables efficient control over the ToT search process. As illustrated in Figure 1 , the predictor serves as a heuristic, supervised scorer, making immediate, context-aware judgments for branch selection at each step in the search. Specifically, at each search step, our predictor evaluates the initial generated thought and assigns it a confidence score. It is important to note that this predictor requires white-box access to the backbone LLM, specifically the ability to extract hidden states from the model’s forward pass. This requirement arises because the semantic representations used by our predictor are derived directly from the model’s internal activations. While this precludes application to closed-source API-only models, the proliferation of high-quality open-weight LLMs (e.g., Llama, Qwen, Gemma) ensures broad practical applicability. If this score exceeds a predefined threshold, the system commits to this “good-enough” path greedily, effectively behaving like a single-chain reasoner and avoiding the cost of generating further alternatives. Conversely, if the score falls below the threshold, indicating uncertainty or a complex decision point, the system dynamically expands the search to a full beam, preserving the robust exploration and error-correction capabilities of traditional ToT. Figure 1 : Overview of DST . We validate our approach on several reasoning challenges—including mathematical reasoning (MATH500 ( 13 ) , GSM8K ( 3 ) , Minerva-Math ( 12 ) , SVAMP ( 17 ) ), general reasoning (GPQA ( 19 ) ), and complex logical reasoning (BBEH ( 9 ) ) using state-of-the-art LLMs. Results confirm that our method achieves accuracy competitive with or superior to standard ToT baselines while reducing token consumption by 26-75%. In summary, our work transforms ToT reasoning from an efficiency bottleneck into a fast, widely deployable paradigm, making structured search feasible anywhere LLM inference is used. Key highlights of our approach: • Novel Predictor Architecture. Unlike prior methods that rely on LLM self-evaluation ( 30 ) , confidence scores ( 4 ) , or probabilistic scoring ( 2 ) , our predictor uniquely combines semantic embeddings with a learned consistency score, enabling assessment of both content quality and logical coherence without step-level supervision. • Efficiency. The predictor prunes unpromising branches during search, reducing token costs by 26-75% while maintaining or even increasing accuracy on popular benchmarks—a 2-4 \times improvement over the most efficient prior ToT variants such as DPTS ( 4 ) . • Plug-and-Play & Domain-Specific. The predictor is decoupled from the backbone LLM, requiring only lightweight domain-specific training on a small dataset (20-200 seed problems), making it easily transferable across various domains such as math, general QA, and program synthesis. • Adaptive Search. Unlike fixed-beam approaches ( 25 ; 33 ) , DST dynamically adjusts its search breadth based on the predictor’s real-time confidence, achieving near-greedy efficiency when confident and full-beam robustness when uncertain. Table 1 : Comparison of Tree-of-Thought reasoning methods. Method Dynamic Branching Evaluation Signal Token Cost (vs. CoT) Domain Adaptation ToT ( 30 ) ✗ LLM Self-Eval \sim 10x ✗ ProbTree ( 2 ) ✗ Probabilistic \sim 5x ✗ DPTS ( 4 ) ✓ Confidence \sim 4x ✗ AGoT ( 16 ) ✓ Task-specific \sim 4x ✗ iToT ( 1 ) ✓ Tool-cost aware \sim 3x ✗ BPP-Search ( 25 ) ✓ RL preference \sim 3x-5x ✗ MA-ToT ( 8 ) ✓ Multi-agent validator \sim 4x ✗ RATT ( 31 ) ✓ Retrieval score \sim 4x ✗ CPO ( 33 ) ✓ Preference optim.