T 2 : An Adaptive Test-Time Scaling Strategy for Contextual Question Answering

Zhengyi Zhao, Shubo Zhang, Zezhong Wang, Huimin Wang, Yutian Zhao, Bin Liang, Yefeng Zheng, Binyang Li, Kam-Fai Wong, Xian Wu · 2025

Read T 2 : An Adaptive Test-Time Scaling Strategy for Contextual Question Answering by Zhengyi Zhao, Shubo Zhang, Zezhong Wang, Huimin Wang, Yutian Zhao, Bin Liang, Yefeng Zheng, Binyang Li, Kam-Fai Wong, Xian Wu (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2505-17427.txt.

Introduction

Zhengyi Zhao and nine co-authors presented T2 in twenty twenty-five, a strategy that adapts how long a language model thinks before answering a question with context. Standard systems reason elaborately on every question, wasting compute on easy ones; blunt budget caps and early stops inject human bias instead. Think-to-Think instead lets the model learn from its own kin.

The method breaks the question into structural elements, generates similar questions, and tries candidate reasoning strategies on them. The strategy that scores best across several criteria is then applied to the real question. Across seven benchmarks, T2 raised accuracy over baselines while cutting compute by up to about twenty-five percent. Simple questions get lean reasoning, hard ones keep the full treatment.

Catalog record

CREATORZhengyi Zhao, Shubo Zhang, Zezhong Wang, Huimin Wang, Yutian Zhao, Bin Liang, Yefeng Zheng, Binyang Li, Kam-Fai Wong, Xian Wu
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE72 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2505-17427

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

T 2 : An Adaptive Test-Time Scaling Strategy for Contextual Question Answering Zhengyi Zhao, Shubo Zhang, Zezhong Wang, Huimin Wang, Yutian Zhao, Bin Liang, Yefeng Zheng, Binyang Li, Kam-Fai Wong, Xian Wu Abstract Recent advances in Large Language Models (LLMs) have demonstrated remarkable performance in Contextual Question Answering (CQA). However, prior approaches typically employ elaborate reasoning strategies regardless of question complexity, leading to low adaptability. Recent efficient test-time scaling methods introduce budget constraints or early stop mechanisms to avoid overthinking for straightforward questions. But they add human bias to the reasoning process and fail to leverage models’ inherent reasoning capabilities. To address these limitations, we present T 2 : Think-to-Think, a novel framework that dynamically adapts reasoning depth based on question complexity. T 2 leverages the insight that if an LLM can effectively solve similar questions using specific reasoning strategies, it can apply the same strategy to the original question. This insight enables to adoption of concise reasoning for straightforward questions while maintaining detailed analysis for complex problems. T 2 works through four key steps: decomposing questions into structural elements, generating similar examples with candidate reasoning strategies, evaluating these strategies against multiple criteria, and applying the most appropriate strategy to the original question. Experimental evaluation across seven diverse CQA benchmarks demonstrates that T 2 not only achieves higher accuracy than baseline methods but also reduces computational overhead by up to 25.2%. 1 Introduction Large language models (LLMs) have demonstrated impressive capabilities in Contextual Question Answering (CQA) tasks Trivedi et al. (2023) ; Press et al. (2023) , but their reasoning approaches often lack adaptability to question complexity. Current CQA systems typically employ either direct answer generation or elaborate step-by-step reasoning for all questions, regardless of difficulty Wei et al. (2022) ; Huang et al. (2024) ; Min et al. (2024) . This one-size-fits-all approach has an accuracy-vs-efficiency delimma. Directly generating answers for all questions will deteriorate the performance on difficult questions, which require multi-hop reasoning. Elaborated reasoning for all questions creates an efficiecy challenge: models frequently generate reasoning chains that are excessively verbose, containing redundant steps that do not contribute to finding the correct answer. Existing analysis reveals that these redundant reasoning paths can unnecessarily extend the length of reasoning chains multiple times beyond what is required. Such as exploring multiple solution approaches when only one is needed Ji et al. (2025) , or verifying simple facts with elaborate explanations Muennighoff et al. (2025) . For example, when asked “What is the capital of France?”, models often generate lengthy discussions about France’s history and geography before providing the straightforward answer “Paris.” This computational inefficiency is particularly concerning as model deployment costs continue to rise. Recent studies on reasoning efficiency Yang et al. (2025) ; Zeng et al. (2025) confirm that blindly increasing reasoning chain length can actually harm performance on simpler tasks. Various attempts have been made to address this through adding a budget or stop mechanism to test-time scaling (TTS) methods Wei et al. (2022) ; Huang et al. (2024) to stop thinking early, but these approaches introduce a human bias to the reasoning process Yuan et al. (2023) and fail to leverage the model’s inherent reasoning abilities. Hence, the fundamental challenge is to develop a reasoning mechanism that can dynamically adjust its computational effort based on question complexity, which means providing concise reasoning for straightforward questions while maintaining detailed analysis for complex problems. Therefore, we present T 2 , a think-to-think framework for efficient TTS strategy. T 2 leverages a key insight: if an LLM can effectively solve similar questions using specific reasoning strategies, it can apply comparable strategies to the original question. The process involves four key steps: (1) Decomposing the original question into its structural elements. For example, given the question: [Given Reference Documents] “Which is taller, the Eiffel Tower or the Empire State Building?” T 2 would identify this as a comparative question involving measurement between two specific places as “Which is [adj], [place 1] or [place 2]?”. (2) Creating a diverse set of similar example questions with the same question structure, each paired with supporting documents and potential reasoning strategies. Each reasoning strategy breaks down similar questions into simpler steps using fundamental reasoning skills (e.g., decomposing similar question “Which is taller, Building A or Building B?” into subquestions about individual heights connected by deductive reasoning for comparison). (3) Evaluating these reasoning strategies using multiple criteria to select the most appropriate strategy for the original question. (4) Applying the selected reasoning strategy to the original question while filtering irrelevant information. By learning from similar examples, the model develops a more nuanced understanding of when detailed reasoning is necessary and when a more direct approach is sufficient. This allows T 2 to balance accuracy and efficiency without relying on pre-determined reasoning templates. We evaluate T 2 across seven diverse CQA datasets ranging from simple factual queries to complex multi-hop reasoning tasks. Our results demonstrate that T 2 achieves superior accuracy (up to a 21.3% increase) compared to other TTS approaches while reducing computational requirements by up to 25.2%. These efficiency gains are particularly clear for simpler questions where redundant reasoning steps are eliminated. While for complex questions, T 2 maintains the reasoning depth required for accuracy without exploring unnecessary paths. Our contributions include: • We introduce T 2 , a framework that enables language models to dynamically select appropriate reasoning strategies through similar examples, balancing efficiency and thoroughness based on question complexity. • We develop a multi-criteria selection method that evaluates potential reasoning strategies based on coverage and uniqueness, ensuring the most suitable approach is applied to each question. • We demonstrate through extensive experiments across diverse CQA benchmarks that our method reduces computational requirements by up to 25.2% with superior accuracy. 2 Related Work Contextual QA. In addressing contextual QA, recent works have explored multi-round retrieval or reasoning approaches, including query rewriting for subsequent retrievals Khattab et al. (2022) ; Ma et al. (2023) ; Shao et al. (2023) ; Jiang et al. (2023) , alternating between retrieval and reasoning steps Trivedi et al. (2023) , and employing multi-round self-asking techniques Press et al. (2023) . They all rely on LLMs’ reasoning abilities. Figure 1: Overview of our T 2 . (a) direct prompt or Chain-of-Thought (CoT), which adopts the same reasoning strategy regardless of question complexity. (b) Adaptive-of-Thought, which designs a question complexity evaluator to pre-categorize all questions, which might bring human bias in the evaluator design process. (c) our T 2 . Instead of pre-categorizing questions into different complexity sets, T 2 generates multiple similar examples for different inputs adaptively and selects the best reasoning strategy for answering. Test-Time Scaling. Recent approaches to enhancing LLM reasoning capabilities focus on increasing computational resources during inference Brown et al. (2024) ; Chen et al. (2024) , termed test-time scaling. These methods includes majority voting Wang et al. (2022) , weighted aggregation Li et al. (2023) , best-of-N Lightman et al. (2023) , Tree-of-Thoughts Yao et al. (2023) , and Monte Carlo Tree Search variants Wu et al. (2024) ; Zhang et al. (2024a) ; Zhao et al. (2024) . Besides, o1 model Jaech et al. (2024) and several follow-up works Guo et al. (2025) ;

Read the complete text