MetaScale: Test-Time Scaling with Evolving Meta-Thoughts
Qin Liu Wenxuan Zhou Nan Xu James Y. Huang Fei Wang, Sheng Zhang Hoifung Poon Muhao Chen · 2025
Read MetaScale: Test-Time Scaling with Evolving Meta-Thoughts by Qin Liu Wenxuan Zhou Nan Xu James Y. Huang Fei Wang, Sheng Zhang Hoifung Poon Muhao Chen (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2503-13447.txt.
Introduction
Qin Liu and co-authors let a language model shop for its own thinking strategy before it answers. Models tend to reuse reasoning patterns from training rather than pick the tactic a task really needs, and fixed strategy add-ons help one task while failing others. Their 2025 framework, MetaScale, keeps a pool of meta-thoughts, short plans for how to think, and tunes the pool while the model works.
A bandit rule picks which strategy to try next, guided by a reward model, while a genetic step breeds the best strategies into new ones. On a hard arena benchmark the method lifted the win rate for a leading model by eleven points, edging past a strong reasoning model under style control. Gains grew as the sampling budget rose, and answers came out more structured and expert in tone. The authors pitch test-time strategy search as a general upgrade path.
Catalog record
| CREATOR | Qin Liu Wenxuan Zhou Nan Xu James Y. Huang Fei Wang, Sheng Zhang Hoifung Poon Muhao Chen |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 35 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2503-13447
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
MetaScale: Test-Time Scaling with Evolving Meta-Thoughts Qin Liu Wenxuan Zhou Nan Xu James Y. Huang Fei Wang, Sheng Zhang Hoifung Poon Muhao Chen Abstract One critical challenge for large language models (LLMs) for making complex reasoning is their reliance on matching reasoning patterns from training data, instead of proactively selecting the most appropriate cognitive strategy to solve a given task. Existing approaches impose fixed cognitive structures that enhance performance in specific tasks but lack adaptability across diverse scenarios. To address this limitation, we introduce MetaScale , a test-time scaling framework based on meta-thoughts —adaptive thinking strategies tailored to each task. MetaScale initializes a pool of candidate meta-thoughts, then iteratively selects and evaluates them using a multi-armed bandit algorithm with upper confidence bound selection, guided by a reward model. To further enhance adaptability, a genetic algorithm evolves high-reward meta-thoughts, refining and extending the strategy pool over time. By dynamically proposing and optimizing meta-thoughts at inference time, MetaScale improves both accuracy and generalization across a wide range of tasks. Experimental results demonstrate that MetaScale consistently outperforms standard inference approaches, achieving an 11 % 11\% performance gain in win rate on Arena-Hard for GPT-4o, surpassing o1-mini by 0.9 % 0.9\% under style control. Notably, MetaScale scales more effectively with increasing sampling budgets and produces more structured, expert-level responses. 1 Introduction Large language models (LLMs; Achiam et al. 2023 ; Dubey et al. 2024 ; Guo et al. 2025 ) have demonstrated remarkable capabilities across a wide range of cognitive tasks. With minimal or no guidance, they can perform human-like multi-step thinking processes Kojima et al. 2022 ; Wang and Zhou 2024 . However, one critical challenge is that they may not proactively determine when or how to apply different thinking processes. Instead, their thinking process is often shaped by pattern matching from training data and often diverges from human reasoning patterns Dziri et al. 2024 ; Bao et al. 2025 , rather than selecting the most appropriate way to think or respond. To address these challenges, researchers have sought to explicitly define the thinking processes of LLMs by mimicking aspects of human cognition and introducing reasoning strategies such as self-verification Weng et al. 2022 ; Madaan et al. 2024 , chain-of-thought prompting Wei et al. 2022 , and reverse thinking Chen et al. 2024 , etc. By constraining LLMs to follow these structured thinking processes, models have shown improved performance on specific tasks. However, these methods are often designed for particular types of tasks, limiting their adaptability and effectiveness across diverse tasks Sprague et al. 2024 ; Xu et al. 2024 . More critically, these approaches impose fixed thinking structures rather than enabling LLMs to adaptively determine the most effective task-specific strategy, potentially limiting their performance. To address these limitations, we propose the idea of meta-thinking , a process where LLMs first reflect on their approach before generating a response. Rather than immediately solving a problem, the model initiates the process by determining how to think and select the most suitable cognitive strategy from a range of available options. For example, when tackling a complex logical puzzle, an LLM might choose to work forward from given conditions, reason backward from the goal, or verify its answer through self-reflection. By incorporating this meta-thinking step, LLMs can dynamically adapt their reasoning process to different tasks, rather than relying on rigid, predefined heuristics. Figure 1: Overview of MetaScale . MetaScale initializes a pool of meta-thoughts, selects and evaluates them using the Upper Confidence Bound (UCB) algorithm, and periodically evolves high-performing meta-thoughts. The best response is returned after exhausting the sampling budgets. Building on this idea, we introduce MetaScale , a test-time scaling framework that incorporates meta-thinking. The goal is to enable LLMs to explore different thinking strategies, and generate the most effective response for a given input. The process begins with the generation of an initial pool of meta-thoughts, where the LLM autonomously formulates reasoning strategies tailored to the given problem. To enhance diversity, we retrieve relevant input-output pairs from instruction-tuning datasets Zhao et al. 2024 and prompt the LLM to abstract thinking patterns specific to the current task. Next, we iteratively prompt the LLM based on the selected meta-thoughts. The selection process is guided by an Upper Confidence Bound (UCB; Auer 2002 ) algorithm, which balances exploration and exploitation. A reward model Liu et al. 2024 then evaluates response quality. To further promote diversity, we periodically apply a genetic algorithm , generating new meta-thoughts by evolving high-reward ones. Finally, the best response found through this process is returned after reaching the sampling budget. Experiments show that MetaScale significantly enhances the problem-solving capabilities of LLMs across diverse tasks, consistently outperforming baseline methods on both various real-world reasoning challenges. More importantly, MetaScale scales effectively with increased sampling budgets, further improving performance by evolving meta-thoughts that leads to task solutions of higher quality. Our contributions are three-fold. First, we introduce MetaScale , a novel test-time scaling strategy that enables LLMs to proactively select and refine cognitive strategies at test time, moving beyond static, predefined heuristics. Second, we adapt the genetic algorithm to evolve high-reward meta-thoughts over multiple iterations, further enhancing adaptability and reasoning efficiency. Third, experiments demonstrate that MetaScale consistently scales with increased sampling budgets, effectively leveraging additional computational resources to refine responses. 2 Method In this section, we introduce MetaScale , a test-time scaling framework that explores diverse cognitive strategies to optimize the reasoning process during inference. We first provide a brief overview and relevant preliminaries in Section 2.1 , followed by a detailed description of meta-thought initialization ( Section 2.2 ), selection ( Section 2.3 ), and evolution ( Section 2.4 ). 2.1 MetaScale Overview MetaScale is a test-time scaling strategy designed to enhance the problem-solving capabilities of LLMs by incorporating meta-thoughts that guide the reasoning process before generating a response. Inspired by human cognitive processes, a meta-thought consists of two key components: 1. Cognitive Mindset , which is an appropriate perspective, expertise, or role that the model adopts to approach the task. 2. Problem-Solving Strategy , which is a structured pattern used to formulate a sound solution based on the given mindset. For instance, when asked to “ Use ABC notation to write a melody in the style of a folk tune ”, a well-formed meta-thought could include a cognitive mindset of “ a musician or music educator with expertise in music theory and composition ”, and a problem-solving strategy indicating “ First present a melody that captures the characteristics of a traditional folk tune, then … ” The model then utilizes these strategies to guide its reasoning process and generate a response. MetaScale develops effective meta-thoughts for an input task in three phases: initialization, selection, and evolution. During initialization, MetaScale generates a diverse pool of reasoning strategies tailored to the input task by leveraging its own prior knowledge and instruction-tuning datasets. Next, a Multi-Armed Bandit (MAB) algorithm is used to explore and select promising meta-thoughts. Finally, MetaScale employs a genetic algorithm to iteratively refine and expand the meta-thought pool. To maintain computational feasibility, MetaScale operates within a fixed sampling budget, and returns the response with the highest score given by a reward model once the budget is exhausted. We illustrate the overall design of