Test-time Scaling Techniques in Theoretical Physics - A Comparison of Methods on the TPBench Dataset
Zhiqi Gao, Tianyi Li, Yurii Kvasiuk, Sai Chaitanya Tadepalli, Maja Rudolph, Daniel J.H. Chung, Frederic Sala, Moritz Münchmeyer · 2025
Read Test-time Scaling Techniques in Theoretical Physics - A Comparison of Methods on the TPBench Dataset by Zhiqi Gao, Tianyi Li, Yurii Kvasiuk, Sai Chaitanya Tadepalli, Maja Rudolph, Daniel J.H. Chung, Frederic Sala, Moritz Münchmeyer (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2506-20729.txt.
Introduction
Zhiqi Gao and a team of co-authors compare ways to boost language model reasoning at answer time in physics in a 2025 paper. Extra computing at test time helps models on math contests, but whether those lessons carry to advanced physics was open. The team run common test time methods on a physics benchmark and set them against math results.
They add a new weak checker that verifies each step with symbols rather than words, tailored to how physics problems are built. That checker lifted parallel search results well past the older methods on the physics set. It also worked well when they tried it back on the math benchmark.
Catalog record
| CREATOR | Zhiqi Gao, Tianyi Li, Yurii Kvasiuk, Sai Chaitanya Tadepalli, Maja Rudolph, Daniel J.H. Chung, Frederic Sala, Moritz Münchmeyer |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 71 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2506-20729
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Test-time Scaling Techniques in Theoretical Physics - A Comparison of Methods on the TPBench Dataset Zhiqi Gao, Tianyi Li, Yurii Kvasiuk, Sai Chaitanya Tadepalli, Maja Rudolph, Daniel J.H. Chung, Frederic Sala, Moritz Münchmeyer Abstract Large language models (LLMs) have shown strong capabilities in complex reasoning, and test-time scaling techniques can enhance their performance with comparably low cost. Many of these methods have been developed and evaluated on mathematical reasoning benchmarks such as AIME. This paper investigates whether the lessons learned from these benchmarks generalize to the domain of advanced theoretical physics. We evaluate a range of common test-time scaling methods on the TPBench physics dataset and compare their effectiveness with results on AIME. To better leverage the structure of physics problems, we develop a novel, symbolic weak-verifier framework to improve parallel scaling results. Our empirical results demonstrate that this method significantly outperforms existing test-time scaling approaches on TPBench. We also evaluate our method on AIME, confirming its effectiveness in solving advanced mathematical problems. Our findings highlight the power of step-wise symbolic verification for tackling complex scientific problems. † † affiliationnotice: Equal contribution 1 Introduction Large language models (LLMs), while increasingly competent at mathematical reasoning, still face challenges with intricate multi-step derivations. To bridge this gap, a variety of test-time scaling techniques have been developed to augment their reasoning capabilities. These techniques, including repeated sampling and self-consistency ( Brown et al. 2024 ) , sampling-plus-verifier pipelines ( Zhao et al. 2025 ; Liu et al. 2025 ) , multi-round reflection ( Tian et al. 2025 ; Muennighoff et al. 2025 ) , hierarchical decomposition ( Teng et al. 2025 ; Simonds & Yoshiyama 2025 ; Yang et al. 2025a ) , and RL-based planners ( Guan et al. 2025 ; Pan et al. 2025 ; Hou et al. 2025 ) , have shown improvement on standard mathematical benchmarks ( Hendrycks et al. 2021 ; Cobbe et al. 2021 ) . This approach presents a distinct computational paradigm from scaling efforts focused on pretraining or extensive finetuning. While test-time compute strategies require more resources than a single inference pass, they can offer a more compute-efficient path to performance gains on complex tasks, challenging traditional scaling laws that often optimize for training costs over inference-time expenditures ( Snell et al. 2024 ; Bian et al. 2025 ) . Much of the progress in test-time scaling has been developed for mathematical settings. However, theoretical physics reasoning presents distinct challenges compared to undergraduate mathematical level reasoning, such as AIME ( Art of Problem Solving ) , research level mathematical reasoning such as FrontierMath ( Glazer et al. 2024 ) , and proof assistant-based mathematics, where results can be auto-verified ( AlphaProof & TEams 2024 ) . For a comparison between the use of math in theoretical physics and in pure math, see ( Chung et al. 2025 ) . In short, theoretical physics uses direct calculation techniques on mathematical concepts that are not as formally defined as in math. In this work, we systematically evaluate prominent test-time scaling methods on the TPBench dataset, a collection of advanced physics problems ranging from undergraduate to research level ( Chung et al. 2025 ) . We find that simple parallel or sequential test time scaling techniques do not perform very well on this dataset. To improve their performance, we develop a novel weak-verifier framework that uses a SymPy-augmented agent to verify the mathematical steps within each candidate solution, providing a robust mechanism for identifying correct reasoning paths. Our contributions are: • A comprehensive empirical study of existing test-time scaling techniques, including sampling-based search and multi-round reasoning, on the TPBench theoretical physics dataset. Our analysis investigates how the effectiveness of these methods compares between physics and mathematical reasoning domains. • A novel step-wise weak-verifier pipeline that integrates symbolic computation (SymPy) to grade and select solutions. Our results show this pipeline surpasses other test-time scaling methods by a significant margin on TPBench, achieving up to a 22% improvement and approaching the theoretical Best-of-N performance. We also test our verifier on the AIME benchmark ( Art of Problem Solving ) , confirming its general effectiveness. Our work highlights the importance of tailoring inference-time strategies to specific scientific domains and demonstrates the power of symbolic verification in various domains. 2 Related Work Test-Time Scaling for Reasoning. Large language models have recently shown that increasing inference-time computation—often called test-time scaling—can significantly boost reasoning performance ( Snell et al. 2024 ; Bian et al. 2025 ) . OpenAI’s proprietary o1 model exemplified this by using extended “slow thinking” procedures to achieve stronger results than standard one-shot prompting. This has spurred a wave of research into inference-time strategies that improve mathematical problem solving without changing model weights. Sampling-Based Search and Verification. A common way to leverage more compute at test time is to sample multiple responses in parallel and choose a final answer by some criterion. This approach uses the randomness of LLMs. Brown et al. show that repeated sampling at large scales steadily increases the probability of including a correct solution at least once, boosting solve rates on code and formal proof tasks dramatically ( Brown et al. 2024 ) . Zhao et al. extend this idea with a “sample, scrutinize and scale” method: generate hundreds of candidate solutions and use the LLM itself as a verifier to score and select the best one ( Zhao et al. 2025 ) . Liu et al. rethink the allocation of inference budget between a solution-proposing policy model and a process reward model, demonstrating that small models with optimal compute splits can rival or beat much larger ones on challenging math benchmarks ( Liu et al. 2025 ) . Iterative and Multi-Round Reasoning. As a comparison to parallel sampling, sequential approaches such as iterative refinement let the model revisit its answer through multiple rounds. Tian et al. propose Think Twice , which feeds the model’s initial answer back into itself for a re-answer step, yielding consistent gains on AIME and other math competition tasks ( Tian et al. 2025 ) . Muennighoff et al. introduce s1 , using “budget forcing” to extend chain-of-thought generation (“Wait” tokens) and fine-tuning on high-quality reasoning traces, force the answer to have a longer CoT and a better performance ( Muennighoff et al. 2025 ) . Wang et al. observe that models often underthink—prematurely abandoning reasoning paths—and propose a Thought Importance Penalty decoding tweak to encourage deeper exploration of each line of thought ( Wang et al. 2025 ) . Yang et al. develop Step Back to Leap Forward , endowing the LLM with self-backtracking capabilities to undo and retry flawed steps to enhance mathematical reasoning capabilities. ( Yang et al. 2025b ) . Hierarchical and Decompositional Strategies. Breaking complex problems into smaller subtasks is another effective paradigm. Teng et al. introduce Atom of Thoughts (AoT), treating reasoning as a Markov chain of atomic sub-questions that are solved independently and contracted back into the main problem, preventing distraction by irrelevant context ( Teng et al. 2025 ) . Simonds and Yoshiyama’s LADDER framework has models that recursively generate simpler variants of a hard problem, solve them, and use these solutions to tackle the original, achieving dramatic gains on integration calculus questions (