Your Model Diversity, Not Method, Determines Reasoning Strategy

Moulik Choraria, Argyrios Gerogiannis, Anirban Das, Supriyo Chakraborty, Berkcan Kapusuzoglu, Chia-Hsuan Lee, Kartik Balasubramaniam, Shi-Xiong Zhang, Sambit Sahu · 2026

Read Your Model Diversity, Not Method, Determines Reasoning Strategy by Moulik Choraria, Argyrios Gerogiannis, Anirban Das, Supriyo Chakraborty, Berkcan Kapusuzoglu, Chia-Hsuan Lee, Kartik Balasubramaniam, Shi-Xiong Zhang, Sambit Sahu (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2604-10827.txt.

Introduction

Choraria and colleagues argue the best way to scale up language-model reasoning depends on the model's diversity profile. That profile, how probability mass spreads across solution paths, must be mapped before any search plan is picked. Their frame splits reasoning uncertainty and shows when deep refinement beats wide sampling, tested on three model types at run time and in training. Low-diversity tuned models gain from deep refinement with light signals, while high-diversity base models are often hurt by it and need breadth or stronger signals.

Catalog record

CREATORMoulik Choraria, Argyrios Gerogiannis, Anirban Das, Supriyo Chakraborty, Berkcan Kapusuzoglu, Chia-Hsuan Lee, Kartik Balasubramaniam, Shi-Xiong Zhang, Sambit Sahu
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE30259
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2604.10827

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Your Model Diversity, Not Method, Determines Reasoning Strategy Moulik Choraria, Argyrios Gerogiannis, Anirban Das, Supriyo Chakraborty, Berkcan Kapusuzoglu, Chia-Hsuan Lee, Kartik Balasubramaniam, Shi-Xiong Zhang, Sambit Sahu Abstract Compute scaling for LLM reasoning requires allocating budget between exploring solution approaches ( breadth ) and refining promising solutions ( depth ). Most methods implicitly trade off one for the other, yet why a given trade-off works remains unclear, and validation on a single model obscures the role of the model itself. We argue that the optimal strategy depends on the model’s diversity profile , the spread of probability mass across solution approaches, and that this must be characterized before any exploration strategy is adopted. We formalize this through a theoretical framework decomposing reasoning uncertainty and derive conditions under which tree-style depth refinement outperforms parallel sampling. We validate it on Qwen-3 4B and Olmo-3 7B families, showing that lightweight signals suffice for depth-based refinement on low-diversity aligned models while yielding limited utility for high-diversity base models, which we hypothesize require stronger compensation for lower exploration coverage. 1 Introduction Exploration via compute scaling has emerged as a powerful axis for improving LLM reasoning ( Wei et al. 2023 ; Shao et al. 2024 ; Snell et al. 2024 ; Welleck et al. 2024 ) . The idea is simple: sample N independent trajectories in parallel, then choose the best candidate ( Cobbe et al. 2021 ; Wang et al. 2023 ) ; tree search methods take it further by iteratively expanding and refining the sampled trajectories via depth-based exploration ( Zhao et al. 2023 ; Zhang et al. 2024 ) . The landmark success of DeepSeek-R1 ( Guo et al. 2025 ) in exploiting exploration during training, has spurred a wave of follow-ups on alternate strategies for training-time exploration, beyond basic parallel sampling ( Xu et al. 2025 ; Zhuang et al. 2025 ; Yao et al. 2025 ) . The template is standard: propose a strategy with some intuitive motivation, evaluate it on 1-2 chosen models, and compare against baselines which use parallel sampling such as GRPO/DAPO ( Shao et al. 2024 ; Yu et al. 2025 ) . Despite the empirical gains, what is missing is an understanding of why a particular strategy works for a particular model. Current work rarely justifies choices like starting from instruct-tuned versus base checkpoints, or favoring parallel sampling over tree-based refinement. We argue that these choices matter, and that any reasoning approach must first characterize its operational regime–its diversity profile, the spread of probability mass across distinct solution approaches–before adopting a strategy . Without this, there is no principled basis for choosing: a strategy effective for one model class may fail on another because the underlying diversity regime differs. To this end, we first develop a theoretical framework (§ 2 ) that decomposes reasoning uncertainty and derives conditions under which depth-based refinement outperforms parallel sampling as a function of the model’s diversity profile. We then empirically validate (§ 3 ) these conditions across two model families, showing that the predicted regime boundaries hold and existing approaches are no longer as effective when applied outside their regime. While we do not claim that diversity profile is the sole factor–dataset choice, prompt design, and model scale can all confound the picture, as we discuss in (§ 4 )–these results offer strong support for our position that the target model’s diversity regime must be characterized before any exploration strategy is adopted. 2 Framework To study the role of model diversity, we first propose a framework to decompose reasoning uncertainty, in the same vein as Bakman et al. 2025 . Our central question: for a fixed model/policy and compute budget, how should a strategy trade-off breadth versus depth exploration. At one extreme, standard parallel sampling represents one means of allocating all compute to breadth; on the other, depth-first refinement resembles tree search. We formalize this trade-off through an abstraction of Monte Carlo Tree Search (MCTS) ( Świechowski et al. 2022 ) . Setup : Let q represent the problem of interest, and the model, parametrized via \phi , generate reasoning trajectories as \tau\sim p_{\phi}(\cdot) . We define \mathcal{T} as the set of all possible reasoning trajectories (for q ), and assume that it admits a partition into m disjoint measurable sets \mathcal{T}=\biguplus_{i=1}^{m}\mathcal{A}_{i} . Here, each \mathcal{A}_{i} intuitively represents a class of high-level solution approaches (e.g. analytical, induction-based etc.). To characterize the probability of finding a correct trajectory, we define: \displaystyle\pi_{i}(\phi) \displaystyle=\mathbb{P}(\tau\in\mathcal{A}_{i}|\tau\sim p_{\phi}(\cdot)) (probability that sample belongs to an approach for model \phi ) \displaystyle\theta_{i}(\phi,B) \displaystyle=\Pr(\text{correct}\mid\tau\in\mathcal{A}_{i};\,\text{budget }B). (model success rate under an approach) \displaystyle\bar{\theta}(\phi,B) \displaystyle=\textstyle\sum_{i}\pi_{i}\,\theta_{i} (marginal model success rate ) Next, we define the the set of viable approaches \mathcal{V}=\{i:\exists\ \tau\in\mathcal{A}_{i},\tau\text{ is correct}\} i.e. approaches that contain at least one correct trajectory. We also define an ideal model \phi^{*} , with the intuition being that it allocates masses to different approaches optimally via \pi(\phi^{*}) . We can now characterize the difficulty of the problem q by decomposing the reasoning uncertainty in into three components: (i) Aleatoric uncertainty: U_{\text{alea}}(q)=H[A\mid\phi^{*}] , which represents the inherent difficulty of identifying the viable latent approaches \mathcal{V} , even for the ideal model. This stems from q and is irreducible. (ii) Epistemic breadth: U_{\text{epi}}^{\text{breadth}}(\phi)=D_{\text{KL}}\!\left(\pi(\phi^{*})\,\|\,\pi(\phi)\right) , which measures the mismatch in approach selection for the given model against the ideal model. We hypothesize that reducing this term requires two stages: first, diverse sampling (breadth exploration) to increase coverage of approaches in the support of \pi(\phi^{*}) ; then, importance-weighted selection among discovered approaches to correct the mismatch between \pi(\phi) and \pi(\phi^{*}) . (iii) Epistemic depth: U_{\text{epi}}^{\text{depth}}(\phi,B)=\sum_{i\in\mathcal{V}}\pi_{i}(\phi^{*})\,D_{\text{KL}}\!\bigl(\mathrm{Bern}(\theta_{i}(\phi^{*},B))\,\|\,\mathrm{Bern}(\theta_{i}(\phi,B))\bigr) i.e. the divergence in within-approach execution quality relative to the ideal model. Our hypothesis is that it complements breadth exploration, and can be reduced by feedback and iterative refinement . With this framework in place, we can now focus on designing an optimal exploration strategy. An MCTS Allocation Strategy. Consider a budget of N trajectories. We split the budget into N_{e}=\alpha N for exploration (discovering distinct approaches) and N_{f}=(1-\alpha)N for refinement (improving within the most promising discovered approach i^{*} ). This mirrors MCTS: N_{e} expands the tree across approaches while N_{f} concentrates on high-value branches. The question is: under what conditions does this outperform i.i.d. sampling? Proposition 1 . Let c_{i}:=-\log(1-\theta_{i}) measure the per-sample refinement value of approach i , c_{\mathrm{rand}}:=-\log(1-\bar{\theta}) that of random sampling, and R^{(t)}:=c_{i^{*}}^{(t)}/c_{\mathrm{rand}} the quality ratio of the best approach over the marginal. Then, assuming oracle identification the optimal approach i^{*} : (a) \mathrm{pass@}N_{\mathrm{mcts}}\geq\mathrm{pass@}N_{\mathrm{rand}} iff \alpha\leq 1-1/R^{(t)} . (b) Discovering i^{*} with probability \geq 1-\eta under i.i.d. exploration requires \alpha\geq\alpha_{\min}(\eta):=\frac{\log(1/\eta)}{N\log(1/(1-\pi_{i^{*}}))}\approx\frac{\log(1/\eta)}{N\pi_{i^{*}}} . Both conditions are jointly satisfiable iff R^{(t)}\geq\frac{1}{1-\alpha_{\min}(\eta)} . The proposition reveals a tension between two competing constraints. Condition (a) imposes an upper bound on the exploration fraction: allocating too large an \alpha diverts budget from refinement, where it would be more effective. Condition (b) imposes a lower bound: discovering rare approaches requires sufficient exploration. The two are jointly satisfiable only when the quality ratio R^{(t)} is large enough to absorb the discovery cost. We note that this analysis assumes all refinement concentrates on a single best approach i^{*} , which represents the most favorable setting for depth-based strategies. Since MCTS cannot outperform random sampling under any weaker refinement

Read the complete text