When Does Combining Language Models Help? A Co-Failure Ceiling on Routing, Voting, and Mixture-of-Agents Across 67 Frontier Models

Josef Chen · 2026

Read When Does Combining Language Models Help? A Co-Failure Ceiling on Routing, Voting, and Mixture-of-Agents Across 67 Frontier Models by Josef Chen (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2606-27288.txt.

Introduction

Chen asks how much combining many language models can ever help, and finds a hard ceiling set by shared failure. Routers, voting, cascades, and mixtures of agents all pick one member model's answer in the end. Accuracy for any such scheme cannot exceed one minus the rate at which every model fails the same query. The field usually reports average pairwise error correlation instead. Chen proves that statistic cannot pin down the shared-failure rate, in a study published in 2026, covering sixty-seven frontier models.

A statistical bound turns one graded held-out set into a zero-cost certificate of the largest possible gain. Across models from twenty-one providers, calibrated models still underprice the all-wrong tail, more so as pools grow. Gains come from models failing on different questions, not from adding more of them. Diverse pools beat same-model ensembles at matched quality.

Catalog record

CREATORJosef Chen
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE85 KB
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2606.27288

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

When Does Combining Language Models Help? A Co-Failure Ceiling on Routing, Voting, and Mixture-of-Agents Across 67 Frontier Models Josef Chen Abstract Multi-model LLM systems, including routing, voting, cascades, fusion, and mixture-of-agents (MoA), are increasingly used to push accuracy beyond any single model. We show that the achievable gain has a ceiling, fixed by a quantity other than the one the field usually reports. For any policy whose output is one of the member models’ answers, such as a router, a majority vote, or a cascade, accuracy cannot exceed 1 − β 1-\beta , where β \beta is the rate at which every model is wrong on the same query. Practice instead reports the average pairwise error correlation ρ \rho , and we prove that ρ \rho cannot identify β \beta : error laws with identical marginals and identical pairwise correlations can still differ in β \beta . A Clopper–Pearson bound on β \beta turns one graded, held-out query set into a $0 certificate on the largest gain any such policy could deliver, before a router is trained. Across 67 67 frontier models from 21 21 providers, among them GPT-5.5, Claude Opus 4.8, Gemini 3.1 Pro, Grok-4.3, DeepSeek V4, Qwen3.7-Max, and Kimi K2.7, a correctly (tetrachoric) calibrated single-factor model still underprices the all-wrong tail, by a margin that widens with the size of the pool: about 2.5 2.5 times on open-ended mathematics ( 90 % 90\% CI 1.7 1.7 to 3.4 3.4 , k = 17 k=17 ), holding under the full 67 67 -model Gaussian copula ( β = 0.052 \beta=0.052 versus a predicted 0.023 0.023 ) and recurring on execution-graded code ( β = 0.079 \beta=0.079 ). At matched quality, a diverse low- ρ \rho ensemble beats a high- ρ \rho Self-MoA one. Re-asking the same GPQA-Diamond questions in free-response rather than multiple-choice form reopens the tail ( β = 0.127 \beta=0.127 ; a five-judge LLM panel, κ \kappa from 0.73 0.73 to 0.92 0.92 ), locating the effect in task format rather than subject. On our pool, and on tasks where answers can be checked, combining models rarely beats the single best model without a strong query-level routing signal; the gains come from models that fail on different questions rather than from adding more of them. 1 Introduction The single-model era is closing. Enterprises serve production traffic across hundreds of models from dozens of providers, selecting per workload on cost, latency, reliability, and capability; a routing layer increasingly mediates spend and governance, and carries the provider risk [ 31 , 12 , 2 ] . The operative question has shifted from which model is best to how a buyer should allocate a token and dollar budget across a heterogeneous, correlated, rapidly depreciating pool . Practitioners answer it with a single diagnostic: the pairwise error correlation \rho between models, low values signalling that diversity will pay. Our central finding is that this diagnostic is the wrong one. What bounds orchestration is \beta , the rate at which all models fail on the same query: no router, vote, or cascade can exceed accuracy 1-\beta , and \rho cannot see \beta . The gap is not academic, because on today’s open-ended tasks the strongest models increasingly fail together. A Clopper–Pearson bound on \beta , measurable from one graded query set, says in advance how much room any such policy has to beat the single best model. What we concede. The equicorrelated variance floor is classical portfolio and ensemble theory [ 26 , 34 , 18 , 36 , 38 ] and, in its Gaussian-copula form for language-model ensembles, 35 ; the oracle upper envelope and the optimality of routing and cascading are due to 3 ; and our tools (linear-programming duality [ 1 ] , Clopper–Pearson intervals, the Gaussian copula, the single-factor probit) are standard. We claim no new routing algorithm. The contribution is their specialization to priced inference orchestration and the market-scale measurement. Contributions. 1. The orchestration ceiling and a finite-sample certificate (§ 5 , Prop. 1 ): no router, vote, or cascade can exceed 1-\beta ; the oracle gain localizes as \Pr[\text{single-best wrong}]-\beta ; and a Clopper–Pearson bound turns one query sample into a certificate on the largest gain any such policy can deliver. 2. Why pairwise \rho underprices co-failure (Prop. 2 ): under tail dependence the single-factor estimate of \beta from \rho is downward-biased, with the bias diverging in pool size and driven by a common-mode atom rather than tail dependence as such. 3. A market-scale measurement (§ 4 – 5 ): on 67 models from 21 provider families, oracle routing gain is positive yet a learned router realizes almost none of it; the \beta / \rho gap and its growth with pool size are measured directly; and two regimes, ceiling-bound (open-ended math) and realizability-bound (science), appear across domains, though the decisive all-models-wrong counts are small (§ 5 ). 4. Supporting economics (App. A ): budget-constrained routing as a priced assignment with a single shadow price (Prop. 4 ); a cost-aware diversification limit (Props. 6 , 7 ); and cascade calibration boundaries (Prop. 8 , Cor. 1 ). These specialize standard tools and are deferred to the appendix, as is an observational option value of breadth under churn (App. E ). query x type t Orchestration route \,\cdot\, cascade \,\cdot\, fuse shadow price \lambda_{B} GPT-5.5 $30 \cdot Claude Opus 4.8 GPT-5.4 \cdot Gemini 3.1 Pro \cdot Grok-4.3 Qwen3.7-Max \cdot GLM-5.2 \cdot Kimi K2.7 DeepSeek V4 \cdot MiniMax M3 \cdot Mistral Llama-4 \cdot Gemma \cdot Phi $0.1 67 models, 21 families: correlated, priced, churning accuracy \leq 1-\beta Ceiling-bound (open-ended math): \beta>0 caps every policy; a correctly-calibrated single-factor model still underprices the tail \sim\!2.5\times (residual common mode), growing with pool size. Realizability-bound (science): \beta\!\approx\!0 , ceiling open; the 0.15 oracle gain is resolvable disagreement a router would still have to realize. Figure 1: Orchestration is allocation. A query is routed, cascaded, or fused over a priced, correlated, fast-churning pool of 67 frontier-to-cheap models; the optimal policy is a per-type bang-per-buck rule with a single shadow price \lambda_{B} on the inference dollar (Prop. 4 ). No selection policy can exceed the ceiling 1-\beta set by the rate \beta at which all models fail at once (Prop. 1 ). The field reports pairwise error correlation \rho to decide whether to orchestrate; \rho is blind to \beta , and headroom is foreclosed in two opposite regimes that \rho cannot tell apart. 2 Related Work Routing and cascades. Learned routers select one model per query [ 31 , 4 ] ; cascades escalate from cheap to strong on low confidence [ 2 , 25 ] . 3 unify the two and prove optimality of a routing strategy and an optimal cascade; 14 characterize the optimal two-model deferral rule and show confidence-based deferral can be sub-optimal when the downstream model’s errors are unmodeled, directly relevant to our calibration-and-edge condition. 12 benchmark routing on pre-computed outcomes. These works treat the oracle as an empirical ceiling and the cascade threshold as a tuned hyperparameter; we add the dollar stopping rule, the calibration dominance boundary, and the volume ceiling they leave implicit. Ensembling and fusion. 13 rank-and-fuse generations; 37 aggregate in layers; 23 sample-and-vote; 24 show that sampling

Read the complete text