Multi-Agent Verification: Scaling Test-Time Compute with Multiple Verifiers

Shalev Lifshitz 1, Sheila A. McIlraith 2,3, Yilun Du 4 · 2025

Read Multi-Agent Verification: Scaling Test-Time Compute with Multiple Verifiers by Shalev Lifshitz 1, Sheila A. McIlraith 2,3, Yilun Du 4 (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2502-20379.txt.

Introduction

Shalev Lifshitz, Sheila McIlraith, and Yilun Du scale test-time compute by adding verifiers in a 2025 paper. Extra compute at answer time can lift language model quality without retraining. Verifiers that grade candidate outputs are a common lever. The number of verifiers itself is proposed as a scaling axis.

Their paradigm, multi-agent verification, combines many verifiers at once. Aspect verifiers are off-the-shelf models prompted to check single facets. A best-of-n algorithm merges sampling with the verifier panel. Scaling patterns beat self-consistency and single reward models. Weak verifiers combined still improved stronger generators. The same model can even generate and verify for self-improvement. No extra training is needed for the building blocks. The authors establish verifier count as a promising dimension. Deployment cost is the trade-off to manage.

Catalog record

CREATORShalev Lifshitz 1, Sheila A. McIlraith 2,3, Yilun Du 4
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE59 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2502-20379

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Multi-Agent Verification: Scaling Test-Time Compute with Multiple Verifiers Shalev Lifshitz 1, Sheila A. McIlraith 2,3, Yilun Du 4 Abstract By utilizing more computational resources at test-time, large language models (LLMs) can improve without additional training. One common strategy uses verifiers to evaluate candidate outputs. In this work, we propose a novel scaling dimension for test-time compute: scaling the number of verifiers . We introduce Multi-Agent Verification (MAV) as a test-time compute paradigm that combines multiple verifiers to improve performance. We propose using Aspect Verifiers (AVs), off-the-shelf LLMs prompted to verify different aspects of outputs, as one possible choice for the verifiers in a MAV system. AVs are a convenient building block for MAV since they can be easily combined without additional training. Moreover, we introduce BoN-MAV, a simple multi-agent verification algorithm that combines best-of- n n sampling with multiple verifiers. BoN-MAV demonstrates stronger scaling patterns than self-consistency and reward model verification, and we demonstrate both weak-to-strong generalization, where combining weak verifiers improves even stronger LLMs, and self-improvement, where the same base model is used to both generate and verify outputs. Our results establish scaling the number of verifiers as a promising new dimension for improving language model performance at test-time. https://ardalabs.ai/MultiAgentVerification/ 1 ArdaLabs.AI, 2 University of Toronto, 3 Vector Institute for AI, 4 Harvard University 1 Introduction Scaling the size of large language models (LLMs) and their training datasets has driven remarkable progress in artificial intelligence ( Brown et al., 2020 ; Chowdhery et al., 2023 ; Hoffmann et al., 2022 ) . However, the growing cost of scaling model size and obtaining unseen high-quality pretraining data has sparked growing interest in methods that improve LLM performance without simply scaling parameters or data. Among these, a promising new direction has emerged: scaling test-time compute , where models spend more computational resources during inference—much like humans spend more time thinking through harder problems. A common strategy for scaling test-time compute is best-of- n sampling ( Stiennon et al., 2020 ; Cobbe et al., 2021 ; Nakano et al., 2021 ) , where n candidate outputs are sampled from a generator LLM and a verifier model scores each candidate output based on its quality or correctness. The highest-scoring output is then selected. Under this strategy, the amount of test-time compute can be scaled up by increasing the number of sampled outputs. However, in this work, we propose a new orthogonal scaling dimension: scaling the number of verifiers . We introduce Multi-Agent Verification (MAV) , a test-time compute paradigm that combines multiple verifiers to improve performance. Typically, verifiers are implemented as reward models which are trained using reinforcement learning from human feedback ( Christiano et al., 2017 ; Stiennon et al., 2020 ; Ouyang et al., 2022 ; Bai et al., 2022a ) . However, relying on reward models as verifiers introduces two crucial limitations for multi-agent verification: (1) each reward model has to be trained on expensive curated preference data, and (2) there is no straightforward way to combine scores generated by heterogeneous reward models trained on different datasets (they produce uncalibrated scores). These limitations make reward models poorly suited for multi-agent verification and restrict our ability to simply scale up the number and type of verifiers at test-time. To address these limitations and enable scalable multi-agent verification, we propose using Aspect Verifiers (AVs) — off-the-shelf LLMs prompted to verify specific aspects of candidate outputs through binary True/False approvals. This approach is motivated by the observation that internet data contains abundant examples of humans providing binary evaluations with feedback (e.g., educational assessments, academic peer reviews, online forums, and automated code tests), which suggests that language models may be naturally suited for binary verification. Unlike reward models, AVs do not require additional training since producing binary approvals falls naturally within the training distribution of their base LLMs, and their binary outputs can be easily combined across multiple models through simple voting mechanisms. Thus, the number and type of aspect verifiers can be easily scaled up without additional training. We note that aspect verifiers are just one possible implementation choice for the verifiers in a MAV system, which address the two key limitations of typical reward model verifiers. By aggregating binary signals across a diverse set of aspect verifiers, we can leverage the growing ecosystem of language models to produce a more robust verification signal. Each verifier can focus on different aspects of outputs like mathematical correctness or logical soundness, and employ different verification strategies such as direct yes/no approval, step-by-step analysis, solution rephrasing, or edge case checking. Thus, a diverse set of aspect verifiers can be obtained by varying three key axes: the base LLM, the aspect to verify, and the verification strategy. To investigate scaling multi-agent verification, we introduce BoN-MAV as a specific algorithm which combines best-of- n sampling with aspect verifiers. This is one implementation of a MAV algorithm, combining traditional best-of- n sampling with multiple verifiers. Given an input, BoN-MAV (1) samples n outputs from a generator LLM, (2) collects binary approvals from a set of m aspect verifiers, and (3) selects the output with the most approvals. We investigate scaling test-time compute with this approach along two orthogonal dimensions: the traditional dimension of increasing the number of sampled candidate outputs n , and our novel test-time scaling dimension of increasing the number of verifiers m . We find that using multiple diverse verifiers to select between candidate outputs is an effective strategy, and that performance improves as we use more verifiers. More specifically, across multiple domains and LLMs, BoN-MAV demonstrates more effective scaling patterns when we increase the number of sampled outputs, compared to best-of- n with reward model verification ( Stiennon et al., 2020 ; Cobbe et al., 2021 ) and self-consistency ( Wang et al., 2022 ; Li et al., 2022b ; Thoppilan et al., 2022 ; Lewkowycz et al., 2022 ) . We also demonstrate weak-to-strong generalization ( Burns et al., 2023 ) , whereby combining many small aspect verifiers can improve the performance of even stronger generator LLMs, and we show that BoN-MAV enables self-improvement by using the same base LLM for both the generator and set of aspect verifiers. Since BoN-MAV is just one simple approach to multi-agent verification, we expect that substantial improvements can be achieved using alternative methods. Overall, our paper makes the following contributions: (1) We introduce Multi-Agent Verification (MAV) as a new test-time paradigm that combines multiple verifiers at test-time, opening a novel scaling dimension: scaling the number of verifiers . (2) We propose Aspect Verifiers (AVs), off-the-shelf LLMs which require no additional training and naturally support combining verification signals from multiple heterogeneous verifiers using voting mechanisms. (3) We demonstrate that BoN-MAV, a simple multi-agent verification algorithm which combines best-of- n with aspect verifiers, improves the performance of various generator LLMs as we scale up the number and type of aspect verifiers. Figure 2 : Multi-agent verification for a single solution. An illustration of how combining multiple aspect verifiers can produce a more robust verification signal. Five different aspect verifiers evaluate an incorrect MATH ( Hendrycks et al., 2021 ) solution sampled from Gemini-1.5-Pro. The verifiers vary across three dimensions: base models (Gemini-1.5-Flash, GPT-4o-mini), aspects to verify (e.g., general correctness,

Read the complete text