ProofAgent Harness: Open Infrastructure for Adversarial Evaluation of AI Agents
Fouad Bousetouane, [0.35em] ProofAgent.ai · 2026
Read ProofAgent Harness: Open Infrastructure for Adversarial Evaluation of AI Agents by Fouad Bousetouane, [0.35em] ProofAgent.ai (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-24134.txt.
Introduction
Bousetouane presents ProofAgent Harness, infrastructure for testing AI agents under pressure across many turns, rather than grading single outputs or fixed tasks. The harness gathers test cases, runs hostile multi-turn trials, and records behaviour traces. Scoring uses several calibrated juror personas plus consensus checks and turn-level evidence. Trials across support, medical triage, privacy and security work, and code generation show strong agents failing narrowly, through fragile turns, weak metrics, manipulation paths, and unsafe reframing. Strikingly, a small quantized local model running the harness can challenge top production agents.
Catalog record
| CREATOR | Fouad Bousetouane, [0.35em] ProofAgent.ai |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 73329 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2605.24134
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
ProofAgent Harness: Open Infrastructure for Adversarial Evaluation of AI Agents Fouad Bousetouane, [0.35em] ProofAgent.ai Abstract AI agents are entering high-risk production settings, where they use tools, retain context, follow policies, handle private data, and interact with users over multiple turns. Yet many evaluation methods still judge isolated outputs or static tasks, missing failures that emerge through trajectory, pressure, and adversarial interaction. We introduce ProofAgent Harness 1 1 1 https://github.com/ProofAgent-ai/proofagent-harness , open infrastructure for scalable, auditable, and adversarial AI agent evaluation. The harness provides evaluation infrastructure around an agent: it curates evaluation intelligence, runs adversarial multi-turn trials, captures behavioral traces, applies post-hoc multi-juror scoring, resolves disagreement, and produces evidence-linked reports. Its open design allows developers and researchers to extend domains, traps, metrics, juror personas, scoring rules, and reporting formats. At its core is Adversarial Multi-Juror Scoring with Turn-Level Audit , which evaluates completed agent behavior under pressure using calibrated juror personas, consensus checks, and turn-level evidence. Experiments across customer support, medical triage, privacy/security, and code-generation agents show that strong agents fail selectively through weak metrics, fragile turns, unsafe reframing, and manipulation paths. We also find that a small quantized local Harness LLM can challenge production agents powered by best-in-class large LLMs, suggesting that evaluation capability emerges from the full harness pipeline rather than model scale alone. ProofAgent Harness turns AI agent evaluation from a static score into scalable adversarial evaluation infrastructure: repeatable, evidence-backed, extensible, and actionable before deployment. 1 Introduction AI agents are moving from demonstrations into production workflows. Unlike traditional chatbots, agents operate across multiple turns, use tools, retrieve knowledge, maintain context, follow policies, and act within business or operational systems [ 3 , 1 , 2 ] . This shift is driven by the growing ability of LLMs to combine reasoning and acting through planning, tool use, environment interaction, and feedback adaptation [ 16 , 12 , 14 ] . In this setting, the LLM may serve as the reasoning core, but the agent is a larger system composed of prompts, tools, memory, policies, guardrails, workflows, and execution logic. This creates a new reliability problem. A standalone LLM failure may be limited to an incorrect answer, while an agent failure can become an unsafe tool call, unauthorized workflow step, privacy leak, hallucinated policy citation, or harmful escalation. Agent failures are therefore not only output-level failures. They are trajectory-level failures that emerge through user messages, tool calls, refusals, partial compliance, memory effects, and adversarial reframing. Recent agent benchmarks and evaluation environments have advanced interactive testing across web tasks, tool-use tasks, and multi-step environments [ 9 , 18 ] . However, many approaches remain limited for production readiness. Static benchmarks often miss adversarial pressure. Single LLM-as-judge methods can produce useful scores, but often lack audit trails linking judgments to specific turns and behaviors. Red-team prompts can expose vulnerabilities, but without standardized scoring, consensus, and evidence-linked reporting, results are difficult to compare across domains, versions, and releases. In practice, the key question is not only whether an agent answered well. The more important question is whether it remained safe, grounded, useful, policy-consistent, and manipulation-resistant across a realistic pressured interaction. This requires evaluation methods that are adaptive rather than static, adversarial rather than passive, and traceable rather than opaque. Human expert review remains important in high-risk settings, but it does not scale as the primary evaluation mechanism. Manual review is expensive, slow, inconsistent, and difficult to apply continuously across every agent version, workflow, domain, and release cycle. Human experts should help curate evaluation intelligence and validate sensitive cases, but the first layer of agent testing must be repeatable, adversarial, automated, and auditable. We introduce ProofAgent Harness , an open-source multi-stage framework for auditable AI agent evaluation. We use the term harness to mean the evaluation infrastructure around an agent. A benchmark defines tasks. An LLM judge assigns scores. A harness coordinates the full evaluation process: domain-aware setup, curated evaluation intelligence, adversarial multi-turn trials, behavioral trace capture, post-hoc multi-juror scoring, consensus, and evidence-linked reporting. In this sense, ProofAgent Harness is not only a scoring method; it is evaluation infrastructure for testing AI agents as deployed systems. ProofAgent Harness operationalizes an expert-style adversarial review process. Experts would define the risk surface, design realistic adversarial scenarios, pressure the system across turns, inspect the complete interaction, resolve ambiguous judgments through consensus, and produce evidence-backed findings. The Human-on-the-Bridge stage curates evaluation intelligence before testing begins, while the harness automates adversarial execution, behavioral trace capture, multi-juror scoring, consensus, and reporting. At the core of ProofAgent Harness is Adversarial Multi-Juror Scoring with Turn-Level Audit , a strategy that turns agent evaluation into a structured jury process over completed behavior under pressure. Multiple calibrated juror personas evaluate the same behavioral trace from different perspectives. A consensus layer detects disagreement and triggers re-voting when needed, while turn-level audit links judgments to concrete evidence. This design reduces common weaknesses of single-judge evaluation, including score flattening, hidden disagreement, and weak traceability. Our experimental evaluation studies agents across customer support, medical triage, privacy/security, and code-generation domains. We evaluate both symmetric settings, where ProofAgent Harness is powered by a Large LLM, and asymmetric settings, where it is powered by a smaller local model. The results show two patterns. First, strong agents rarely fail uniformly; they fail selectively through weak metrics, fragile turns, unsafe reframing, or manipulation paths. Second, an asymmetric Harness can still generate meaningful adversarial pressure when embedded in the full ProofAgent Harness pipeline, although calibration against a symmetric Harness remains necessary in subtle cases. These findings suggest that the evaluation capability of a harness is not determined by model scale alone. It emerges from the structure of the evaluation process: curated traps, domain-aware planning, multi-turn pressure, behavioral trace capture, persona-based scoring, consensus, and evidence-linked reporting. This makes ProofAgent Harness reusable across domains, rerunnable across versions, integrable into regression testing, and extensible through new traps, metrics, juror personas, and scoring rules. By releasing ProofAgent Harness as open source evaluation infrastructure, we aim to help establish adversarial AI agent evaluation as a shared practice across research and industry. The framework is intentionally extensible: new domains, traps, metrics, juror personas, and scoring policies can be added by the community. Our goal is for ProofAgent Harness to serve as a foundation for repeatable, auditable, and evidence-backed evaluation of AI agents before they are trusted in production. As AI agents move from demonstrations to real operational workflows, evaluation must also evolve. ProofAgent Harness is a step toward that shift: from static benchmarks to adversarial behavioral testing, from isolated scores to evidence-linked reports, and from confidence by demonstration to confidence by proof. The rest of this paper is organized as follows. Section 2 reviews prior work on LLM-as-judge evaluation, agent benchmarks, red teaming, and human-centered oversight. Section 3 presents the ProofAgent Harness framework. Section 4 formalizes Adversarial Multi-Juror Scoring with Turn-Level Audit. Section 5 describes the experimental setup and results, and Section 6 concludes. 2 Related Work AI agent evaluation sits at the intersection of model evaluation, interactive task benchmarking, adversarial testing, and human-centered oversight. Figure 1 summarizes the four evaluation families discussed in this section: LLM-as-judge evaluation, task and environment benchmarks, adversarial and red-team evaluation, and human-centered oversight. Each family contributes useful capabilities, but none alone fully addresses the