Bench-2-CoP: Can We Trust Benchmarking for EU AI Compliance?
Matteo Prandi, Vincenzo Suriani, Federico Pierucci, Marcello Galisai, Daniele Nardi, Piercosma Bisconti · 2025
Read Bench-2-CoP: Can We Trust Benchmarking for EU AI Compliance? by Matteo Prandi, Vincenzo Suriani, Federico Pierucci, Marcello Galisai, Daniele Nardi, Piercosma Bisconti (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2508-05464.txt.
Introduction
Matteo Prandi and five co-authors check whether common AI test sets can be trusted to judge if AI models meet new European rules. The EU AI Act and its code of practice target broad risks, but old test sets were not built for that aim. Their work came out in twenty twenty-five.
The team built a frame that maps a huge pool of test questions to the act's list of skills and traits, using an AI judge method. Most test weight fell on just two traits, false made up output and weak trust in results. Key skills tied to loss of control, such as dodging human watch, got no coverage at all in the whole pool.
Catalog record
| CREATOR | Matteo Prandi, Vincenzo Suriani, Federico Pierucci, Marcello Galisai, Daniele Nardi, Piercosma Bisconti |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 71 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2508-05464
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Bench-2-CoP: Can We Trust Benchmarking for EU AI Compliance? Matteo Prandi, Vincenzo Suriani, Federico Pierucci, Marcello Galisai, Daniele Nardi, Piercosma Bisconti Abstract The rapid advancement of General Purpose AI (GPAI) models necessitates robust evaluation frameworks, especially with emerging regulations like the EU AI Act and its associated Code of Practice (CoP). Current AI evaluation practices depend heavily on established benchmarks, but these tools were not designed to measure the systemic risks that are the focus of the new regulatory landscape. This research addresses the urgent need to quantify this "benchmark-regulation gap." We introduce Bench-2-CoP, a novel, systematic framework that uses validated LLM-as-judge analysis to map the coverage of 194,955 questions from widely-used benchmarks against the EU AI Act’s taxonomy of model capabilities and propensities. Our findings reveal a profound misalignment: the evaluation ecosystem dedicates the vast majority of its focus to a narrow set of behavioral propensities. On average, benchmarks devote 61.6% of their regulatory-relevant questions to ’Tendency to hallucinate’ and 31.2% to ’Lack of performance reliability’, while critical functional capabilities are dangerously neglected. Crucially, capabilities central to loss-of-control scenarios, including evading human oversight, self-replication, and autonomous AI development, receive zero coverage in the entire benchmark corpus. This study provides the first comprehensive, quantitative analysis of this gap, demonstrating that current public benchmarks are insufficient, on their own, for providing the evidence of comprehensive risk assessment required for regulatory compliance and offering critical insights for the development of next-generation evaluation tools. Figure 1 : Overall workflow of our methodology. 1 Introduction The proliferation of powerful General Purpose AI (GPAI) models has been measured by an ecosystem of evaluation benchmarks. Prominent examples such as MMLU, Big Bench Hard, CommonsenseQA, and GPQA are now standard in developer reports for demonstrating model knowledge and reasoning. However, these benchmarks primarily focus on performance and foundational capabilities, not the holistic safety considerations required by emerging regulations like the EU AI Act. The Act, and its accompanying Code of Practice (CoP), demand a new paradigm of evaluation focused on "systemic risks" that arise from a complex interplay of a model’s capabilities (what it can do), propensities (its behavioral tendencies), and affordances (the context of its use). This creates a critical benchmark-regulation gap: the tools used for evaluation are not designed to measure compliance with the nuanced, lifecycle-based requirements of the new regulatory landscape. Analyses of the AI ecosystem show that industry practices lag in areas like formal adequacy assessments and incident reporting—precisely the areas that technical benchmarks do not cover. This gap is compounded by systemic limitations within the benchmarks themselves. A comprehensive review of leading LLM benchmarks reveals that many fail to meet standards of functionality (how well they test real-world capabilities) and integrity (resistance to manipulation) [ 17 ] . They often rely on static, exam-style formats poorly suited to dynamic behaviors like contextual reasoning or ethical sensitivity, and can suffer from evaluator bias and cultural narrowness. This flawed foundation encourages models to be optimized for benchmark-specific tricks rather than genuine intelligence. A critical symptom of this is benchmark leakage, where test data contaminates training sets, casting doubt on the validity of reported performance. For instance, OpenAI’s technical report confirms that GPT-4’s training incorporated data from benchmarks like GSM8K and MATH to enhance its reasoning abilities [ 20 , 6 , 10 ] . Such practices blur the line between evaluation and optimization, inflating performance claims and reinforcing the need for more robust evaluation protocols. Beyond these methodological flaws, current benchmarks are ill-equipped to assess the novel risks posed by increasingly autonomous AI agents. Researchers warn that agentic systems, capable of independent goal-directed action, may pursue objectives misaligned with human intent, leading to deceptive or harmful behaviors [ 3 ] . This has spurred calls for "safety-by-design" principles, such as the "Scientist AI" paradigm, which advocates for non-agentic models focused on assisting human reasoning and minimizing the risk of uncontrolled actions. This distinction between agentic and non-agentic systems highlights a profound shortcoming of current evaluation practices, which largely ignore how a model’s autonomy, deployment context, and interaction dynamics shape real-world systemic risk. This study aims to systematically quantify this benchmark-regulation gap. Our primary contribution is to provide the first comprehensive, empirical mapping of AI benchmark coverage against the EU AI Act’s requirements for GPAI with systemic risk. To do so, we introduce Bench-2-CoP, a novel, systemic framework that operationalizes regulatory text into a measurable taxonomy. The resulting analysis offers a data-driven resource for policymakers to refine the Code of Practice and for developers to design evaluation suites that ensure the safety and compliance of next-generation AI. Paper Organization The remainder of this paper is organized as follows. Section 2 reviews the dual landscape of AI evaluation practices and the emerging regulatory requirements of the EU AI Act. Section 3 details our Bench-2-CoP framework, including the development of our risk taxonomy and the multi-LLM mapping methodology. Section 4 presents the quantitative results of our analysis, identifying key alignment and gap areas. Finally, Section 5 discusses the implications of our findings, acknowledges the study’s limitations, and outlines future directions for research. 2 Background and Related Work 2.1 The Regulatory Landscape: The EU AI Act and its Code of Practice The EU AI Act establishes a comprehensive, risk-based regulatory framework for artificial intelligence. Its primary goal is to ensure that AI systems placed on the Union market are safe and respect fundamental rights. A key innovation of the Act is its specific focus on General Purpose AI (GPAI) models, with the most stringent requirements reserved for those designated as posing "systemic risks" (GPAI-SR). A model is typically considered to have systemic risk based on its computational power (measured in cumulative floating-point operations, or FLOPs), which serves as a proxy for its potential impact and advanced capabilities. The associated Code of Practice (CoP) operationalizes the Act’s principles into concrete commitments for developers of these high-impact models. The CoP defines systemic risks as severe, Union-wide threats to public health, safety, democracy, or the environment. To manage these, it mandates a continuous, lifecycle-based approach to risk management. This process is not a one-time check but an ongoing cycle of: • Risk Identification: Proactively identifying potential harms before and during development. • Risk Assessment: Evaluating the likelihood and severity of identified harms. • Mitigation Measures: Implementing and documenting technical and organizational safeguards to reduce risks. • Post-Deployment Monitoring: Continuously tracking model performance and real-world impacts to detect emergent risks. • Incident Reporting: Establishing protocols for documenting and reporting serious incidents to relevant authorities. A central concept in the CoP’s methodology is the classification of model characteristics into a three-part taxonomy, designed to create a structured understanding of how risks emerge: • Capabilities : The inherent functional abilities of a model—what it can do. These are context-independent properties, such as the ability to write code or process multiple data modalities. • Propensities : The behavioral tendencies of a model—what it is likely to do in certain situations. These are emergent behaviors observed during testing, such as a tendency to hallucinate or exhibit bias. • Affordances : The ways in which the model’s deployment context and use cases can enable or encourage certain outcomes. Affordances are not properties of the model alone but