Is your AI Model Accurate Enough? The Difficult Choices Behind Rigorous AI Development and the EU AI Act \acmConference@name ( \acmConference@shortname ), \acmConference@date , \acmConference@venue Conference: The 2026 ACM Conference on Fairness, Accountability, and Transparency; June 25–28, 2026; Montreal, QC, Canada The 2026 ACM Conference on Fairness, Accountability, and Transparency (FAccT ’26), June 25–28, 2026, Montreal, QC, Canada DOI: 10.1145/3805689.3806436 ISBN: 979-8-4007-2596-8/2026/06 CCS: Social and professional topics Governmental regulations CCS: Computing methodologies Machine learning CCS: Applied computing Law

Lucas G. Uberti-Bona Marin, Bram Rijsbosch, Kristof Meding, Gerasimos Spanakis, Gijs van Dijck, Konrad Kollnig · 2026

Read Is your AI Model Accurate Enough? The Difficult Choices Behind Rigorous AI Development and the EU AI Act \acmConference@name ( \acmConference@shortname ), \acmConference@date , \acmConference@venue Conference: The 2026 ACM Conference on Fairness, Accountability, and Transparency; June 25–28, 2026; Montreal, QC, Canada The 2026 ACM Conference on Fairness, Accountability, and Transparency (FAccT ’26), June 25–28, 2026, Montreal, QC, Canada DOI: 10.1145/3805689.3806436 ISBN: 979-8-4007-2596-8/2026/06 CCS: Social and professional topics Governmental regulations CCS: Computing methodologies Machine learning CCS: Applied computing Law by Lucas G. Uberti-Bona Marin, Bram Rijsbosch, Kristof Meding, Gerasimos Spanakis, Gijs van Dijck, Konrad Kollnig (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2604-03254.txt.

Introduction

Uberti-Bona and colleagues argue that AI accuracy is not a purely technical trait but a bundle of value-laden choices tied to context. They analyze four such choices under the EU AI Act, which demands a fitting accuracy level for high-risk systems: picking metrics, balancing them, measuring against typical data, and setting acceptance cutoffs. Making those choices plain, they say, guides regulators, auditors, and developers alike.

Catalog record

CREATORLucas G. Uberti-Bona Marin, Bram Rijsbosch, Kristof Meding, Gerasimos Spanakis, Gijs van Dijck, Konrad Kollnig
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE59671
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2604.03254

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Is your AI Model Accurate Enough? The Difficult Choices Behind Rigorous AI Development and the EU AI Act \acmConference@name ( \acmConference@shortname ), \acmConference@date , \acmConference@venue Conference: The 2026 ACM Conference on Fairness, Accountability, and Transparency; June 25–28, 2026; Montreal, QC, Canada The 2026 ACM Conference on Fairness, Accountability, and Transparency (FAccT ’26), June 25–28, 2026, Montreal, QC, Canada DOI: 10.1145/3805689.3806436 ISBN: 979-8-4007-2596-8/2026/06 CCS: Social and professional topics Governmental regulations CCS: Computing methodologies Machine learning CCS: Applied computing Law Lucas G. Uberti-Bona Marin, Bram Rijsbosch, Kristof Meding, Gerasimos Spanakis, Gijs van Dijck, Konrad Kollnig Abstract . Technical and legal debates frequently suggest that “accuracy” is an objective, measurable, and purely technical property. We challenge this view, showing that evaluating AI performance fundamentally depends on context-dependent normative decisions. These techno-normative choices are crucial for rigorous AI deployment, as they determine which errors are prioritised, how risks are distributed, and how trade-offs between competing objectives are resolved. This paper provides a legal-technical analysis of the choices that shape how accuracy is defined, measured, and assessed, using the 2024 European Union AI Act – which mandates an “appropriate level of accuracy” for high-risk systems – as a primary case study. We identify and analyse four choices central to any robust performance evaluation: (1) selecting metrics, (2) balancing multiple metrics, (3) measuring metrics against representative data, and (4) determining acceptance thresholds. For each choice, we study its relationship to the AI Act’s accuracy requirement and associated documentation obligations, show how its technical implementation embeds implicit or explicit assumptions about acceptable risks, errors, and trade-offs, and discuss the implications for the practical implementation of the AI Act by examples and related technical standards. By making the techno-normative dimensions of accuracy explicit, this paper contributes to broader interdisciplinary debates on AI governance and regulation, and offers specific guidance for regulators, auditors, and developers tasked with translating (legal) safety requirements into technical practice. † † cc-license: by 1. Introduction When is an AI model accurate enough to be deployed? Consider an AI-based skin cancer detection app that claims to detect melanoma with “99.8% accuracy” ( 1 ) . At first glance, such a system might appear highly promising. Yet, because melanoma is extremely rare in the general population, a system that always predicts the absence of melanoma – when looking at the whole population – could achieve a similarly high accuracy score, while failing to be any useful. High accuracy, thus, on its own, does not necessarily imply meaningful or safe performance. This example illustrates some of the challenges in AI model performance evaluations (also referred to as accuracy evaluations, or model testing). The objective of performance evaluations is to generate quantitative evidence – most commonly in the form of performance metric scores, such as accuracy – about how well a model performs with respect to its intended purpose. From a legal perspective, this stage thus plays a crucial role: it is where final decisions are made about whether a model’s performance is adequate for deployment, and where risks associated with the use of a model should be assessed and documented. Recognising this importance, the 2024 EU AI Act introduced a legal requirement that directly targets the performance evaluation stage of AI model development, requiring that high-risk AI systems (which an AI-based melanoma detector would likely qualify as) “achieve an appropriate level of accuracy” before they may be placed on the EU market ( 10 ) , hereby aiming to ensure that only safe and trustworthy AI systems are used in the EU. Performance evaluations are neither a trivial nor a purely technical exercise. They require making a series of interdependent design choices – such as selecting and balancing performance metrics, defining test procedures, and setting performance thresholds – that are often highly context-dependent and normatively-laden: They encode explicit or implicit assumptions about what errors are acceptable, which risks are prioritised, and how trade-offs are taken into account by a model. In the case of an AI-based melanoma detection system, for example, a performance metric like accuracy treats false negatives and false positives as equally important. This reflects a judgment about the relevant risks of melanoma going undiagnosed versus healthy moles incorrectly being flagged as melanoma. Figure with four quadrants each of which represents one of the four techno-normative choices made during evaluation presented in this paper Figure 1 . Illustration of the four techno-normative choices of performance evaluations, as examined in this paper. Depending on (1) the chosen metric(s), (2) how they are balanced, (3) how they are measured, and (4) the way in which the acceptance region is defined, different models may be considered more or less accurate. These four choices are thus both central to performance evaluations and directly relevant to the AI Act’s requirement for high-risk AI systems to “achieve an appropriate level of accuracy”. Figure with four quadrants each of which represents one of the four techno-normative choices made during evaluation presented in this paper We refer to such choices as techno-normative choices , as both the choice and its technical implementation determine how these normative assumptions materialise in practice. These normative dimensions can easily be taken for granted in data science and ML research ( 37 ; 33 ) . Insufficient attention to the implications of these choices, however, can have serious consequences for affected persons, particularly in high-stakes domains such as healthcare, public administration, and law enforcement. Against this backdrop, the open-ended formulation of the AI Act’s accuracy requirement raises difficult questions. What then counts as an appropriate level of accuracy? And how can this be meaningfully assessed across a diverse range of high-risk use cases? At present, neither legal guidelines, nor technical practices provide clear answers to these questions. On the technical side, an extensive body of research has examined the challenges of performance evaluations ( 6 ; 26 ; 31 ; 40 ) , yet robust performance evaluations in real-world settings remain challenging ( 23 ; 37 ; 30 ) . A key source of this difficulty is the mismatch between the dominant learner-centric evaluation paradigm in machine learning (focused on comparing models to one another) and the application-centric perspective often required in high-risk deployment contexts, where performance depends on broader system-level considerations ( 23 ) . The AI Act’s accuracy requirement implicitly calls for the latter, while evaluation practices in the ML literature continue to largely reflect the former ( 23 ; 6 ; 26 ) . On the legal side, there currently are no adopted standards, benchmarks, guidelines, or case law, that help clarify what constitutes an “appropriate level of accuracy” under the AI Act. Moreover, the accuracy requirement has so far received little focused attention in legal scholarship. Although work is underway on EU harmonised standards that intend to further operationalise the AI Act’s requirements ( 13 ) , the statistical complexity of AI models and diversity of application contexts make this process challenging. Moreover, standardisation alone is unlikely to resolve the underlying normative and context-specific complexities of performance evaluations ( 29 ) . Taken together, these gaps create a risk that the AI Act’s accuracy obligation will prove difficult to implement, evaluate, and enforce in practice. As a result, compliance could default to more superficial,

Read the complete text