Validity Is What You Need

Sebastian Benthall, Andrew Clark · 2025

Read Validity Is What You Need by Sebastian Benthall, Andrew Clark (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2510-27628.txt.

Introduction

Sebastian Benthall and Andrew Clark argue in 2025 that agentic AI should be judged as delivered software, not as a base model. Agentic AI means programs that act on their own inside complex firms. Large language models sparked the boom, but the systems at stake are full applications, which are built products put to work for users.

Success thus rests on checks by end users and key stakeholders, not on base model scores. The tools for such checks differ sharply from model tests. With sound checks in place, the authors note, simpler and faster models can often run the core logic. Their claim is blunt: for agentic AI, validity is what you need.

Catalog record

CREATORSebastian Benthall, Andrew Clark
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE30 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2510-27628

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Validity Is What You Need Sebastian Benthall, Andrew Clark Abstract . While AI agents have long been discussed and studied in computer science, today’s Agentic AI systems are something new. We consider other definitions of Agentic AI and propose a new realist definition. Agentic AI is a software delivery mechanism, comparable to software as a service (SaaS), which puts an application to work autonomously in a complex enterprise setting. Recent advances in large language models (LLMs) as foundation models have driven excitement in Agentic AI. We note, however, that Agentic AI systems are primarily applications, not foundations, and so their success depends on validation by end users and principal stakeholders. The tools and techniques needed by the principal users to validate their applications are quite different from the tools and techniques used to evaluate foundation models. Ironically, with good validation measures in place, in many cases the foundation models can be replaced with much simpler, faster, and more interpretable models that handle core logic. When it comes to Agentic AI, validity is what you need. LLMs are one option that might achieve it. There is undeniably a current wave of excitement and investment in Agentic AI (Pounds, 2024), which is envisioned as an application of foundation models, such as LLMs, to perceive, reason, act, and learn. We examine several definitions of Agentic AI, as well as industry trends and arguments in the research literature, to assess the current state and future of Agentic AI. After considering definitions of Agentic AI from computer science, law, and industry, we propose our own ‘realist’ definition of agentic AI. Agentic AI, today, refers to a software delivery mechanism, similar to SaaS, which puts a complex, multi-step application to work within or between enterprises or their customers. It aspires to replace or significantly augment employees in a complex environment with multiple stakeholders and limits of information flow. Building an agentic AI in those conditions is much like the economic practice of mechanism design: building the mechanism that, when used, optimizes the design objectives. Given the power of AI systems to optimize, it has long been recognized that one of the greatest challenges of AI is alignment or, more technically, specifying the system’s goals ( Ji et al. 2023 ) . Agentic AI, being situated and applied in complex domains, requires a complex specification which must be validated and verified at the application level by the end-using person. Thus, the frontier for innovation in Agentic AI is in the design and validation process, turning social and organizational goals into metrics of success and operational tests. More complex agentic applications will require more complex testing and governance systems, and these testing and governance systems will be what trains the foundational technologies to serve as applications. We propose a multi-stage process for designing agentic systems, and question how AI can improve on this design and governance process. The irony of agentic AI is that while its use is motivated by the new capacity of Foundational AI technologies, effective AI validation may well reduce the need for these foundation models over time. When goals are well specified, AI systems must use specific, rather than general, intelligence, and compete on performance guarantees and marginal efficiency. Smaller, interpretable models are likely to be the key components of Agentic AI systems that endure to maturity. 1. Agency and AI This paper is written during the first point in history when “agentic AI” has entered the mainstream, industrial vocabulary. However, it is certainly not the first time artificially intelligent agents have been discussed in the AI scientific community. One challenge with reasoning about “Agentic AI” is that the terms “agent” and “intelligent” are evolving. We note that Larry’s Tesler’s adage that “AI is whatever hasn’t been done yet” withstands the test of time – our understanding of “intelligence” often excludes those tasks that automation has mastered. Likewise, we consider how the meaning of artificial “agency” has evolved over time. We consider classical computer science, legal, contemporary computer science, and industry definitions. We then clarify how we will use the term, positing a realistic definition fit to the current business use cases. The word “agent” comes originally from the Latin verb agere , meaning “to do or to drive”, and the past participle agens , meaning “doing” or “one who is doing”. In this basic sense, anything that does anything is an agent. The simplest model of an artificial agent is based on a sensorimotor loop, with sensors providing information from the environment, some kind of decision process (which may be entirely reflexive) determining next actions, and actuators which translate those actions back into the environment. For decades in the AI community, an intelligent agent has been defined to be an agent which acts effectively and purposefully to achieve goals. The very widely used Russell and Norvig textbook ( Russell and Norvig 1995 ) uses the “intelligent agent” as its unifying theme across many paradigms of AI. They take care to distinguish agents from mere programs. Agents, unlike programs, operate under autonomous control, perceive their environment, persist for a prolonged time period, adapt to change, and take on goals. Rational agents are those that act to achieve the best expected outcome. In more recent work, Russell 2019 emphasizes that agents are goal-oriented. Russell’s sense of agency is closely connected to the formal model of a utility maximizing agent, and especially one that maximizes utility over time. These themes are reiterated by other contemporary researchers (such as Chan et al. 2023 ) who distinguish “agency” from mere software. “Agents” accomplish goals less through specification of operations, and more through goal directedness. They also act with less human mediation, and more long-term planning. This definition of intelligent agency fits well into the science of rational behavior that has been well established over time. The Bellman equation, developed by Bellman 1954 , forms the mathematical backbone of both economic optimization and modern reinforcement learning: V(x)=\max_{a\in\Gamma(x)}\{F(x,a)+\beta V(T(x,a))\} Where V(x) is the value function at state x, F(x,a) is the immediate reward for taking action a in state x, \beta is the discount factor, and T is the transition function to next period states. In this paradigm of agent rationality, the agent is trained to learn an optimal policy that chooses an optimal action a at any state x. It is no accident that “agentic AI” often builds on reinforcement learning today. However, these criteria of ‘agency’ are not as precise as they may look. There are many trivial ways to satisfy a Bellman equation. Modern LLMs are models trained to minimize a loss function on the task of autoencoding massive datasets. A user of a modern chatbot queries these models and gets the “best” trained response, but this is a form of pattern matching ( Shojaee et al. 2025 ) rather than reasoning towards some other externally specified goal. Because these services are equipped with “like” and “unlike” buttons, or other ways of allowing users to flag their (dis)approval of responses, there is a widespread practice of Reinforcement Learning from Human Feedback (RLHF) ( Bai et al. 2022 ) , whereby these chatbots are gradually trained, much like social media curation algorithms, to maximize positive user engagement. They

Read the complete text