A Framework for the Assurance of AI-Enabled Systems

Ariel S. Kapusta, David Jin, Peter M. Teague, Robert A. Houston, Jonathan B. Elliott, Grace Y. Park, Shelby S. Holdren · 2025

Read A Framework for the Assurance of AI-Enabled Systems by Ariel S. Kapusta, David Jin, Peter M. Teague, Robert A. Houston, Jonathan B. Elliott, Grace Y. Park, Shelby S. Holdren (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2504-16937.txt.

Introduction

Ariel Kapusta and co-authors frame how the United States military can trust AI systems in a 2025 paper. Defense leaders want AI fielded fast, yet learning systems raise fresh safety, security, and ethics doubts that slow uptake. The team proposes a claims based frame for risk and assurance.

Programs would state clear claims that a system meets its mission without undue risk, then back each claim across the system life. Shared definitions keep talks on assurance straight. The frame suits all buying paths and aims to field strong tools swiftly without skipping hard risks or trust.

Catalog record

CREATORAriel S. Kapusta, David Jin, Peter M. Teague, Robert A. Houston, Jonathan B. Elliott, Grace Y. Park, Shelby S. Holdren
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE39 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2504-16937

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

A Framework for the Assurance of AI-Enabled Systems Ariel S. Kapusta, David Jin, Peter M. Teague, Robert A. Houston, Jonathan B. Elliott, Grace Y. Park, Shelby S. Holdren Abstract The United States Department of Defense (DOD) looks to accelerate the development and deployment of AI capabilities across a wide spectrum of defense applications to maintain strategic advantages. However, many common features of AI algorithms that make them powerful, such as capacity for learning, large-scale data ingestion, and problem-solving, raise new technical, security, and ethical challenges. These challenges may hinder adoption due to uncertainty in development, testing, assurance, processes, and requirements. Trustworthiness through assurance is essential to achieve the expected value from AI. This paper proposes a claims-based framework for risk management and assurance of AI systems that addresses the competing needs for faster deployment, successful adoption, and rigorous evaluation. This framework supports programs across all acquisition pathways provide grounds for sufficient confidence that an AI-enabled system (AIES) meets its intended mission goals without introducing unacceptable risks throughout its lifecycle. The paper’s contributions are a framework process for AI assurance, a set of relevant definitions to enable constructive conversations on the topic of AI assurance, and a discussion of important considerations in AI assurance. The framework aims to provide the DOD a robust yet efficient mechanism for swiftly fielding effective AI capabilities without overlooking critical risks or undermining stakeholder trust. a]Office of the Chief Digital and Artificial Intelligence Officer, U.S. Department of Defense, 9010 Defense Pentagon, Washington, D.C., USA b]The MITRE Corporation, 7515 Colshire Drive, McLean, VA, USA c]John Hopkins University Applied Physics Laboratory, 11100 Johns Hopkins Rd, Laurel, MD, USA keywords Artificial Intelligence (AI), AI Assurance, Claims-based Assurance, Risk Management, Machine Learning (ML), Autonomous Systems, Test and Evaluation (T&E), Safety and Security, DODD 3000.09, System-of-Systems, Continuous Oversight, Generative AI, Adversarial ML, Legal Compliance, DevSecOps/MLOps Integration † † authorinfo: Further author information: (Send correspondence to David Jin) David Jin: E-mail: [email protected] Ariel S. Kapusta: E-mail: [email protected] Cleared for Open Publication on Mar 11, 2025 by the U.S. DOD Office of Prepublication and Security Review, Case Number 25-T-1410. Approved for Public Release, Distribution Unlimited. MITRE Public Release Case Number 25-0757 1 INTRODUCTION The United States (U.s.) Department of Defense (DOD) recognizes the transformative potential of artificial intelligence (AI) in augmenting mission-critical systems for improved decision support, enhanced situational awareness, and autonomous operations. From data analytics for intelligence or logistics to sophisticated autonomous platforms in land, air, or maritime environments, AI-enabled systems (AIES) can offer improved speed and precision in addressing fast-evolving threats. The potential for AI reaches across a wide spectrum of defense applications. Some AIES focus on analytics and decision-support, synthesizing information from disparate data streams and generating insights to guide human decision-makers. Others involve more advanced autonomy with humans in-the-loop, on-the-loop, or out-of-the-loop. AI functions may drive unmanned platforms in air, land, or maritime domains, granting a capability to adapt to changing mission parameters without human intervention at each step. In many cases, the AI components may operate in environments where conditions change rapidly, inputs may deviate from training data, and unexpected scenarios may arise. Although it is unlikely for a passenger jet to land on a highway, autonomous cars are expected to avoid collisions, even in such rare events. However, many of the same qualities that make AI so powerful—its capacity for learning, large-scale data ingestion, and problem-solving—raise new technical, security, and ethical challenges. AIES may experience drift, such as from input data drift, sensor degradation, actuation failures, or material fatigue. Some AI algorithms are designed to learn over time (either online or in increments), which is, functionally, internal drift in the AIES. Given these complexities, many traditional T&E, safety, and cybersecurity processes—originally developed around deterministic software or hardware systems—can prove insufficient. Defense organizations accustomed to validating deterministic, conventional software systems may find themselves unprepared for the complexity, variability, vulnerabilities, uncertainties, brittleness, and opacity of many AIES. Despite these challenges, to remain competitive the DOD must field AI-based solutions more quickly than has historically been possible. Many potential AI-based solutions have seen slow adoption, due to uncertainty in development, testing, assurance, acquisitions, processes, and requirements. This paper introduces a framework for risk management and assurance of AI systems that addresses the twin imperatives of speed and rigor across the entire lifecycle of the system. For consistency in understanding, we provide terminology definitions in Section 2 . By centering on explicit “assurance claims,” the framework provides a systematic way to capture all relevant evidence for a given AI-enabled system, from training data provenance to cybersecurity test reports and ethical compliance artifacts. In doing so, it brings together several often-stovepiped perspectives: safety, security, legal, and performance. The framework’s process requires the creation of explicit arguments supported by evidence which ensure risks are understood and managed as required. The framework requires the creation of specific artifacts, including an assurance plan that defines how assurance is maintained over the system’s lifecycle. These artifacts also provide information to stakeholders, supporting confidence in the AI system by those who must use, interact with, or be subjected to the system. With clear steps, a modular and iterative approach, tangible completion criteria, and explicit approval gates, this framework enables a faster path for developers to get new products into the hands of users. The clarity and consistency in requirements to achieve assurance are the primary means of acceleration provided by the framework. Future work to provide templates, guides, and examples that fit within the framework will further enable acceleration of development and adoption efforts. For example, a database of hazards common to AIES could increase consistency, quality, and speed of hazard identification and risk assessment activities. The AI assurance process extends across the lifecycle of an AI system, from system development, through system fielding, and throughout system operations. Figure 1 illustrates the three phases of the AI assurance process: prepare for assurance, establish assurance, and maintain assurance Figure 1: Phases of the AI assurance process This paper’s key contributions are as follows: • A framework process for AI assurance. • Relevant definitions that may enable constructive conversations on the topic of AI assurance. • Discussion of important considerations in AI assurance. 2 Definitions For consistency in understanding, we provide the following terminology definitions that are used in this paper. D-1 AI System An engineered or machine-based system that can, for a given set of objectives, generate outputs such as predictions, recommendations, or decisions influencing real or virtual environments. AI systems are designed to operate with varying levels of autonomy. [Source NIST AI RMF [ 1 ] ] D-2 AI Assurance Process The process of managing risk, ensuring effectiveness, and leveraging the DOD AI ethical principles throughout the lifecycle of an AI system (D-1) to establish and maintain its assurance (D-3). D-3 Assurance Grounds for sufficient confidence that an AI system (D-1), while operating within its defined scope, will achieve its intended outcomes without introducing unacceptable risks, throughout its lifecycle. D-4 Assurance Case A structured argument, supported by a body of evidence, demonstrating that an assurance claim (D-5) is true. D-5 Assurance Claim A statement that relevant stakeholders must have sufficient confidence in for the assurance (D-3) of an AI system (D-1) to be established. NOTE: The top-level

Read the complete text