Surveying the Operational Cybersecurity and Supply Chain Threat Landscape when Developing and Deploying AI Systems
Michael R. Smith, Joe Ingram · 2025
Read Surveying the Operational Cybersecurity and Supply Chain Threat Landscape when Developing and Deploying AI Systems by Michael R. Smith, Joe Ingram (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2508-20307.txt.
Introduction
AI systems bring new parts to attack: training data, model files, and the supply chain that builds them. Michael Smith and Joe Ingram survey those risks in a 2025 paper on running AI in the real world. Attackers increasingly aim at the behavior of the model itself, not just the network around it.
The authors walk the full life cycle of an AI system, from development to deployment, flagging where each stage can be hit. Goals like flooding a system with false alarms or quietly degrading accuracy need new defenses. Past exploits are reviewed for lessons, along with the field experience of the authors. The paper urges security frameworks shaped for AI rather than borrowed from classic software.
Catalog record
| CREATOR | Michael R. Smith, Joe Ingram |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 33 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2508-20307
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Surveying the Operational Cybersecurity and Supply Chain Threat Landscape when Developing and Deploying AI Systems Michael R. Smith, Joe Ingram Abstract The rise of AI has transformed the software and hardware landscape, enabling powerful capabilities through specialized infrastructures, large-scale data storage, and advanced hardware. However, these innovations introduce unique attack surfaces and objectives which traditional cybersecurity assessments often overlook. Cyber attackers are shifting their objectives from conventional goals like privilege escalation and network pivoting to manipulating AI outputs to achieve desired system effects, such as slowing system performance, flooding outputs with false positives, or degrading model accuracy. This paper serves to raise awareness of the novel cyber threats that are introduced when incorporating AI into a software system. We explore the operational cybersecurity and supply chain risks across the AI lifecycle, emphasizing the need for tailored security frameworks to address evolving threats in the AI-driven landscape. We highlight previous exploitations and provide insights from working in this area. By understanding these risks, organizations can better protect AI systems and ensure their reliability and resilience. Keywords: Artificial Intelligence, Cybersecurity, Adversarial Machine Learning, Supply Chain 1 Introduction The ascension of artificial intelligence (AI) has profoundly transformed the software and hardware landscape, driving advancements across industries and enabling innovative solutions to complex problems. AI systems rely on specialized software infrastructures and run-time environments to optimize intricate mathematical operations as well as large-scale data storage to manage AI models and data both on disk and in memory. Likewise, advancements in specialized hardware, such as graphical processing units (GPUs) and tensor processing units (TPUs), have enabled the efficient execution of computationally-intensive tasks. These advancements have collectively contributed to the remarkable effectiveness and efficiency of AI systems. However, the same components that make AI systems powerful also introduce unique and novel attack surfaces that can be exploited by malicious actors. Further, the AI portion is only one component in a larger system. From vulnerabilities in data pipelines to risks in hardware and software supply chains, the operational and cybersecurity concerns surrounding AI systems are multifaceted and demand specialized attention. Despite the growing prevalence of AI, traditional cybersecurity assessments often fail to adequately address the specific risks associated with AI systems and the associated changes in objectives by attackers. This oversight leaves critical gaps in security, exposing AI systems to threats that can compromise their integrity, reliability, and safety. This paper surveys many of the operational cybersecurity and supply chain concerns inherent in the development and deployment of AI systems. We distinguish this work from the large and growing body of literature on adversarial AI that exploits algorithmic vulnerabilities to manipulate the behavior of AI models. These attacks target weaknesses in the data, training process or inference mechanisms. For example, evasion attacks perturb inputs causing a model to make incorrect predictions ( Szegedy et al., 2013 ) . We provide a brief overview and how it relates to cyber vulnerabilities—specifically the motivation of an attacker. Further, adversarial AI as presented in many academic venues is often removed from operational considerations and may fail in real-world situations ( Zhang et al., 2022 ; Grosse et al., 2024 ) . Cyber attacks on the AI portion of a system are often more practical to achieve than the mathematical adversarial AI attacks (assuming cyber access) ( Apruzzese et al., 2023 ) . We examine vulnerabilities across the AI threat landscape including the learned models, specialized data, software infrastructure, run-time environment, and hardware, highlighting the unique challenges and research in each. By identifying these risks and proposing mitigation strategies, this paper aims to provide a comprehensive overview for assessing the security of AI systems against evolving threats in an increasingly AI-driven world. The following section provides an overview of what we mean by an AI system. Section 3 briefly overviews adversarial AI, the objectives that can be adopted by cyber attackers, and assumptions in executing attacks. Section 4 connects the objectives of adversarial AI and how it changes the concept of operations (CONOPS) of a cyber attack on an AI system. We then survey the AI attack landscape and how these objectives can be met by an attacker in each component of the AI threat landscape. We share examples in the academic literature, proofs-of-concept presented in cyber blogs and reports, and experiential insights where applicable. Section 6 presents the various stages of the AI development-to-deployment lifecycle and where these attacks can occur. We conclude with a discussion of the future of security in AI systems. 2 Brief Overview of AI Systems, Large Language Models, and Agentic Systems Figure 1: Typical AI lifecycle comprised of data curation and labeling, preprocessing, model development, deployment and operation of the AI system. Each phase exposes an additional attack surface. When we discuss AI systems, we refer to software and hardware systems where the AI represents a single component of a larger system. Figure 1 shows an exemplar AI lifecycle from development to deployment. While considerable effort is exerted to develop the model in the development stage, when deployed, the AI model is a software component connected within a larger system of sensors, data storage, and other system components. The AI component can be anything that is data-driven including complex object detection models or relatively simpler linear/logistic regression models. In natural language processing, AI has evolved significantly. Large Language Models (LLMs), such as OpenAI’s GPT ( Brown et al., 2020 ) or Google’s Gemini ( Team et al., 2024 ) , are advanced AI models trained on large corpora of text data to understand and generate human-like language. These models excel in tasks such as text summarization, translation, question answering, and code generation. Agentic systems extend the capabilities of LLMs by integrating autonomous decision-making and execution mechanisms. These systems are designed to perform complex workflows with little-to-no human intervention. They leverage LLMs for reasoning and communication while interfacing with external tools, APIs, and environments. For instance, an agentic system tasked with cybersecurity monitoring might autonomously analyze network logs, detect anomalies, and initiate mitigation actions. While autonomy enhances efficiency and scalability, it also introduces cybersecurity risks. For example, attackers can exploit vulnerabilities in input validation, API dependencies, or decision logic to manipulate the system’s actions or outcomes. Together, LLMs and agentic systems exemplify the growing sophistication of AI technologies, offering transformative capabilities while presenting new challenges in security. As these systems become increasingly integrated into critical applications, addressing their vulnerabilities will be paramount for ensuring their safe and effective deployment. 3 Brief Background on Adversarial AI and Attack Assumptions There is a growing body of adversarial attacks on AI to exploit vulnerabilities in models to manipulate their behavior, degrade performance, leak private information, or achieve other malicious objectives. These attacks typically have a narrow focus on the model or data and target weaknesses in data, training processes, or inference mechanisms. They pose a significant risk to AI systems deployed in security-critical applications. However, most are based on mathematical computations of gradients—which may not be true attack vectors in practical attacks on AI systems ( Apruzzese et al., 2023 ) . These same objectives can be achieved by cyber attackers. Figure 2: Evasion attack example. Left: Clean image that is correctly predicted as a school bus. Middle: Adversarial