Securing AI Systems: A Guide to Known Attacks and Impacts Note: This paper is an extended version of the document ( (2025) J-AISI ) published by Japan AI Safety Institute. CCS: General and reference Surveys and overviews CCS: Computing methodologies Artificial intelligence CCS: Security and privacy
Naoto Kiribuchi, Kengo Zenitani, Takayuki Semitsu · 2025
Read Securing AI Systems: A Guide to Known Attacks and Impacts Note: This paper is an extended version of the document ( (2025) J-AISI ) published by Japan AI Safety Institute. CCS: General and reference Surveys and overviews CCS: Computing methodologies Artificial intelligence CCS: Security and privacy by Naoto Kiribuchi, Kengo Zenitani, Takayuki Semitsu (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2506-23296.txt.
Introduction
Naoto Kiribuchi, Kengo Zenitani, and Takayuki Semitsu surveyed known attacks on AI systems in two thousand twenty-five and tied each to its harm. AI built into wider systems faces threats that exploit quirks of learning models, in both predictive and generative forms. Their guide is an extended version of a document from the Japan AI Safety Institute.
They name eleven major attack types and map each to impacts such as data leakage, system takeover, and resource drain. Those impacts are sorted by the classic security triad of confidentiality, integrity, and availability. The guide aims to give developers, security staff, and policymakers a shared base for spotting AI risks and building defences.
Catalog record
| CREATOR | Naoto Kiribuchi, Kengo Zenitani, Takayuki Semitsu |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 84 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2506-23296
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Securing AI Systems: A Guide to Known Attacks and Impacts Note: This paper is an extended version of the document ( (2025) J-AISI ) published by Japan AI Safety Institute. CCS: General and reference Surveys and overviews CCS: Computing methodologies Artificial intelligence CCS: Security and privacy Systems security Naoto Kiribuchi, Kengo Zenitani, Takayuki Semitsu Abstract . Embedded into information systems, artificial intelligence (AI) faces security threats that exploit AI-specific vulnerabilities. This paper provides an accessible overview of adversarial attacks unique to predictive and generative AI systems. We identify eleven major attack types and explicitly link attack techniques to their impacts—including information leakage, system compromise, and resource exhaustion—mapped to the confidentiality, integrity, and availability (CIA) security triad. We aim to equip researchers, developers, security practitioners, and policymakers, even those without specialized AI security expertise, with foundational knowledge to recognize AI-specific risks and implement effective defenses, thereby enhancing the overall security posture of AI systems. Figure 1. Overview of known eleven major attacks (A–K) on AI systems. The figure shows a schematic of an AI system divided into training and operation environments. It identifies eleven attack types (labeled A to K), targeting components such as the training data, pre-trained and fine-tuned AI models, application inputs and outputs, internal and external knowledge sources, and integrated external systems. Data flow arrows illustrate interactions between these components, indicating precisely where each attack occurs within the AI system structure. 1. Introduction The field of artificial intelligence (AI) has undergone rapid advancement in recent years, resulting in revolutionary changes across diverse domains, including healthcare, finance, and content generation. Innovations have not only expanded the potential applications of AI but also significantly increased its societal influence. As AI systems become integrated into critical infrastructures and decision-making processes, there has been a notable increase in global awareness and demand for ensuring their safe, reliable, and ethical operation — collectively referred to as AI safety . In response to these mounting global concerns, Japan has assumed a pioneering role by spearheading the Hiroshima AI Process ( of the Group of Seven G7 ) , a comprehensive policy framework that was adopted in 2023. Concurrently, the United Kingdom and the United States established national AI Safety Institutes 1 1 1 The UK AI Safety Institute was renamed the AI Security Institute on February 14, 2025 ( Kyle 2025 ) . 2 2 2 The plan to transform the US AI Safety Institute into the US Center for AI Standards and Innovation (CAISI) was announced on June 3, 2025 ( of Commerce 2025 ) . to address AI-related risks. In alignment with these global developments, the Japan AI Safety Institute (J-AISI) was established in February 2024. It functions as Japan’s central hub for AI safety, with the mission of aggregating technical expertise and promoting both domestic and international collaboration toward the responsible deployment of AI technologies. The secretariat of J-AISI is situated within the Information-technology Promotion Agency (IPA), thereby ensuring effective coordination and support for its activities ( (2024b) J-AISI ) . Within the broader domain of AI safety, AI security has emerged as a crucial subset focused explicitly on protecting AI systems from malicious actions designed to compromise their confidentiality, integrity, or availability. As AI systems handle increasingly sensitive data and perform critical societal functions, failures in AI security could result in severe consequences, including privacy violations, operational disruptions, economic damages, and threats to public safety. Consequently, a clear understanding of AI security risks and practical strategies to mitigate them is vital, not only for AI specialists but also for developers, security practitioners, and stakeholders responsible for deploying, operating, and managing AI systems across various sectors. This paper contributes to AI security by providing broad stakeholders, especially security practitioners, including those without specialized AI security, with a comprehensive yet highly accessible overview of attacks against AI systems shown by Figure 1 . A comprehensive survey of the latest and most advanced attacks is beyond the scope of this paper. Due to the abundance of research on AI security, it can be challenging for practitioners, especially those who are new to the field, to develop a coherent and comprehensive understanding of its landscape and underlying context. Accordingly, a practical guide is needed to support a comprehensive understanding of AI security. The distinguishing features of this work are its emphasis on the impacts of attacks on AI systems and intuitive visual summaries illustrating their relationships. By explicitly linking each established attack type with specific impacts on AI systems, this paper furnishes readers with actionable insights useful for security management practices. Developers and practitioners can leverage this structured knowledge to perform risk assessments or red-teaming exercises and identify critical attack vectors specifically relevant to their own AI systems. Furthermore, this paper summarizes the referenced attack literature in Table 4 organized by modality (e.g., text, image, or tabular), model type (e.g., neural network, logistic regression, or decision tree), and attacker’s capabilities. This table offers readers practical guidance for evaluating realistic risk scenarios within broader cybersecurity contexts. In summary, the primary contributions of this paper are as follows: • Identifying eleven attack types on AI systems by incorporating globally established references from various AI relevant stakeholders. • Explicitly linking each attack type to its specific impacts on AI systems, supporting practical risk assessments, including red-teaming exercises. • Identifying attacker’s capabilities associated with each attack, supporting realistic evaluations of AI security scenarios and risks. • Providing intuitive summaries that facilitate an easy understanding of attack mechanisms and their implications, suitable for developers and security practitioners without specialized AI security expertise. • Organizing referenced attack literature according to data modalities and AI model types, serving as a clear roadmap for readers pursuing further in-depth study. The remainder of this paper is organized as follows. Section 2 reviews related works in AI security, contextualizing our contributions within existing frameworks and literature. Section 3 details the AI system architecture and defines the scope of the attacks considered in this paper. Section 4 introduces our categorization of eleven major attack types and discusses their potential impacts on AI systems, mapped to the CIA security triad. Section 5 provides in-depth descriptions of each identified attack, including its mechanisms, attacker capabilities, and affected AI modalities and model types, supported by examples from the literature. Finally, we conclude the paper by summarizing our key findings and suggesting directions for future research in Section 6 . 2. Related Works We review related works on adversarial threats to AI systems, highlighting their contributions and limitations to contextualize this paper within AI security. We deeply respect the significant contributions of the existing frameworks and literature discussed here, which have led the global conversation on AI security. The purpose of this paper is not to negate or replace these works but rather to provide additional insights that can contribute to their practical application. By highlighting the differences and supplementary perspectives, we aim to promote understanding among these works, point out key attack characteristics especially from the viewpoint of security practitioners, and contribute to the future development of international discussions. 2.1. Academic Surveys Several academic surveys have systematically analyzed adversarial threats and defense methods relevant to predictive AI systems. Kong et al. provide a comprehensive classification that focuses on