Preventing Adversarial AI Attacks Against Autonomous Situational Awareness: A Maritime Case Study
Mathew J. Walter, Aaron Barrett, Kimberly Tam · 2025
Read Preventing Adversarial AI Attacks Against Autonomous Situational Awareness: A Maritime Case Study by Mathew J. Walter, Aaron Barrett, Kimberly Tam (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2505-21609.txt.
Introduction
Mathew Walter, Aaron Barrett, and Kimberly Tam wrote this 2025 paper on shielding ship AI from trick inputs. Self-driving vessels lean on AI to read their surroundings, and attackers can craft inputs that fool the models. Past defenses guard one model at a time and lack good security scores. The authors propose a wider shield, which they call Data Fusion Cyber Resilience.
The shield blends several inputs through data fusion and grades the setup with a new security score. In real-world demos and number tests, fused systems lost far less ground under attack than single-input ones, in some attack types losing nothing at all. The score also helps the system doubt a bad contact and choose more wisely.
Catalog record
| CREATOR | Mathew J. Walter, Aaron Barrett, Kimberly Tam |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 73 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2505-21609
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Preventing Adversarial AI Attacks Against Autonomous Situational Awareness: A Maritime Case Study Mathew J. Walter, Aaron Barrett, Kimberly Tam Abstract Adversarial artificial intelligence (AI) attacks pose a significant threat to autonomous transportation, such as maritime vessels, that rely on AI components. Malicious actors can exploit these systems to deceive and manipulate AI-driven operations. This paper addresses three critical research challenges associated with adversarial AI: the limited scope of traditional defences, inadequate security metrics, and the need to build resilience beyond model-level defences. To address these challenges, we propose building defences utilising multiple inputs and data fusion to create defensive components and an AI security metric as a novel approach toward developing more secure AI systems. We name this approach the Data Fusion Cyber Resilience (DFCR) method, and we evaluate it through real-world demonstrations and comprehensive quantitative analyses, comparing a system built with the DFCR method against single-input models and models utilising existing state-of-the-art defences. The findings show that the DFCR approach significantly enhances resilience against adversarial machine learning attacks in maritime autonomous system operations, achieving up to a 35% reduction in loss for successful multi-pronged perturbation attacks, up to a 100% reduction in loss for successful adversarial patch attacks and up to 100% reduction in loss for successful spoofing attacks when using these more resilient systems. We demonstrate how DFCR and DFCR confidence scores can reduce adversarial AI contact confidence and improve decision-making by the system, even when typical adversarial defences have been compromised. Ultimately, this work contributes to the development of more secure and resilient AI-driven systems against adversarial attacks. I Introduction Artificial intelligence (AI) is rapidly permeating various aspects of our lives, offering significant benefits through task automation. This includes automating cyber-physical systems, such as transportation and industrial operations. The maritime sector is one of several domains embracing AI to capitalise on many benefits, ensuring organisations remain competitive. The International Maritime Organisation (IMO) categorises autonomy into four degrees, with the highest levels being degrees three and four. The proposed benefits of higher degrees of autonomy include significant operational benefits such as reduced crew and greater payload capacity, military utilisation in dangerous, contested, or Global Navigation Satellite System (GNSS) degraded/denied environments, greater automated decision making as well as increased safety and social benefits [ 1 , 2 , 3 , 4 , 5 , 6 , 7 , 8 ] . Whilst AI can provide significant operational benefits, current research shows that AI models can harbour a significant number of vulnerabilities unique to AI systems and processes if they are not developed to be resilient. The terms adversarial AI (AAI) and adversarial machine learning (AML) were coined to describe these vulnerabilities [ 9 , 10 ] . Organisations have acknowledged this threat by formulating measures such as OWASP’s machine learning vulnerabilities top 10, NIST’s AI Risk Management Framework (AI RMF), and MITRE’s Adversarial Threat Landscape for Artificial-Intelligence Systems (ATLAS) threat modelling. Globally, government organisations, such as the United Kingdom’s National Cyber Security Centre (NCSC), contributed to this cause with the “Guidelines for Secure AI System Development” in 2023 [ 11 ] . A significant emphasis was placed on adopting a secure-by-design development approach [ 12 , 11 ] . Many of the AAI concerns have already materialised within the domains of AAI and explainable AI (XAI) [ 13 , 14 ] , including adversarial attacks against autonomous vehicles [ 15 , 16 , 17 ] . Furthermore, attacks against AI within critical national infrastructure (CNI) and transportation have the potential for devastating consequences, resulting in a significant loss of money, reputation and life [ 18 ] . Such threats become increasingly likely with the exponential-like uptake in AI, greater reliance on AI for critical decision-making, and more effective AAI methods expanding the threat landscape. The defence and resilience of AI systems remains an underdeveloped field with numerous key challenges. The limitations of AAI defences can be grouped into three main categories, which we aim to address in this work: 1. Limited Scope of Traditional Defences : Traditional AAI defences are often restricted to countering a single type of attack. They can lack consistent accuracy, and many operate effectively within limited or restricted conditions. Therefore, we explore the development of defences that are effective across multiple attack types. 2. Inadequate Security Metrics : Existing metrics, such as model confidence, offer limited insight into attacks and are insufficient for integrating security into the system’s decision-making process. Existing metrics to measure and understand risks from AAI are very limited. We emphasise the need for security and robustness metrics, such as the security-inclusive confidence score proposed in this paper. 3. Resilience Beyond Model Defences : Existing traditional defences do not consider resilience, the ability to continue functioning during an attack, which is critical for autonomy. By adopting a defence-in-depth approach, we investigate whether it is possible to create a more robust system to mitigate the effects of attacks — even if the model’s defences are bypassed. This work proposes and evaluates a novel approach, the Data Fusion Cyber Resilience (DFCR) method, to build more secure AI systems by using multiple input sources, data fusion methods, and defence-oriented components tailored to the specific application and environment to address these challenges. This approach enhances system security and resilience, effectively overcoming the aforementioned limitations compared to single AAI defence methods such as input image compression or adversarial training. Moreover, we demonstrate how the proposed method can provide defence over a range of attacks, rather than being limited to mitigating a single type of attack, unlike most AAI defence methods. It can also be utilised to generate metrics which incorporate system security and develop more resilient AI systems. In this paper, we emphasise an important terminology distinction between AI models and AI systems. AI models refer specifically to standalone models, while AI systems incorporate the model as part of a broader framework, including processes such as data preprocessing, feature extraction, model defences and post-processing. We measured the impact of the DFCR method in two ways. First, we conducted sea trials for both AAI and AAI defences to evaluate their real-world practicality. This was a critical aspect of the study, as previous research [ 19 , 20 ] highlighted that evaluations conducted in low-entropy laboratory settings often exhibit different behaviours when applied in the complex and dynamic conditions of real-world environments. To enhance the realism of the evaluations, we employed maritime autonomous systems (MAS) during these trials. Since real-world environments are the ultimate intended operational domain for AI tools, to better understand the actual effects of attacks and defences, this study emphasised evaluating defences in situ. Sea trials enabled one to consider and compare attacks on both operational and theoretical levels, providing insights into the limitations and practicality of the methods under real-world conditions and revealing notable disparities between laboratory-based and in situ AAI research. The second method of impact measurement quantitatively evaluated the DFCR approach by comparing it against existing state-of-the-art defences and single-input models to assess the attack success rate. Through these methods of assessing impact, we are able to evaluate and demonstrate the paper’s novel contributions: 1. Building defences utilising multiple inputs and data fusion to create defensive components (DFCR), and a novel AI security