Landseer: Exploring the Machine Learning Defense Landscape
Ayushi Sharma, Rosemary Agbozo, Santiago Torres-Arias, Zahra Ghodsi · 2026
Read Landseer: Exploring the Machine Learning Defense Landscape by Ayushi Sharma, Rosemary Agbozo, Santiago Torres-Arias, Zahra Ghodsi (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-27148.txt.
Introduction
Sharma and co-authors offer a modular frame for combining machine learning defenses that are usually tested alone. Their Landseer system wraps each defense as a plug-in module and runs tests across many measures, alone and in combination. A first survey found 35 current defenses, and after filtering for work that could be rerun, the tests exposed gaps in how well defenses reproduce across families. The authors present the frame as a base for more reliable combined defenses in real deployments.
Catalog record
| CREATOR | Ayushi Sharma, Rosemary Agbozo, Santiago Torres-Arias, Zahra Ghodsi |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 89762 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2605.27148
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Landseer: Exploring the Machine Learning Defense Landscape Ayushi Sharma, Rosemary Agbozo, Santiago Torres-Arias, Zahra Ghodsi Abstract . Machine learning systems face diverse threats that undermine robustness, privacy, and fairness. Although many defenses have been proposed, each typically addresses a single risk in isolation. Real-world deployments, however, require these defenses to be composed to meet multiple guarantees simultaneously. The process of composing defenses is complex and not well understood, and its impact on performance and security remains unclear. We present Landseer , a modular framework for integrating machine learning (ML) defenses into the ML lifecycle and systematically evaluating their composition. Landseer encapsulates defenses as containerized modules, allowing existing and new techniques to be plugged in with minimal effort. Its evaluation engine automates experiments across multiple metrics, supporting the study of defenses both individually and in combination. In a preliminary study, we identified 35 state-of-the-art machine learning defenses. After filtering for reproducibility, we analyzed their performance using Landseer’s unified evaluation process. Our findings reveal gaps in replicability across defense families and provide insights into the challenges and opportunities in integrating multiple defenses, establishing a foundation for improving the reliability of machine learning systems. 1. Introduction The increasing popularity of Machine Learning (ML) systems in critical applications and industries such as cybersecurity ( 8 ; 5 ) , healthcare ( 12 ; 77 ; 45 ) , and finance ( 86 ) has raised growing concerns about the susceptibility of these systems to various types of attacks and trust concerns. Notable incidents include physical evasion of traffic sign detectors using small stickers ( 30 ) , adversarial perturbations that can alter medical image diagnoses ( 97 ) , and data poisoning attacks that lead to backdoors ( 112 ) . These threats target various trustworthy ML dimensions including performance ( 47 ; 100 ) , robustness ( 31 ; 67 ) , privacy ( 84 ; 54 ; 20 ; 107 ; 32 ; 62 ) , and fairness ( 49 ; 78 ; 99 ) , among others. In response, industry and academia have developed defense techniques ( 72 ; 65 ; 23 ) to prevent attacks targeting any of the trustworthy machine learning qualities described above. For example, robustness risks can be addressed by adversarial training ( 110 ; 114 ) , and backdoors can be mitigated via data sanitization ( 11 ) . Each defense is typically designed to be applied at a specific stage of the ML lifecycle and can serve as a building block for constructing secure ML systems. In real-world scenarios, multiple risks or attack vectors are often present simultaneously. As a result, it may be necessary to incorporate various defense strategies to address a wide range of security and reliability requirements. Consequently, we need the ability to combine defenses while keeping their purpose and effectiveness intact. Unfortunately, combining multiple ML defenses is not trivial. Composability of defenses is not well understood, and the aggregate effect on system performance and security guarantees is difficult to predict. This difficulty is due to the diversity of existing techniques. For any one risk class, a variety of defenses exist that are applied at different stages of development or deployment and are configured by a unique set of parameters. As a result, the landscape of possible defenses that could be applied to an ML system grows with combinatorial complexity, making the impact of combining multiple trustworthy machine learning defenses challenging to evaluate. Due to the aforementioned challenges, prior efforts to combine ML defenses and study their interactions and effects on system performance and risks have been mainly limited to pairwise evaluations consisting of two ML defenses at a time ( 33 ; 19 ; 68 ; 92 ; 28 ; 96 ) . Duddu et al. ( 28 ) studied the interactions between defenses and risks, examining the unintended effect of one defense on other risks (not targeted by the defense). However, this work does not offer any insight into how defenses interact with each other. In a follow-up, Duddu et al. ( 29 ) proposed a non-empirical model to predict if combining specific existing defenses will be effective. However, the model is primarily proposed for pairwise combinations, which limits expandability when multiple defenses are considered in the same or different stages of the pipeline. In this paper, we close this gap by developing Landseer , a comprehensive framework to integrate and systematically analyze ML defenses in a typical ML pipeline to discover automated insights on defense interactions. We define and explore the notion of composability for ML defenses, investigating structural requirements for a defense to coexist with other tools in the pipeline, as well as qualitative interference between defenses that affect trust metrics. Landseer ’s architecture consists of an onboarding module, a pipeline explorer, and an interference analysis engine. The onboarding module takes a tool description and corresponding software artifact and generates a reproducibility record together with a structurally composable artifact. The pipeline explorer is the core empirical component that collects data on qualitative composability of defense combinations. The interference analysis engine performs root-cause analysis on collected data to identify sources of interference and provide additional insights about the tools and their interactions. We demonstrate the capabilities of Landseer by surveying 35 defense techniques from literature and examining their composability. Based on this survey, we identify integration barriers and compile guidelines for researchers to avoid composability obstacles. From the examined tools, we onboard 11 and demonstrate that Landseer is able to reproduce existing results on interactions of defenses, and can further surface new insights. Notably, our results show that defense interactions depend on several factors, including the stage at which a defense is applied and the order of defenses within a stage, which have not been examined in prior work. Landseer can help ML practitioners identify best-performing combinations according to a security policy, and enable ML researchers to extract new insights into ML defense interactions to build easy-to-integrate defenses and lower barriers for their adoption. Our contributions can be summarized as follows: • Define and study the notion of composability of various Trustworthy Machine Learning (TML) techniques in a Machine Learning Pipeline. Our taxonomy outlines two composability criteria, structural and qualitative, that describe a defense’s ability to be integrated in an ML deployment. • Design and implement Landseer , a generalizable framework to adopt and analyze the composability of defenses. • Test a corpus of 35 TML techniques against Landseer to replicate previous literature on TML defense interactions, as well as discover new adverse interactions (or interferences) among tools across 700 total combinations. • Identify new insights to aid in tool composability through two case studies to help practitioners and researchers in future tool development. 2. Background 2.1. Machine Learning Operations Machine learning operations (MLOps) describe the activities and lifecycle to create and deploy machine learning models. Example operations include collection and processing of data, the use of this data to train a model, and deployment of the model to users. At a high level, we can categorize the MLOps lifecycle into the following stages: (i) data processing involves collecting a dataset and processing it through cleaning and transformation operations, (ii) training consists of designing and training an ML model