Architectural Backdoors for Within-Batch Data Stealing and Model Inference Manipulation
Nicolas Küchler, Ivan Petrov, Conrad Grobler, Ilia Shumailov · 2025
Read Architectural Backdoors for Within-Batch Data Stealing and Model Inference Manipulation by Nicolas Küchler, Ivan Petrov, Conrad Grobler, Ilia Shumailov (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2505-18323.txt.
Introduction
Nicolas Küchler, Ivan Petrov, Conrad Grobler, and Ilia Shumailov demonstrated in 2025 a darker species of neural network backdoor, one that lives in the architecture and feeds on batching. Classic backdoors flip a model's answer on a trigger. Architectural versions, planted in the model's design, had recently raised the stakes.
Modern serving groups many users' requests into one batch for speed. The team shows a backdoor can be built to exploit exactly that. Information leaks between requests sharing a batch, and an attacker can steer answers sent to other users in it. The theft is not of one answer but of neighbors' data at scale. Readers, above all model deployers, get a warning with a mechanism: the batch, that innocent efficiency, is an attack surface.
Catalog record
| CREATOR | Nicolas Küchler, Ivan Petrov, Conrad Grobler, Ilia Shumailov |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 47 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2505-18323
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Architectural Backdoors for Within-Batch Data Stealing and Model Inference Manipulation Nicolas Küchler, Ivan Petrov, Conrad Grobler, Ilia Shumailov Abstract For nearly a decade the academic community has investigated backdoors in neural networks, primarily focusing on classification tasks where adversaries manipulate the model prediction. While demonstrably malicious, the immediate real-world impact of such prediction-altering attacks has remained unclear. In this paper we introduce a novel and significantly more potent class of backdoors that builds upon recent advancements in architectural backdoors. We demonstrate how these backdoors can be specifically engineered to exploit batched inference, a common technique for hardware utilization, enabling large-scale user data manipulation and theft. By targeting the batching process, these architectural backdoors facilitate information leakage between concurrent user requests and allow attackers to fully control model responses directed at other users within the same batch. In other words, an attacker who can change the model architecture can set and steal model inputs and outputs of other users within the same batch. We show that such attacks are not only feasible but also alarmingly effective, can be readily injected into prevalent model architectures, (e.g. Transformers), and represent a truly malicious threat to user privacy and system integrity. Critically, to counteract this new class of vulnerabilities, we propose a deterministic mitigation strategy that provides formal guarantees against this new attack vector, unlike prior work that relied on Large Language Models to find the backdoors. Our mitigation strategy employs a novel Information Flow Control mechanism that analyzes the model graph and proves non-interference between different user inputs within the same batch. Using our mitigation strategy we perform a large scale analysis of models hosted through Hugging Face and find over 200 models that introduce (unintended) information leakage between batch entries due to the use of dynamic quantization. \paperurl \reportnumber 1 Introduction Machine learning models are increasingly deployed using batched inference to optimize hardware utilization. While efficient, this common practice potentially introduces significant security vulnerabilities by processing multiple users’ data concurrently within shared environments, thereby compromising the isolation between them. In fact, recent academic work highlights that this cross-batch isolation is already unintentionally broken in some widely used Mixture-of-Experts architectures ( Hayes et al. 2024 ; Yona et al. 2024 ) , and it can lead to (inefficient) user data leakage. In this paper we present a novel and potent class of vulnerabilities building upon recent advancements in architectural backdoors ( Bober-Irizar et al. 2023 ; Langford et al. 2025 ) . We demonstrate how these backdoors can be specifically engineered to exploit the batched inference process itself, enabling large-scale, cross-user data theft and manipulation. Our attack effectively generalizes and weaponizes inefficient leakage reported by Hayes et al. 2024 ; Yona et al. 2024 to work in arbitrary models (e.g. Transformer architectures), requiring only minor edits to the model architecture. Unlike prior research on parameter-based backdoors ( Gu et al. 2017 ) , which primarily focused on manipulating model predictions for classification tasks, the architectural backdoors presented here target the batching mechanism to facilitate information leakage between concurrent user requests. This allows attackers not only to potentially steal sensitive input data from other users within the same batch but also to gain control over the model responses directed towards them. We demonstrate that such attacks are not merely theoretical but are feasible, alarmingly effective, and can be injected into prevalent ML models, e.g. Large Language Models (LLMs) with Transformer architecture, representing a truly malicious threat to user privacy and system integrity in modern ML deployments. Figure 1 : Architectural backdoor that exploits batched inference. The left side depicts a model processing batched inputs and producing batched outputs, with a backdoor (a "Trigger Detector" for detecting the attacker’s trigger, and "Signal Integration" for deviating from normal operation) integrated into the architecture. The right side details three attacks when attacker’s and victim’s inputs are processed in the same batch: stealing victim output : the attacker’s output is manipulated to include the victim’s output; changing victim output : the attacker alters the output intended for the victim; and steering victim output : the attacker influences the victim’s output, guiding it towards a specific outcome. Critically, addressing this new class of vulnerabilities requires robust countermeasures. In response, we propose a deterministic mitigation strategy designed to provide formal guarantees against this specific attack vector. This contrasts with prior work often relying on heuristic or probabilistic detection methods, e.g. using LLMs ( Langford et al. 2025 ) . Our work highlights the urgent need to re-evaluate the security assumptions underlying batched inference and develop provably secure mechanisms for shared ML systems. Overall, we make the following contributions: • We present a novel class of architectural backdoors that enable compromise of batched inference. Our backdoors enable extremely efficient leakage; • We develop a robust and provable method to defend against our backdoors; • We run a large scale analysis of models hosted at Hugging Face and find over 200 models that introduce (unintended) information leakage between batch entries due to dynamic quantization. 2 Threat Model This section defines the threat model for architectural backdoors embedded within a model graph that are specifically designed to exploit batched inference for cross-user data compromise. We consider a standard deployment where a model server (cloud or on-premise) loads a model graph to serve user requests. Users submit sensitive inputs (e.g., prompts, images) and expect their data and the resulting outputs to remain isolated from others sharing the infrastructure. Assumptions. We assume the server infrastructure (including batching and routing) operates correctly, and that the compilation toolchain accurately translates the model graph into executable computations without introducing modifications. A user’s position within a batch is either input-data-independent or randomized prior to inference, thus offering no exploitable signal. While the model graph itself is stateless during inference, it can manage state (e.g., KV-Cache) via its inputs and outputs. Crucially, the handling of this state (outside the model graph) is assumed to introduce no information leakage. Input and output tensor shapes are considered unprotected and are assumed not to encode secret information, although padding can be employed to obscure these details if necessary. Adversary. The adversary is an entity capable of influencing or directly modifying the machine learning model’s architecture before it is deployed for inference. This setting is realistic and could be either a malicious actor in the model supply chain (e.g., during pre-training, fine-tuning, or distribution); an insider with access to model development or deployment pipelines; or potentially the hosting provider itself if they are untrusted and can manipulate the models they serve. The adversary’s goal is not merely to manipulate the model’s prediction for their own input (as in traditional backdoor attacks) but to break isolation between users within the same inference batch. The adversary can pursue multiple goals here: • Confidentiality Violation for Data Theft : To steal sensitive input data submitted by other users who are processed in the same batch as the adversary’s request. • Integrity Violation for Response Manipulation : To manipulate or fully control the model’s output generated for other users within the same batch. The overall attack happens in multiple stages: 1) Adversary manipulates the model by injecting architectural backdoors into the model, which are designed to be dormant