Medical Multimodal Model Stealing Attacks via Adversarial Domain Alignment Yaling Shen, Zhixiong Zhuang, Kun Yuan, Maria-Irina Nicolae, Nassir Navab, Nicolas Padoy, Mario Fritz Abstract Medical MLLM are becoming an instrumental part of healthcare systems, assisting medical personnel with decision making and results analysis. Models for radiology report generation are able to interpret medical imagery, thus reducing the workload of radiologists. As medical data is scarce and protected by privacy regulations, medical MLLM represent valuable intellectual property. However, these assets are potentially vulnerable to model stealing, where attackers aim to replicate their functionality via black-box access. So far, model stealing for the medical domain has focused on classification; however, existing attacks are not effective against MLLM . In this paper, we introduce ADA-Steal ( ADA-Steal ), the first stealing attack against medical MLLM . ADA-Steal relies on natural images, which are public and widely available, as opposed to their medical counterparts. We show that data augmentation with adversarial noise is sufficient to overcome the data distribution gap between natural images and the domain-specific distribution of the victim MLLM . Experiments on the IU X-Ray and MIMIC-CXR radiology datasets demonstrate that ADA-Steal enables attackers to steal the medical MLLM without any access to medical data. LLM large language model MLLM multimodal large language model ADA-Steal Adversarial Domain Alignment 1 Bosch Center for Artificial Intelligence, Germany 2 Technical University of Munich, Germany 3 Munich Center for Machine Learning, Germany 4 Saarland University, Germany 5 University of Strasbourg, France 6 IHU Strasbourg, France 7 CISPA Helmholtz Center for Information Security, Germany {zhixiong.zhuang, irina.nicolae}@bosch.com, {yaling.shen, kun.yuan, nassir.navab}@tum.de, npadoy@unistra.fr, fritz@cispa.de Introduction In recent years, the development of medical MLLM ( MLLM ) has garnered widespread attention due to their potential to revolutionize healthcare. These models could support clinical decision-making ( Seenivasan et al. 2022 ; Chen et al. 2024a ; Yuan et al. 2024b ) , enhance diagnostic accuracy ( Yuan et al. 2024a ; Chen et al. 2024b ) , and promote equitable distribution of medical resources ( Jia et al. 2024 ; Yuan et al. 2023 ; Zhang et al. 2023 ) . One of the most important use cases is radiology report generation, where the medical MLLM takes a radiology image (e.g., chest X-ray) as input and generates a detailed diagnostic report. Since medical data is usually not publicly available and medical expertise is scarce, a well-performing medical MLLM becomes valuable intellectual property (IP). However, these models are potentially vulnerable to model stealing attacks ( Tramèr et al. 2016 ; Orekondy, Schiele, and Fritz 2019a ) , which replicate the functionality of a machine learning model through black-box access. This threat is particularly significant in the medical field because a duplicated model could conflict with IP and privacy regulations, and could also facilitate further attacks. By designing a payload on the copied model, malicious actors can then attack the original model, e.g., with a transfer jailbreak attack ( Huang et al. 2024 ) to compel a medical MLLM to output fraudulent or fabricated medical information. Figure 1: ADA-Steal trains an attacker MLLM to replicate a victim MLLM for radiology report generation from natural images by first enriching the reports and then aligning the attacker distribution to the medical domain. Model theft typically involves two steps: (i) creating a transfer dataset by querying the victim model with public or synthetic data ( Orekondy, Schiele, and Fritz 2019a ; Truong et al. 2021 ) to obtain pseudo-labels; and (ii) training the attacker model using these pseudo-labels as ground truth. Nevertheless, current model stealing methods present significant limitations when applied to medical MLLM . First, due to patient privacy concerns, there are few public medical datasets available for querying, leading to a limited and homogeneous transfer dataset. Second, existing methods primarily target image classification, where every class prediction from the victim model is useful. In contrast, medical text generation involves a much larger output space (i.e., vocabulary), with only a subset of tokens being medically relevant and valuable for training a medical model. While Knockoff Nets ( Orekondy, Schiele, and Fritz 2019a ) showed that a diabetic image classification model can be stolen using the non-medical ImageNet dataset, we demonstrate that such images are not suitable for stealing medical MLLM . When using non-medical images, medical MLLM produce simplistic, repetitive reports, with few containing relevant disease. This underscores the importance of aligning the query distribution with the victim model’s data distribution for successful model stealing, as highlighted in previous works ( Orekondy, Schiele, and Fritz 2019a ; Truong et al. 2021 ; Zhuang, Nicolae, and Fritz 2024 ) . To address these problems, we introduce ADA-Steal , the first data-free method for stealing medical MLLM focused on radiology report generation. This method addresses the issues by: (i) using an open-source oracle model to diversify reports without requiring prior medical knowledge, and (ii) integrating the new reports into the query images via targeted adversarial attacks. This process enables non-medical query data to produce more diverse and medically relevant reports, effectively aligning the non-medical domain data with the medical domain data. Contributions. (i) We are the first to investigate the feasibility of model stealing attacks against medical MLLM and to identify the associated challenges. (ii) We propose ADA-Steal ( ADA-Steal ), the first model theft method to replicate the functionality of medical MLLM without requiring expert knowledge or access to medical domain data. (iii) We validate our ADA-Steal method on the IU X-Ray and MIMIC-CXR test datasets, showing that it approaches the victim model’s performance in both natural language generation metrics and clinical efficacy metrics, even when using the non-medical CIFAR100 dataset. (iv) We conduct ablation studies to analyze the effects of different components of our method, showing that ADA-Steal can increase the diversity of the attacker dataset by medical report enrichment and domain alignment. Related Work Knowledge distillation. Knowledge distillation ( Buciluǎ, Caruana, and Niculescu-Mizil 2006 ; Hinton, Vinyals, and Dean 2015 ) helps transfer the knowledge from a complex and larger “teacher” model to a compact and simpler “student” model, which is similar to our victim-attacker design. However, unlike knowledge distillation, where the student model has the same data distribution as the teacher model’s training data, in our problem formulation, the attacker has no prior knowledge of the victim’s black-box model, e.g., unknown architecture, data distributions or training parameters. Although data-free knowledge distillation ( Fang et al. 2019 ; Micaelli and Storkey 2019 ; Choi et al. 2020 ) further assumes the absence of the teacher model’s training data, its requirement of white-box access to the teacher model for backpropagation is a major difference to our setup. Model stealing. Model stealing, or model theft, typically has one of two objectives: exact replication of the model or its components, or functionality replication, where the attacker aims to mimic the model’s behavior. The first type focuses on extracting the model’s hyperparameters ( Wang and Gong 2018 ) , architecture ( Oh et al. 2018 ) , or learned parameters ( Tramèr et al. 2016 ) . The second type ( Orekondy, Schiele, and Fritz 2019a ; Truong et al. 2021 ; Zhuang, Nicolae, and Fritz 2024 ) , involves training a model that mimics the victim’s performance without prior knowledge of its training data or architecture. In this work, we present the first functionality model stealing attack against MLLM for radiology report generation. While previous methods like Knockoff Nets ( Orekondy, Schiele, and Fritz 2019a ) replicate medical image classifiers using natural images, they deal with a much smaller output space compared to text generation. Bert-Thieves ( Krishna et al. 2020 ) , another related approach, targets language models, but does not handle images and benefits from publicly available text data that shares a similar distribution with the victim model. In contrast, medical data is hard to obtain, and only a specific subset of the vocabulary is relevant in this context, making it more challenging for the attacker, who may lack prior knowledge in the medical domain. Security of MLLM . With the ability to understand and reason about different data types, MLLM are vulnerable to evasion attacks targeting each data modality, such as malicious image and text constructs ( Schlarmann and Hein 2023 ; Liu et al. 2024 ) . In contrast to these attacks that are designed to induce erroneous or disallowed responses, the model stealing attack we present aims to mimic the functionality of the victim medical MLLM for radiology report generation, using only black-box access to the victim model. Figure 2: The overview of our proposed approach with three iterative phases: (I) attacker model training, (II) medical report enrichment (in gray dash box), and (III) domain alignment (in green dash box). Threat Model In this section, we formalize the threat model for stealing black-box medical MLLM in radiology report generation. First, we introduce preliminary concepts and notations. We then formalize the victim model as well as the attacker’s objective and knowledge based on the real-world setup. Notations. We model MLLM predicting the probability of the next token y_{l} given the preceding language tokens y_{