Systems-Theoretic and Data-Driven Security Analysis in ML-enabled Medical Devices
Gargi Mitra, Mohammadreza Hallajiyan, Inji Kim, Athish Pranav Dharmalingam, Mohammed Elnawawy, Shahrear Iqbal, Karthik Pattabiraman, Homa Alemzadeh · 2025
Read Systems-Theoretic and Data-Driven Security Analysis in ML-enabled Medical Devices by Gargi Mitra, Mohammadreza Hallajiyan, Inji Kim, Athish Pranav Dharmalingam, Mohammed Elnawawy, Shahrear Iqbal, Karthik Pattabiraman, Homa Alemzadeh (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2506-15028.txt.
Introduction
Gargi Mitra and seven co-authors published a paper in 2025 on the security of medical devices that learn. Smart monitors and diagnostic aids can be attacked through their models, their connections, and the gadgets they link to, and a fooled model can endanger a patient directly. The authors argue that risk checks must happen before such devices reach the market, not after harm.
They begin by mining recalls, adverse event reports, and known flaw lists to map real threats. On that base they present a toolkit mixing system-level hazard analysis with data-driven probing, meant to help analysts run full pre-market assessments. The goal is to make security a design habit for makers of learning devices, in the same way sterility already is.
Catalog record
| CREATOR | Gargi Mitra, Mohammadreza Hallajiyan, Inji Kim, Athish Pranav Dharmalingam, Mohammed Elnawawy, Shahrear Iqbal, Karthik Pattabiraman, Homa Alemzadeh |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 64 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2506-15028
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Systems-Theoretic and Data-Driven Security Analysis in ML-enabled Medical Devices Gargi Mitra, Mohammadreza Hallajiyan, Inji Kim, Athish Pranav Dharmalingam, Mohammed Elnawawy, Shahrear Iqbal, Karthik Pattabiraman, Homa Alemzadeh Abstract The integration of AI/ML into medical devices is rapidly transforming healthcare by enhancing diagnostic and treatment facilities. However, this advancement also introduces serious cybersecurity risks due to the use of complex and often opaque models, extensive interconnectivity, interoperability with third-party peripheral devices, Internet connectivity, and vulnerabilities in the underlying technologies. These factors contribute to a broad attack surface and make threat prevention, detection, and mitigation challenging. Given the highly safety-critical nature of these devices, a cyberattack on these devices can cause the ML models to mispredict, thereby posing significant safety risks to patients. Therefore, ensuring the security of these devices from the time of design is essential. This paper underscores the urgency of addressing the cybersecurity challenges in ML-enabled medical devices at the pre-market phase. We begin by analyzing publicly available data on device recalls and adverse events, and known vulnerabilities, to understand the threat landscape of AI/ML-enabled medical devices and their repercussions on patient safety. Building on this analysis, we introduce a suite of tools and techniques designed by us to assist security analysts in conducting comprehensive premarket risk assessments. Our work aims to empower manufacturers to embed cybersecurity as a core design principle in AI/ML-enabled medical devices, thereby making them safe for patients. 1 Introduction Machine Learning (ML)-driven applications are becoming increasingly popular in the medical field. ML-enabled medical devices (software or software-driven hardware) assist physicians in critical activities such as remote patient monitoring, controlling surgical equipment, automatic drug administration, and preliminary/advanced disease diagnosis [ 110 ] . These tasks require high accuracy and reliability, and the loss of either of these can endanger patient safety. However, the use of ML in interconnected medical devices has expanded the threat surface of medical systems [ 22 , 4 , 68 , 80 , 64 , 20 , 81 , 124 , 71 , 24 , 133 , 89 , 78 , 59 , 77 ] making them more vulnerable to cyberattacks. If an adversary compromises such a device, it can force the ML engine to make incorrect predictions or decisions, which can have catastrophic consequences, such as wrong diagnoses and treatments, leading to severe health complications or even the death of the patient. Detecting and mitigating cyberattacks in ML-based medical applications is significantly more challenging than in traditional systems for two reasons. First, these applications rely on large datasets and often employ complex, unexplainable models, making their behavior difficult to interpret even for developers. Second, they are highly interconnected with third-party devices that collect patient data for real-time predictions, which are subsequently transmitted to downstream systems or patients and physicians for clinical decision-making and treatment. This high degree of connectivity increases the attack surface, while the complexity of ML models complicates attack detection. Adversaries can exploit the vulnerabilities in the ML models and interface devices to poison training data [ 88 ] , inject erroneous inputs during inference [ 49 ] , or modify model parameters through compromised configuration files [ 126 ] . We are particularly interested in inference-time false data injection attacks, which are the easiest to execute and most difficult to detect. A recent study on the FDA adverse event reports involving ML-enabled medical devices indicated that over 80% of the reported events were related to data acquisition problems, leading to no data or erroneous data capture [ 76 ] . Several studies have also highlighted the vulnerability of data acquisition systems to adversarial examples [ 50 , 57 , 20 ] . The safety-critical nature of ML-enabled medical devices makes it crucial to identify and address these security vulnerabilities before deployment. Figure 1 : Growing number of AI/ML-enabled medical devices and the rise of security-awareness among device manufacturers (Data as of April 2025) In this paper, we focus on pre-market security risk assessment, which is the process of identifying, assessing the severity of, and mitigating potential security risks in a given medical device before it is approved for market release. This process is crucial for ensuring patient safety, regulatory compliance, and cyber resilience, as well as reducing post-deployment threat mitigation costs. We inspected the publicly available device summaries [ 118 ] the manufacturers submitted to the FDA for pre-market approval to determine whether manufacturers conducted security risk assessments for ML-enabled devices. Our investigation reveals that for over 65% of these devices, the manufacturers either do not provide any information about the assessment method in their documentation, or employ inadequate assessment methods (see Figure 1 ). In fact, until 2014, no device summary mentioned any security risk assessment. Among the remaining devices, a few use proprietary mechanisms that make it challenging to assess the adequacy of their approach, while others utilize existing risk assessment techniques. These techniques, as we discuss in the subsequent sections, are insufficient for securing interconnected ML-enabled medical systems. However, on a positive note, there is a growing security awareness among manufacturers, reflected in the increasing mention of security risk assessments in recent pre-market summaries. Security practitioners and researchers have made significant efforts in assessing and ensuring the safety and security of medical devices by developing advanced methods for qualitative and quantitative risk assessment (e.g., fault tree analysis (FTA), failure mode and effect analysis (FMEA)) and formal assurance case reports [ 9 , 62 , 61 ] ) and security analysis [ 14 ] , model-based design and verification [ 16 , 95 , 7 ] , closed-loop validation [ 63 ] , encryption, and authentication [ 65 ] . However, less attention has been paid to the end-to-end system security of ML-enabled medical devices by considering the interactions of the ML-enabled device with other interconnected system components. Current security assessment methods primarily focus on algorithm, hardware, software, and firmware vulnerabilities, but they often overlook the inherent vulnerabilities of the ML models used in medical devices, how they can be exploited by first exploiting vulnerabilities in interconnected devices, and the potential impact of the ML mispredictions on patient safety . To bridge this gap, it is imperative to perform a holistic system-theoretic analysis of ML-enabled medical systems. In this paper, we first present our experience with developing tools and techniques to automate the extraction of large-scale data on real-world security vulnerabilities and safety incidents for ML-enabled medical devices from public data and knowledge sources. Further, we devise techniques that use this data to enable security practitioners to perform system-theoretic analysis to identify potential threats, new attack paths, and their safety impacts. This will help medical device manufacturers anticipate post-deployment security risks early at design time, assess the severity of the risks, and implement risk prevention and mitigation strategies. For instance, a company developing an ML-enabled device that integrates with third-party commodity off-the-shelf cameras can use our techniques to identify known security vulnerabilities in compatible camera models, evaluate the likelihood of their exploitation, and assess potential risks to patient safety based on previously reported failures and adverse events of both ML-enabled and non-ML-enabled devices with the same functionality. Based on these insights, the company can either implement appropriate security measures and safety mechanisms or provide guidance to