When Secure Isn’t: Assessing the Security of Machine Learning Model Sharing Conference: arXiv; preprint; 2025

Gabriele Digregorio, Marco Di Gennaro, Stefano Zanero, Stefano Longari, Michele Carminati · 2025

Read When Secure Isn’t: Assessing the Security of Machine Learning Model Sharing Conference: arXiv; preprint; 2025 by Gabriele Digregorio, Marco Di Gennaro, Stefano Zanero, Stefano Longari, Michele Carminati (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2509-06703.txt.

Introduction

Loading a shared machine learning model can quietly run hostile code, warn Gabriele Digregorio and four co-authors in a 2025 security paper. Model hubs and frameworks made sharing easy, and safety claims are common. The authors test whether those claims hold up in practice.

Their evaluation finds most frameworks and hubs handle risks only partly, often pushing responsibility onto users. Worse, checking products that advertise security settings turned up several previously unknown vulnerabilities allowing arbitrary code execution. The file format used for sharing, they argue, cannot by itself guarantee safety. A user survey adds a twist: safety marketing leads users to trust settings that the tests showed to be weak. The paper ends with suggestions for hardening the model-sharing ecosystem, aimed at both platform builders and practitioners.

Catalog record

CREATORGabriele Digregorio, Marco Di Gennaro, Stefano Zanero, Stefano Longari, Michele Carminati
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE86 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2509-06703

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

When Secure Isn’t: Assessing the Security of Machine Learning Model Sharing Conference: arXiv; preprint; 2025 Gabriele Digregorio, Marco Di Gennaro, Stefano Zanero, Stefano Longari, Michele Carminati Abstract . The rise of model-sharing through frameworks and dedicated hubs makes ML significantly more accessible. Despite their benefits, these tools expose users to underexplored security risks, while security awareness remains limited among both practitioners and developers. To enable a more security-conscious culture in ML model sharing, in this paper we evaluate the security posture of frameworks and hubs, assess whether security-oriented mechanisms offer real protection, and survey how users perceive the security narratives surrounding model sharing. Our evaluation shows that most frameworks and hubs address security risks partially at best, often by shifting responsibility to the user. More concerningly, our analysis of frameworks advertising security-oriented settings and complete model sharing uncovered six 0-day vulnerabilities enabling arbitrary code execution. Through this analysis, we debunk the misconceptions that the model-sharing problem is largely solved and that its security can be guaranteed by the file format used for sharing. As expected, our survey shows that the surrounding security narrative leads users to consider security-oriented settings as trustworthy, despite the weaknesses shown in this work. From this, we derive takeaways and suggestions to strengthen the security of model-sharing ecosystems. PoC proof-of-concept ML Machine Learning AI Artificial Intelligence DL Deep Learning 1. Introduction In recent years, the adoption of ML ( ML ) has grown rapidly, with advanced tools once limited to experts now accessible to a broad audience. Among other factors, this trend is driven by the rise of platforms for sharing pre-trained models ( HuggingFaceHub ; KaggleModels ; kerashub2024 ; TensorFlowHub ; PyTorchHub ) . Such models are often advertised as ready-to-use, lowering entry barriers and enabling practitioners with different levels of expertise to incorporate ML into their systems and workflows. In many ways, this trend reflects the evolution of traditional software development, where open-source repositories enable the reuse and adaptation of algorithms, code, and libraries. Today, models are shared and reused, enabling a collaborative ecosystem that is reshaping how ML is practiced. However, unlike the well-studied risks of code sharing and software supply chains ( duan2021towards ; ohm2020backstabber ; ladisa2023sok ; jiang2022empirical ; casey2024large ; shenao2025sok ; meiklejohn2025machine ) , the security implications of ML model sharing remain largely underexplored. Prior work on malicious code injection in models ( hua2024malmodel ; wang2021evilmodel ; wang2022evilmodel2 ; zhao2024models ) and on hub security gaps ( jiang2023exploring ; jiang2023empirical ; zhao2024models ; Jones2024What ) is fragmented, resulting in limited awareness among users, framework developers, and sharing hubs. The immaturity of the model sharing security field is further illustrated by Zhu et al. ( zhu2025my ) , who show how TensorFlow APIs can be abused for file and network access at inference time. The growing relevance of the problem is also reflected in the recent DEF CON 33 talk by Parzian ( ParzianDefcon33 ) , which highlighted persistent risks in model sharing, such as insecure pickle deserialization. Our work addresses the fragmented and immature state of security research on model sharing, aiming to raise awareness among everyday ML users of its hidden risks, debunk common misconceptions, and encourage the security community to study these threats with the same rigor applied to open-source software at large. We consider a threat model where attackers craft malicious model artifacts to compromise a victim’s system at load time, with arbitrary code execution as the goal. The paper, in its structure, is driven by the research questions defined below. RQ1. What is the security posture of the model sharing mechanisms adopted by popular ML frameworks and hubs? RQ2. Are approaches claiming security guarantees, while offering full model object sharing, actually secure? RQ3. Is the user’s perception of the security posture consistent with reality, or does the security narrative affect their understanding of the sharing-associated risks? To answer RQ1, we analyze the security of the most popular ML frameworks and hubs, focusing on their sharing approaches and characteristics. We observe that while some of them do not address security at all, others provide security-oriented mechanisms based on shifting responsibility to other parts of the sharing workflow or by restricting the expressiveness of the model representation. Some go further, with certain frameworks providing complete model-sharing capabilities while explicitly promoting themselves as security-oriented, and some hubs emphasizing the same security focus through safeguards such as content scanning. This narrative is reinforced through documentation and naming choices (e.g., Keras’s “safe mode” ( KerasSavingLoading ) ), and is often justified at the framework level by the use of data-based formats, assumed to reduce the risk of arbitrary code execution. This last category led to our RQ2, which motivates a vulnerability assessment of the configurations offered by these sharing frameworks and hubs. Our analysis uncovered six 0-day vulnerabilities (i.e., previously undisclosed), each enabling arbitrary code execution. These findings debunk the widespread assumption that data-based formats (e.g., JSON) are inherently secure when used to share full model objects. Our answer to RQ2 reveals a critical gap between the security narrative and the actual implementation. The lack of public scrutiny and awareness is further evidenced by the low number of pre-existing Common Vulnerabilities and Exposures (referred to as CVEs ) for these mechanisms (i.e., before our work). For instance, our findings represent the first officially recognized CVEs targeting Keras’s “safe mode” ( CVE-2025-1550 ; CVE-2025-8747 ) . This understanding led to RQ3. To answer it, we conducted a survey targeting ML practitioners to understand how frameworks and hubs’ security narratives influence user perception. Results worryingly indicate that design terminology and documentation significantly shape users’ sense of security. We conclude our paper with takeaways and suggestions for users and developers, synthesized from the observations and insights obtained through our research questions. Contributions. This paper makes the following contributions: • We systematize and analyze the security of ML model-sharing mechanisms, covering both framework-level and hub-level perspectives. • We assess methods that claim security guarantees while offering full model object sharing, identifying several CVEs, and challenging the common assumption that data-based formats are inherently secure. • We reveal a disconnect between the perceived security narrative (and the resulting community belief) and the reality through a survey on model sharing. • We provide takeaways and suggestions for the community, to promote a more security-aware culture in model sharing. Open Science. We have made all artifacts necessary to evaluate and reproduce our findings available on GitHub. These include the PoC ( PoC ) exploits, all model artifacts used in our experiments (e.g., Keras .keras and Skops .skops models), and their generation scripts. In addition, we provide the raw survey responses in CSV format, along with the distributed questionnaire, survey collection utilities, and any other analysis scripts developed for this work. All files are available at the following repositories: https://github.com/necst/security-model-sharing . Table 1. Framework-level sharing: documented characteristics and security considerations. Framework Setting File Format Self-contained Security model (security posture + commentary) Data-based Code-based Weights Model \star SO Comments Keras ( chollet2015keras ; KerasSavingLoading ) safe_mode=True JSON - Blocks untrusted Lambda deserialization; custom objects must be registered safe_mode=False JSON pickle Can restore models with unrestricted code; arbitrary code execution is possible by design Legacy HDF5 pickle Legacy format; can

Read the complete text