What Lurks Within? Concept Auditing for Shared Diffusion Models at Scale

Xiaoyong Yuan, Xiaolong Ma, Linke Guo, Lan Zhang · 2025

Read What Lurks Within? Concept Auditing for Shared Diffusion Models at Scale by Xiaoyong Yuan, Xiaolong Ma, Linke Guo, Lan Zhang (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2504-14815.txt.

Introduction

Xiaoyong Yuan, Xiaolong Ma, Linke Guo, and Lan Zhang build an auditor for shared image models in a 2025 paper. Fine-tuned diffusion models now circulate on open platforms, and some may secretly generate sensitive or stolen content. No practical tool checked them before release.

Older audits craft prompts and judge the pictures that come out, which is slow and unreliable. Their method, PAIA, inspects the model itself rather than its outputs. It asks whether the model has learned a target concept, with no prompts or images needed. On 320 controlled models and 771 real community models, accuracy passed 90 percent while audit time fell by a factor of 18 to 40.

Catalog record

CREATORXiaoyong Yuan, Xiaolong Ma, Linke Guo, Lan Zhang
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE73 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2504-14815

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

What Lurks Within? Concept Auditing for Shared Diffusion Models at Scale Xiaoyong Yuan, Xiaolong Ma, Linke Guo, Lan Zhang Abstract . Diffusion models (DMs) have revolutionized text-to-image generation, enabling the creation of highly realistic and customized images from text prompts. With the rise of parameter-efficient fine-tuning (PEFT) techniques like LoRA, users can now customize powerful pre-trained models using minimal computational resources. However, the widespread sharing of fine-tuned DMs on open platforms raises growing ethical and legal concerns, as these models may inadvertently or deliberately generate sensitive or unauthorized content, such as copyrighted material, private individuals, or harmful content. Despite the increasing regulatory attention on generative AI, there are currently no practical tools for systematically auditing these models before deployment. In this paper, we address the problem of concept auditing: determining whether a fine-tuned DM has learned to generate a specific target concept. Existing approaches typically rely on prompt-based input crafting and output-based image classification but suffer from critical limitations, including prompt uncertainty, concept drift, and poor scalability. To overcome these challenges, we introduce Prompt-Agnostic Image-Free Auditing (PAIA), a novel, model-centric concept auditing framework. By treating the DM as the object of inspection, PAIA enables direct analysis of internal model behavior, bypassing the need for optimized prompts or generated images. It integrates two key components: a prompt-agnostic strategy that mitigates prompt sensitivity by analyzing model behavior during late-stage denoising, and an image-free detection method based on conditional calibrated error, which compares the internal dynamics of a fine-tuned model against its base version. We evaluate PAIA on 320 controlled models trained with curated concept datasets and 690 real-world community models sourced from a public DM sharing platform, covering a wide range of concepts including celebrities, cartoon characters, videogame entities, and movie references. Evaluation results show that PAIA achieves over 90 90 % detection accuracy while reducing auditing time by 18 – 40 × 18\textendash 40\times compared to existing baselines. To our knowledge, PAIA is the first scalable and practical solution for pre-deployment concept auditing of diffusion models, providing a practical foundation for safer and more transparent diffusion model sharing. 1. Introduction Diffusion models (DMs) have revolutionized text-to-image (T2I) generation, enabling the synthesis of highly realistic and semantically rich images from natural language prompts ( croitoru2023diffusion ; ruiz2023dreambooth ; zhang2023adding ) . Through iteratively refining noise into coherent visual content, DMs have surpassed traditional generative methods such as GANs ( goodfellow2014generative ) and VAEs ( kingma2013auto ) in both visual fidelity and flexibility. This leap in generative quality has been further accelerated by parameter-efficient fine-tuning (PEFT) methods like Low-Rank Adaptation (LoRA) ( hu2021lora ) , which allow users to customize large pre-trained models, such as Stable Diffusion ( rombach2021highresolution ) , for specific concepts or styles using limited compute and memory resources. This shift has catalyzed a thriving ecosystem of community-driven model customization. Users can now fine-tune and distribute their models with minimal technical expertise, aided by user-friendly toolkits ( automatic1111 ; comfy_website ) and supported by sharing platforms such as CivitAI ( civitai_website ) , HuggingFace ( huggingface_website ) , and SeaArt ( seaart_website ) . These platforms host tens of thousands of customized DMs, covering a broad spectrum of visual concepts, artistic styles, and application domains. While this democratization has opened new frontiers in creativity and accessibility, it has also introduced significant risks. Fine-tuned DMs can be misused or repurposed to generate inappropriate, offensive, or legally problematic content. Studies have documented instances where models replicate copyrighted characters ( zhang2023copyright ; shan2023glaze ; zhu2024watermark ) , impersonate real individuals via deepfakes ( hao2024doesn ) , or produce not safe for work (NSFW) content ( zhang2024generate ; Yang2023SneakyPromptJT ; Chin2023Prompting4DebuggingRT ) . Despite growing concerns, technical oversight remains minimal. Current auditing practices on public model hubs rely heavily on user-supplied tags with little automation or verification. For example, CivitAI ( civitai_website ) , one of the most widely used platforms, relies primarily on user-provided metadata to flag models, such as ‘‘mature content’’ or depictions of ‘‘real people,’’ which are loosely defined, inconsistently applied, and easily circumvented. The absence of standardized review procedures or systematic validation has already led to legal consequences. The community reports and discussions suggest that CivitAI has received takedown requests related to the unauthorized use of copyrighted or personal content due to the models trained on proprietary content 1 1 1 CivitAI Facilitates the Use of Stolen Intellectual Property, https://luddite.pro/civitai-facilitates-use-stolen-intellectual-property 2 2 2 Square Enix may have filed a DMCA takedown notice with CivitAI, https://www.reddit.com/r/StableDiffusion/comments/11zubsj/it_appears_that_someone_acting_on_behalf_of . Figure 1 . Overview of PAIA: A Model-Centric Concept Auditing for Fine-Tuned Diffusion Models. Community users increasingly fine-tune and share DMs on platforms such as CivitAI ( civitai_website ) , HuggingFace ( huggingface_website ) , and SeaArt ( seaart_website ) , introducing risks of unauthorized or sensitive content generation. Existing approaches rely on observable behaviors—prompts and outputs—that are inherently unstable, easily manipulated, and costly to evaluate. We conduct a systematic study and propose a model-centric framework PAIA that bypasses these limitations by auditing internal model behavior directly. To mitigate such risks, some efforts have introduced proactive defenses, such as built-in safety filters designed to block harmful content by detecting unsafe prompts or screening generated images ( Rando2022RedTeamingTS ; Yang2023SneakyPromptER ) . However, their effectiveness is limited. These filters are often implemented as optional add-ons and can be easily disabled after model download, leaving the core DM unrestricted. More importantly, both these reactive and proactive defenses suffer from a shared limitation: they rely on observable behaviors, prompts and outputs, that are unstable, easy to manipulate, and difficult to verify at scale. These limitations call for a deeper shift in auditing methodology: rather than focusing on what a model produces, we ask a more direct and scalable question - what lurks within ? In this work, we study the problem of concept auditing : given a specific concept or a curated set of concepts, such as copyrighted characters, company logos, or celebrity identities, can a shared, fine-tuned diffusion model generate content that embodies those concepts? Rather than attempting to flag all forms of inappropriate content, we focus on concept-specific auditing at scale. This reflects real-world enforcement needs, where moderation and legal action often target clearly defined entities tied to intellectual property. To keep the scope practical and aligned with real-world use cases, we focus on concepts that are visually distinctive and often subject to content moderation or IP protection. These include individual celebrity identities ( e.g . , “Taylor Swift”), characters from well-known cartoons ( e.g . , “Muppets”), and entities from games and movies. Our framework is designed to accommodate this diversity through an example-based approach: a concept is considered present if the model can generate recognizable outputs aligned with a small set of reference examples of the concept. This flexible definition allows us to support a broad range of concept granularities without relying on a rigid taxonomy. A detailed discussion of how we define and structure concept scopes for auditing can be found in Section 8.1 . A common approach to this problem is to audit models through their outputs: by crafting prompts that might trigger the concept of interest, generating images from the model, and using a classifier to determine whether the concept

Read the complete text