Customize Multi-modal RAI Guardrails with Precedent-based predictions
Cheng-Fu Yang, Thanh Tran, Christos Christodoulopoulos, Weitong Ruan, Rahul Gupta, Kai-Wei Chang · 2025
Read Customize Multi-modal RAI Guardrails with Precedent-based predictions by Cheng-Fu Yang, Thanh Tran, Christos Christodoulopoulos, Weitong Ruan, Rahul Gupta, Kai-Wei Chang (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2507-20503.txt.
Introduction
Cheng-Fu Yang and co-authors proposed in 2025 a flexible guardrail that filters images by rules each user sets, such as bans on hateful or explicit content. Users often need custom rules and can give few examples. Fixed-rule training does not transfer well, while training-free methods cannot fit all rules in their input window.
Their method judges a new image by precedents: past cases near it, with their reasoning kept. A critique-and-revise loop gathers high-quality precedents, and two strategies use them for the verdict. Tests beat past methods in both few-example and full-data settings. The method also adapted better to brand-new rules, the authors report.
Catalog record
| CREATOR | Cheng-Fu Yang, Thanh Tran, Christos Christodoulopoulos, Weitong Ruan, Rahul Gupta, Kai-Wei Chang |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 54 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2507-20503
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Customize Multi-modal RAI Guardrails with Precedent-based predictions Cheng-Fu Yang, Thanh Tran, Christos Christodoulopoulos, Weitong Ruan, Rahul Gupta, Kai-Wei Chang Abstract A multi-modal guardrail must effectively filter image content based on user-defined policies, identifying material that may be hateful, reinforce harmful stereotypes, contain explicit material, or spread misinformation. Deploying such guardrails in real-world applications, however, poses significant challenges. Users often require varied and highly customizable policies and typically cannot provide abundant examples for each custom policy. Consequently, an ideal guardrail should be scalable to the multiple policies and adaptable to evolving user standards with minimal retraining. Existing fine-tuning methods typically condition predictions on pre-defined policies, restricting their generalizability to new policies or necessitating extensive retraining to adapt. Conversely, training-free methods struggle with limited context lengths, making it difficult to incorporate all the policies comprehensively. To overcome these limitations, we propose to condition model’s judgment on ‘‘precedents’’, which are the reasoning processes of prior data points similar to the given input. By leveraging precedents instead of fixed policies, our approach greatly enhances the flexibility and adaptability of the guardrail. In this paper, we introduce a critique-revise mechanism for collecting high-quality precedents and two strategies that utilize precedents for robust prediction. Experimental results demonstrate that our approach outperforms previous methods across both few-shot and full-dataset scenarios and exhibits superior generalization to novel policies. 1 1 1 The code is available at: https://github.com/joeyy5588/Customize-Guardrail . † † footnotetext: Work was done at Amazon. 1 Introduction Responsible AI (RAI) ( Amazon 2024 ) aims to develop artificial intelligence systems that are safe, fair, helpful, and interpretable. Guardrails ( Gehman et al. 2020 ; Welbl et al. 2021 ) are approaches designed to realize this goal, it can help ensure that AI behavior aligns with societal values, ethics, and the specific needs of diverse users. As AI systems increasingly interact with diverse types of media, the need for multi-modal guardrails becomes crucial. This system is required to filter out image contents that may promote hate and violence, reinforce harmful stereotype, contain sexual material, or propagate misleading or fraudulent information ( Qu et al. 2024 ; Crone et al. 2018 ; Liu et al. 2023 ; Wu et al. 2020 ; Helff et al. 2024 ) . Figure 1 : (a) Models fine-tuned on pre-defined policies struggle with novel policies and require extensive retraining to adapt. As the number of policy scales, during inference it becomes infeasible to include all the polices and may include irrelevant ones. (b) Our proposed method addresses these limitations by conditioning on structured precedents , which explicitly capture the model’s reasoning processes from past decisions. This enables the guardrail to flexibly adapt and accurately handle dynamically evolving and novel policies without extensive retraining. Different AI service/model providers often require different policies to regulate their AI systems ( Dong et al. 2024b ) : one may prioritize filtering out erotic content, while another may be more concerned with violent material. Furthermore, even within the same category, definitions of what content should be blocked can vary significantly due to cultural differences–certain gestures or expressions might be considered offensive in one culture but neutral in another. An effective RAI guardrail system should, therefore, be both customizable and transparent. Not only should it accommodate user-defined policies, but it should also provide clear explanations for each filtering decision, enhancing user trust and accountability. However, a particular challenge arises from the need for flexibility in incorporating new rules, which often come in the form of few-shot learning tasks ( Dong et al. 2024a ) . When users introduce new, specific filtering requirements, there is typically little data available to define these unique policies. As shown in Fig. 1 (a), prior fine-tuning methods ( Schramowski et al. 2022 ; Qu et al. 2023 ; Helff et al. 2024 ; Schuhmann 2023 ; Praneeth 2019 ) , become less feasible as they are pre-trained on pre-defined policies and involve substantial computational overhead for retraining with each new policy. While in-context learning methods ( Dwivedi et al. 2023 ; Oba et al. 2023 ) offers an alternative, it becomes less viable as the number of policies grows. Limited context length constrains the ability to include all policies in the prompt, and including examples from irrelevant policies may degrade the guardrail’s effectiveness. Consequently, there is a need for methods that support scalable, flexible, and efficient customization of RAI guardrails without excessive retraining or labeling demands. To address the above challenges, we propose to condition model’s prediction on a precedent database, as shown in Fig. 1 (b). A precedent is a structured example of model’s reasoning process, capturing how it previously predicts whether an image is policy-violating (PV) according to a specified policy and the rationale behind its decision. As model learns to reason with precedents, which capture contextually relevant nuances that static policy definitions may fail to encapsulate, it can naturally evolve and expand to accommodate new rules without extensive retraining. In this paper, we first introduce a critique-revise mechanism to collect high-quality precedents, where the model reviews its own analyses, critiques incorrect predictions, and revises them accordingly. Importantly, this entire process operates without human in the loop. To utilize the precedent database effectively, we introduce two complementary methods that can be applied at training and inference time, respectively. First, for users with access to a local model like LLaVA ( Liu et al. 2024 ) , we demonstrate that fine-tuning with precedents collected via the critique-revise mechanism yields significant performance gains. Second, at inference time, the database can be leveraged through retrieval-augmented generation (RAG) to retrieve the most relevant precedent. This allows our method to scale efficiently even with a large number of diverse policies, and applicable for proprietary models like GPT-4V ( Achiam et al. 2023 ) and Claude ( Anthropic 2023 ) . We evaluate our method on UnsafeBench ( Qu et al. 2024 ) , which encompasses 11 distinct RAI policies. Compared to prior baselines, our approach achieves substantial improvements of 6.6% and 6.8% in F1 scores across few-shot and full-dataset settings, respectively. Moreover, when we isolate each policy from the training set and treat it as a novel category—simulating the model’s generalization capability to unseen policies—our method outperforms existing approaches by 16.7% in terms of F1. These results underscore our method’s effectiveness in efficiently scaling customizable RAI guardrails across diverse and dynamically evolving policy landscapes. 2 Related Work Datasets for Policy-Violating Visual Content. Several efforts have been made to evaluate the effectiveness of current models in detecting policy-violating (PV) visual content. For instance, Wu et al. 2020 introduced a video dataset specifically for violence detection, while Lee et al. 2024 benchmarked text-to-image (T2I) models to assess issues related to gender, fairness, and toxicity. Jha et al. 2024 proposed methods to evaluate nationality-based stereotypes in T2I models, highlighting biases that may arise across cultural representations. In the realm of multi-modal large language models (MLLM), Liu et al. 2023 examined the resilience of these models against malicious attacks. Meanwhile, OpenAI ( OpenAI , content no longer publicly available) established 11 categories for categorizing unsafe images, providing a foundational taxonomy for PV content detection. Building on this taxonomy, UnsafeBench ( Qu et al. 2024 ) gathered an extensive dataset, which includes