Omni-SafetyBench: A Benchmark for Safety Evaluation of Audio-Visual Large Language Models
Leyi Pan, Zheyu Fu, Yunpeng Zhai, Shuchang Tao, Sheng Guan, Shiyu Huang, Lingzhe Zhang, Zhaoyang Liu, Bolin Ding, Felix Henry, Lijie Wen, Aiwei Liu · 2025
Read Omni-SafetyBench: A Benchmark for Safety Evaluation of Audio-Visual Large Language Models by Leyi Pan, Zheyu Fu, Yunpeng Zhai, Shuchang Tao, Sheng Guan, Shiyu Huang, Lingzhe Zhang, Zhaoyang Liu, Bolin Ding, Felix Henry, Lijie Wen, Aiwei Liu (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2508-07173.txt.
Introduction
Leyi Pan and a large team build a test bench for the safety of models that take sound, image, and text at once. Their paper is from two thousand twenty-five. Such models face joint attacks that mix sound and image, and their guard can shift when the same ask comes in a new form. Past benches miss joint samples and matched cross-form cases.
Their bench holds tens of thousands of cases in many forms, all drawn from a small seed set. New scores gauge both attack success and how steady a model's guard stays across forms. Of the top models tested, few pass a mid bar on both scores. Joint sound-image inputs drag safety down the most.
Catalog record
| CREATOR | Leyi Pan, Zheyu Fu, Yunpeng Zhai, Shuchang Tao, Sheng Guan, Shiyu Huang, Lingzhe Zhang, Zhaoyang Liu, Bolin Ding, Felix Henry, Lijie Wen, Aiwei Liu |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 57 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2508-07173
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Omni-SafetyBench: A Benchmark for Safety Evaluation of Audio-Visual Large Language Models Leyi Pan, Zheyu Fu, Yunpeng Zhai, Shuchang Tao, Sheng Guan, Shiyu Huang, Lingzhe Zhang, Zhaoyang Liu, Bolin Ding, Felix Henry, Lijie Wen, Aiwei Liu Abstract The rise of Omni-modal Large Language Models (OLLMs), which integrate visual and auditory processing with text, necessitates robust safety evaluations to mitigate harmful outputs. However, no dedicated benchmarks currently exist for OLLMs, and prior benchmarks designed for other LLMs lack the ability to assess safety performance under audio-visual joint inputs or cross-modal safety consistency. To fill this gap, we introduce Omni-SafetyBench, the first comprehensive parallel benchmark for OLLM safety evaluation, featuring 24 modality combinations and variations with 972 samples each, including dedicated audio-visual harm cases. Considering OLLMs’ comprehension challenges with complex omni-modal inputs and the need for cross-modal consistency evaluation, we propose tailored metrics: a Safety-score based on conditional Attack Success Rate (C-ASR) and Refusal Rate (C-RR) to account for comprehension failures, and a Cross-Modal Safety Consistency Score (CMSC-score) to measure consistency across modalities. Evaluating 6 open-source and 4 closed-source OLLMs reveals critical vulnerabilities: (1) no model excels in both overall safety and consistency, with only 3 models achieving over 0.6 in both metrics and top performer scoring around 0.8; (2) safety defenses weaken with complex inputs, especially audio-visual joints; (3) severe weaknesses persist, with some models scoring as low as 0.14 on specific modalities. Our benchmark and metrics highlight urgent needs for enhanced OLLM safety, providing a foundation for future improvements. Warning: This paper contains examples of harmful contents. † † footnotetext: ∗ Equal Contribution. † Corresponding authors. 1 Introduction Omni-modal large language models (OLLMs) have made rapid progress in comprehending and producing content based on integrated inputs from visual, audio, and text modalities. Drawing on data from various modalities, these models are able to perform more complex tasks, such as describing audio-visual scenes and executing voice instructions based on visual information. However, despite these advancements, ensuring their safety remains a critical concern that prevents these models from causing harm or acting in unethical, incorrect, or biased ways ( Yi et al. 2024 ; Liu et al. 2024b ) . Developing corresponding benchmarks serves as the cornerstone for reasonable assessment of safety evaluation. Numerous safety benchmarks have been established for text-only LLMs and vision-language models ( Zhang et al. 2023 ; Liu et al. 2024c ) , and recent work has extended to specialized benchmarks for audio-LLMs and video-LLMs ( Li et al. 2025a ; Liu et al. 2025 ; Lu et al. 2025 ) . However, for OLLMs capable of processing audio-visual joint inputs, there is currently a lack of benchmarks specifically designed to evaluate their safety. Table 1 compares existing safety benchmarks with our Omni-SafetyBench. Safety evaluation for OLLMs presents unique challenges that existing safety benchmarks fail to address: • Limited modality coverage : OLLMs can independently accept inputs from different modalities, including text, images, videos, and audio, while current benchmarks focus only on fixed modality inputs or specific combinations such as text-image pairs. • Absence of audio-visual joint evaluation : OLLMs can process audio-visual joint inputs, yet existing benchmarks do not contain such test samples to assess this critical capability. • Cross-modal safety consistency : For OLLMs with numerous possible input modality combinations, the consistency of safety performance across different modal transformations of the same source data is particularly noteworthy—a phenomenon we term cross-modal safety consistency . Models with weak cross-modal safety consistency are vulnerable to exploitation through simple modality conversion techniques, enabling large-scale jailbreaking attacks. Currently, no existing benchmark provides parallel evaluation capabilities or can assess cross-modal safety consistency. Modality Paradigm Modality Type Modality Variation Unimodal (1) Text-only - (2) Image-only - (3) Video-only - (4) Audio-only - Dual-modal Image-Text (5) Diffusion Image (6) TYPO Image (7) Diffusion+TYPO Image Video-Text (8) Diffusion Video (9) TYPO Video (10) Diffusion+TYPO Video Audio-Text (11) TTS Audio (12) TTS+Noise Audio Omni-modal (Audio-visual Input) Image-Audio-Text (13) Diff. Image with TTS Audio (14) Diff. Image with TTS+Noise Audio (15) TYPO Image with TTS Audio (16) TYPO Image with TTS+Noise Audio (17) Diff.+TYPO Image with TTS Audio (18) Diff.+TYPO Image with TTS+Noise Audio Video-Audio-Text (19) Diff. Video with TTS Audio (20) Diff. Video with TTS+Noise Audio (21) TYPO Video with TTS Audio (22) TYPO Video with TTS+Noise Audio (23) Diff.+TYPO Video with TTS Audio (24) Diff.+TYPO Video with TTS+Noise Audio (a) Taxonomy of Omni-SafetyBench. Figure 1: Overview of Omni-SafetyBench. (b) Harmful category distribution. #Image #Video #Audio 3888 3888 2916 (c) Multimedia data statistics. To address these challenges, we introduce Omni-SafetyBench, the first comprehensive parallel benchmark designed specifically for OLLM safety evaluation. Our benchmark uniquely features (1) diverse modality combinations as test inputs, (2) dedicated audio-visual joint harmful samples, and (3) 24 parallel test cases across different modalities to assess cross-modal safety consistency. Building upon MM-SafetyBench ( Liu et al. 2024c ) , we selected 972 entries as seed data to construct an extensive parallel benchmark that spans multiple modal combinations. As shown in Figure 1 , Omni-SafetyBench encompasses three modality paradigms: unimodal, dual-modal, and omni-modal. Each paradigm is further subdivided based on modality types and variations. For images, we include diffusion-generated images, typographic images, and hybrid images combining both diffusion-generated and typographic elements. Video data comprises diffusion-generated videos, typographic videos, and hybrid videos that merge diffusion-generated and typographic content. Audio data features text-to-speech audio both with and without noise overlay. The benchmark consists of totally 24 distinct subcategories, with each containing 972 samples. Harmful category distribution and multimedia data statistics of Omni-SafetyBench are presented in Figures 1(b) and 1(c) , respectively. For evaluation, we make two special considerations tailored to the characteristics of OLLMs. (1) OLLMs can accept inputs from various modality combinations, where more complex combinations often present greater understanding challenges. We argue that discussing safety without considering comprehension is meaningless, as failure to generate harmful content due to poor understanding does not indicate strong safety. Therefore, we first evaluate comprehension based on question-answer pairs, then conduct safety evaluation only on understood inputs. This yields conditional Attack Success Rate (C-ASR) and conditional Refusal Rate (C-RR) , from which we derive a Safety-score based on these two metrics. (2) As mentioned earlier, cross-modal safety consistency represents a crucial aspect of the safety of OLLMs. To address this, we design a Cross-Modal Safety Consistency Score (CMSC-score) to evaluate each model’s safety consistency across the 24 parallel subcategories in Omni-SafetyBench. Figure 2: Safety profiles of evaluated OLLMs based on Safety-score and CMSC-score performance. We evaluate 6 open-source and 4 closed-source state-of-the-art OLLMs using Omni-SafetyBench. Based on their performance on both Safety-score (overall safety performance) and CMSC-score (cross-modal safety consistency), we identify four distinct safety profiles among these models, as illustrated in Figure 2 . Through extensive experiments, our findings reveal several critical insights: (1) Current models struggle to excel in both overall safety performance and consistency. Only gemini-2.5-pro series and Qwen2.5-Omni-7b achieve above 0.6 in both metrics, with even the best-performing model reaching just 0.8 in both dimensions. (2) OLLMs’ safety defenses weaken significantly when facing more complex modality combinations, with audio-visual joint inputs emerging as the most effective trigger for exploiting safety vulnerabilities across most models. (3) The shortest planks identified by Omni-SafetyBench are particularly concerning: even