π©π π«π¬π‘πΎππΌπ \mathsf{JALMBench} : Benchmarking Jailbreak Vulnerabilities in Audio Language Models
Zifan Peng, Yule Liu, Zhen Sun, Mingchen Li, Zeren Luo, Jingyi Zheng, Wenhan Dong, Xinlei He, Xuechao Wang, Yingjie Xue, Shengmin Xu, Xinyi Huang Β· 2025
Read π©π π«π¬π‘πΎππΌπ \mathsf{JALMBench} : Benchmarking Jailbreak Vulnerabilities in Audio Language Models by Zifan Peng, Yule Liu, Zhen Sun, Mingchen Li, Zeren Luo, Jingyi Zheng, Wenhan Dong, Xinlei He, Xuechao Wang, Yingjie Xue, Shengmin Xu, Xinyi Huang (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2505-17568.txt.
Introduction
Zifan Peng and co-authors build JALMBench, a benchmark for jailbreak attacks on large audio language models. Their 2025 paper fills a gap: these models are spreading into real use, but no shared adversarial audio dataset or unified test frame existed for their safety. The benchmark holds over eleven thousand text samples and more than 245,000 audio samples, over a thousand hours in all.
Twelve mainstream models face eight attack methods, four adapted from text and four born in audio, plus five defenses. Safety proved to hinge on modality and design: text alignment partly carried into audio, and interleaved audio text training generalized best. Standard moderation tools helped only slightly, so the authors call for audio specific defenses.
Catalog record
| CREATOR | Zifan Peng, Yule Liu, Zhen Sun, Mingchen Li, Zeren Luo, Jingyi Zheng, Wenhan Dong, Xinlei He, Xuechao Wang, Yingjie Xue, Shengmin Xu, Xinyi Huang |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 89 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2505-17568
Browse the full library Β· About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
π©π π«π¬π‘πΎππΌπ \mathsf{JALMBench} : Benchmarking Jailbreak Vulnerabilities in Audio Language Models Zifan Peng, Yule Liu, Zhen Sun, Mingchen Li, Zeren Luo, Jingyi Zheng, Wenhan Dong, Xinlei He, Xuechao Wang, Yingjie Xue, Shengmin Xu, Xinyi Huang Abstract Large Audio Language Models (LALMs) have made significant progress. While increasingly deployed in real-world applications, LALMs face growing safety risks from jailbreak attacks that bypass safety alignment. However, there remains a lack of an adversarial audio dataset and a unified framework specifically designed to evaluate and compare jailbreak attacks against them. To address this gap, we introduce π©π π«π¬π‘πΎππΌπ \mathsf{JALMBench} , a comprehensive benchmark that assesses LALM safety against jailbreak attacks, comprising 11,316 text samples and 245,355 audio samples (>1,000 hours). π©π π«π¬π‘πΎππΌπ \mathsf{JALMBench} supports 12 mainstream LALMs, 8 attack methods (4 text-transferred and 4 audio-originated), and 5 defenses. We conduct in-depth analysis on attack efficiency, topic sensitivity, voice diversity, and model architecture. Additionally, we explore mitigation strategies for the attacks at both the prompt and response levels. Our systematic evaluation reveals that LALMsβ safety is strongly influenced by modality and architectural choices: text-based safety alignment can partially transfer to audio inputs, and interleaved audio-text strategies enable more robust cross-modal generalization. Existing general-purpose moderation methods only slightly improve security, highlighting the need for defense methods specifically designed for LALMs. We hope our work can shed light on the design principles for building more robust LALMs. 1 Introduction Powered by Large Language Models (LLMs), Large Audio Language Models (LALMs) ( Chu et al., 2024 ; Zeng et al., 2024a ; Nguyen et al., 2025 ) incorporate audio as a new modality and show remarkable performance in a wide range of tasks, including speech understanding ( Arora et al., 2024 ) , spoken question answering ( Nachmani et al., 2024 ) , audio captioning ( Wu et al., 2024 ) , etc. However, existing studies ( Gong et al., 2025 ; Zhang et al., 2025 ) demonstrate that multimodal models are vulnerable to jailbreak attacks. For LALMs, jailbreak methods similar to those used for LLMs ( Yi et al., 2024 ) can be applied, which can be transferred to audio inputs from text ( text-transferred attacks ). Recent research ( Kang et al., 2025 ) also shows that the adversary can directly manipulate the audio to conduct attacks ( audio-originated attacks ). However, the field of LALM safety lacks a unified evaluation framework and large-scale benchmark datasets. This gap is primarily caused by inconsistent code implementations across studies and the high cost of querying Text-to-Speech (TTS) services. As a result, research on attacks against LALMs remains fragmented, leading to isolated development of attack methods and making fair comparisons between existing techniques difficult. To address this gap, we introduce \mathsf{JALMBench} , a comprehensive benchmarking framework for evaluating jailbreak attacks in LALMs. Summary of \mathsf{JALMBench} is shown in Figure 1 . The main content of this paper can be outlined as follows: - Dataset. \mathsf{JALMBench} contains 245,355 audio samples over 1,000 hours and 11,316 text samples. These samples are divided into three parts. The first part consists of harmful queries, including 246 original text samples, their audio counterparts with TTS, and 4,182 audio samples with variations in accents, gender, TTS methods, and languages. The second part includes 11,070 jailbreak text queries generated via 4 text-based attacks, along with their audio counterparts using TTS. The final part contains 229,857 jailbreak audio queries generated via 4 audio-originated attacks. - Evaluation. We use \mathsf{JALMBench} to evaluate 12 mainstream LALMs against different attacks with text and audio inputs. For non-adversarial harmful queries, the average attack success rate (ASR) in the audio modality (21.5%) is higher than in the text modality (17.0%). For jailbreak attacks, the strongest attack (AdvWave) yields an ASR of 96.2%. These results demonstrate the jailbreak vulnerability of current LALMs. Figure 1: The framework and summary of \mathsf{JALMBench} . - Analysis. In addition, we conduct an in-depth analysis from multiple perspectives: attack efficiency, topic sensitivity, voice diversity, and architecture. Regarding efficiency, while achieving an ASR above 60% typically requires at least 100 seconds of processing, an ASR of around 40% can be attained within just 10 seconds, highlighting the feasibility of low-cost, real-world jailbreak attempts ( Figure 5 ). For topics, we find that LALMs are relatively effective at rejecting explicit hate content but remain vulnerable to subtler categories such as misinformation ( Figure 5 ). For voice diversity, our analysis reveals that non-US accents tend to increase ASR, likely due to underrepresentation in the training data ( Tables 1 and 10 ). For the effect of architecture, we uncover several insights into alignment behaviors during attacks, suggesting that certain input transformations may exploit gaps in model generalization or modality fusion. The encoding strategy inherently determines the safety properties of the system: discrete tokenization may better preserve the safety characteristics inherent to the textual modality compared to continuous feature extraction. - Potential Defenses. Despite the revealed vulnerability, to the best of our knowledge, no prior work has explored defense strategies specifically tailored to LALM-based jailbreak attacks. As a first step, we investigate two practical defense approaches, prompt-level and response-level moderation. Both strategies improve safety, with the best method in each category reducing average ASR by 19.6 and 18.0 percentage points, respectively. Moreover, prompt-level mitigation incurs a non-negligible utility performance drop, revealing a trade-off between safety and utility. The moderate effectiveness of current mitigation techniques suggests that future work should explore defenses specifically designed for the audio modality. Our contributions can be summarized as follows: β’ We introduce \mathsf{JALMBench} , a comprehensive benchmark for evaluating jailbreak attacks on LALMs. It includes 245,355 audio samples, over 1,000 hours of audio data, and a unified modular evaluation framework with standardized APIs and implementable classes. β’ We benchmark the robustness of 12 LALMs against 8 types of text-transferred and audio-originated attacks and conduct in-depth analysis of LALM behaviors, revealing key vulnerability patterns such as attention drift and misclassification tendencies. β’ We evaluate prompt-level and response-level defense strategies to assess the robustness and reliability of LALMs against adversarial threats and explore the corresponding utility of LALMs. These defense strategies achieve only a small improvement in average safety performance (11.3%), highlighting that specific effective defenses for LALMs remain largely unexplored. 2 Related Work Large Audio Language Models (LALMs). Powered by the ability of LLMs in different areas ( Lee et al., 2024 ; Liu et al., 2025b ; Luo et al., 2025 ) , LALMs have shown remarkable performance in a wide range of tasks, including speech understanding, spoken question answering, and audio captioning. LALMs typically employ a speech encoder to convert raw audio into high-level acoustic representations, which are then processed with text embeddings together ( Chang et al., 2024 ) . Current LALMs can be primarily categorized into two groups based on their audio encoding strategies. The first category employs continuous feature extraction, where pre-trained speech encoders, such as Whisper ( Radford et al., 2023 ) , extract acoustic features from audio. These features are mapped into a single embedding spaceβs vector per audio segment and concatenated with textual embeddings before being processed by the backbone LLM. The second category uses token-based audio encoding strategies, converting audio inputs into discrete symbol sequences. Neural audio encoders, such as