LALM-as-a-Judge: Benchmarking Large Audio-Language Models for Safety Evaluation in Multi-Turn Spoken Dialogues
Amir Ivry, Shinji Watanabe · 2026
Read LALM-as-a-Judge: Benchmarking Large Audio-Language Models for Safety Evaluation in Multi-Turn Spoken Dialogues by Amir Ivry, Shinji Watanabe (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2602-04796.txt.
Description
Amir Ivry and Shinji Watanabe benchmark audio-language models as safety judges for spoken dialogue in a 2026 study. Evaluating harmful content in speech has stayed text-centric, missing tone of voice and errors that transcription introduces.
Their benchmark contains many thousands of multi-turn spoken dialogues, each with one localized unsafe turn drawn from several harm categories and severity levels. Six models, open and closed, were tested in text-only, audio-only, and combined setups. The authors measured sensitivity, severity ordering, and bias by turn position. Audio added evidence beyond the words themselves. Combined setups helped unevenly, with some models anchored to text, others balanced or conservative. The paper closes with practical guidance for choosing models, modalities, and prompts.
Catalog record
| CREATOR | Amir Ivry, Shinji Watanabe |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 73 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2602-04796