EvA: An Evidence-First Audio Understanding Paradigm for LALMs
Xinyuan Xie, Shunian Chen, Zhiheng Liu, Yuhao Zhang, Zhiqiang Lv, Liyin Liang, Benyou Wang, The Chinese University of Hong Kong, Shenzhen, Didi Chuxing [email protected] · 2026
Read EvA: An Evidence-First Audio Understanding Paradigm for LALMs by Xinyuan Xie, Shunian Chen, Zhiheng Liu, Yuhao Zhang, Zhiqiang Lv, Liyin Liang, Benyou Wang, The Chinese University of Hong Kong, Shenzhen, Didi Chuxing [email protected] (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2603-27667.txt.
Introduction
Xie and co-authors argue audio models fail first at hearing, not reasoning. EvA keeps acoustic evidence intact before any reasoning begins. Two paths merge sound cues without crushing time detail. A new training set backs the design. It holds about 54,000 ordered captions. It also holds 500,000 evidence-based question pairs. EvA tops open-source perception scores on three benchmarks. Human raters praise its fine caption detail.
Catalog record
| CREATOR | Xinyuan Xie, Shunian Chen, Zhiheng Liu, Yuhao Zhang, Zhiqiang Lv, Liyin Liang, Benyou Wang, The Chinese University of Hong Kong, Shenzhen, Didi Chuxing [email protected] |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 57931 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2603.27667
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
EvA: An Evidence-First Audio Understanding Paradigm for LALMs Xinyuan Xie, Shunian Chen, Zhiheng Liu, Yuhao Zhang, Zhiqiang Lv, Liyin Liang, Benyou Wang, The Chinese University of Hong Kong, Shenzhen, Didi Chuxing [email protected] Abstract Large Audio Language Models (LALMs) still struggle in complex acoustic scenes because they often fail to preserve task-relevant acoustic evidence before reasoning begins. We call this failure the evidence bottleneck: state-of-the-art systems show larger deficits in evidence extraction than in downstream reasoning, suggesting that the main limitation lies in upstream perception rather than reasoning policy. To address this problem, we propose EvA (Evidence-First Audio), a dual-path architecture that combines Whisper and CED-Base through non-compressive, time-aligned fusion. EvA first aggregates intermediate CED layers to preserve multi-scale acoustic cues, then aligns the aggregated CED features to the Whisper timeline and adds the two streams without changing sequence length. We also build EvA-Perception, a large-scale open-source training set with about 54K event-ordered captions (150 h) and about 500K QA pairs. Under a unified zero-shot protocol, EvA achieves the best open-source Perception scores on MMAU, MMAR, and MMSU, and improves over Kimi-Audio-7B on all reported metrics, with the largest gains on perception-heavy splits. These results support the evidence-first hypothesis: stronger audio understanding depends on preserving acoustic evidence before reasoning. 1 Introduction Large Audio Language Models (LALMs) aim to empower machines with the ability to listen, understand, and reason from sound. While recent systems like Qwen2.5-Omni ( 31 ) and Kimi-Audio ( 8 ) have demonstrated impressive performance on various benchmarks, their capabilities degrade sharply when confronted with complex acoustic scenes involving overlapping events, faint signals, or fine-grained temporal cues. Figure 1: Perception–reasoning performance gap comparison between models and humans. Model performance is averaged over Qwen2.5–Omni and Kimi–Audio–7B. We argue that this weakness comes mainly from upstream perception, not from downstream reasoning. As shown in Fig. 1 , the gap between leading LALMs and humans is larger on perception-centric tasks than on reasoning-centric tasks. On MMSU, the model–human gap is 48.4 points in perception (42.8% vs. 91.2%) but only 13.3 points in reasoning. We refer to this pattern as the “evidence bottleneck”: current LALMs are more limited by how well they preserve and represent acoustic evidence than by how well they reason over that evidence. We attribute this evidence bottleneck to three common design choices in current LALMs. First, reasoning-centric optimization : post-training methods such as SFT and GRPO mainly improve how the model reasons over available evidence, but they cannot recover acoustic evidence that has already been lost upstream. Once task-relevant acoustic details are discarded in the encoder or interface, later stages can only work with the remaining signal. Second, frequency information loss : many encoders process spectrograms as 1D sequences, which weakens or removes frequency-localized cues that are important for non-speech audio. Third, weak alignment interfaces : existing dual-path systems often rely either on lossy temporal compression, such as Q-Former modules ( 24 ) , or on simple feature concatenation without a shared temporal coordinate ( 11 ) . Both choices make it harder for the LLM to use speech and non-speech evidence jointly. To address this bottleneck, we introduce EvA (Evidence-First Audio) , a dual-path architecture designed to preserve acoustic evidence before reasoning. EvA uses two complementary streams: a Whisper encoder for speech content and a non-speech encoder (CED-Base ( 9 ) ) for non-speech events. Its core design is a two-stage, non-compressive fusion process. First, EvA performs hierarchical evidence aggregation within the CED path by combining intermediate and final CED layers, preserving multi-scale acoustic cues. Second, it applies time-aware additive fusion to align the CED features to the Whisper timeline and combine the two streams without reducing sequence length. Together, these steps preserve more acoustic evidence and make speech and non-speech cues available to the LLM on a shared timeline. To train this architecture, we develop EvA-Perception, a large-scale audio QA dataset with 54K event-ordered captions and 500K QA pairs. Built from the temporal annotations in AudioSet-Strong ( 13 ) , it is designed to teach models to preserve and use fine-grained acoustic evidence. Fine-tuned on only 380 hours of audio, EvA achieves strong and consistent gains on complex audio understanding and acoustic-scene benchmarks, including MMAU ( 22 ) , MMAR ( 17 ) , MMSU ( 26 ) , and CochlScene ( 14 ) . The gains are largest on perception-heavy tasks, which is consistent with our evidence-first hypothesis. Our main contributions are summarized as follows: • Problem Diagnosis We identify and empirically support the “evidence bottleneck” in state-of-the-art LALMs: the main limitation lies in upstream perception, not downstream reasoning. • The EvA Architecture We propose EvA, a dual-path architecture that preserves acoustic evidence before reasoning through hierarchical aggregation and non-compressive, time-aligned fusion. • Open-source Dataset and Strong Model We release EvA-Perception, a dataset for training evidence-aware audio understanding, and the EvA model, which achieves the best open-source Perception results on MMAU, MMAR, and MMSU. 2 Related Works 2.1 Large Audio Language Models Large Audio Language Models (LALMs) have progressed rapidly, with systems such as Qwen2-Audio ( 6 ) , Qwen2.5-Omni ( 31 ) , and Kimi-Audio ( 8 ) . Most of these models rely on a single Whisper encoder ( 21 ) , which works well for speech but is less effective at preserving non-speech evidence. Prior dual-path designs partly address this limitation, but SALMONN ( 24 ) relies on lossy Q-Former compression, and GAMA ( 11 ) combines features without a strong shared temporal coordinate. A detailed structural comparison with SALMONN-style Q-Former fusion is provided in Appendix A.11 . These limitations motivate our dual-path, non-compressive, time-aligned fusion design. 2.2 Audio Understanding Benchmarks Audio-language evaluation has shifted from captioning datasets such as AudioCaps ( 15 ) and Clotho ( 10 ) to more demanding benchmarks including MMAU ( 22 ) , MMAR ( 17 ) , and MMSU ( 26 ) . These benchmarks require both fine-grained perception and higher-level reasoning over complex acoustic scenes. In this setting, performance is often limited by how well models preserve acoustic evidence before reasoning, which directly motivates our evidence-first perspective and the construction of EvA-Perception. Figure 2: Model architecture. The left half of the left panel shows the Kimi-Audio backbone, while the right half illustrates the additional EvA Path modules. Audio is encoded into four frequency-band features by the CED Encoder, pooled across frequencies, fused via cross-attention, temporally aligned with Whisper, and integrated through gated additive fusion. 3 Motivation: the Evidence Bottleneck The perception–reasoning gap in Sec. 1 suggests an architectural bottleneck rather than a pure training issue. We analyze the LALM pipeline through a conceptual information-flow lens to identify where acoustic evidence is attenuated. We first examine the constraints of single-path models (Sec. 3.1 ), then show how dual-path architectures can relax these constraints in practice (Sec. 3.2 ). Formal derivations are provided in Appendix A.4 . Notation. We use Z to denote the latent acoustic evidence required for the downstream task, such as event identities, temporal boundaries, and event order. Let X be the raw audio waveform, H the encoder hidden representation, O the representation passed to the language model, and Y the model output. We denote mutual information by I(\cdot;\cdot) and assume only that a