LongAV-Compass: Towards Unified Evaluation of Minute-Scale Audio-Visual Generation Across T2AV, I2AV, and V2AV

Tengfei Liu Yang Shi Xuanyu Zhu Jiafu Tang Liu Yang Qixun Wang Zhuoran Zhang, Yuqi Tang Fengxiang Wang Yuhao Dong Xinlong Chen Bozhou Li Bohan Zeng Yue Ding, Xiaohan Zhang Jialu Chen Haotian Wang Yuanxing Zhang Pengfei Wan Leye Wang · 2026

Read LongAV-Compass: Towards Unified Evaluation of Minute-Scale Audio-Visual Generation Across T2AV, I2AV, and V2AV by Tengfei Liu Yang Shi Xuanyu Zhu Jiafu Tang Liu Yang Qixun Wang Zhuoran Zhang, Yuqi Tang Fengxiang Wang Yuhao Dong Xinlong Chen Bozhou Li Bohan Zeng Yue Ding, Xiaohan Zhang Jialu Chen Haotian Wang Yuanxing Zhang Pengfei Wan Leye Wang (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-26244.txt.

Introduction

The 2026 benchmark from Tengfei Liu and nineteen colleagues, LongAV-Compass, judges minute-long audio-video generation across text, image, and video prompts. Its 284 curated cases span text-to-audio-video, image-to-audio-video, and video-to-audio-video tasks. They are scored on more than twenty dimensions from segment quality and cross-segment consistency to story coherence and sound-picture sync. Trials on eleven representative models, checked against human judgment, expose losses of identity, narrative, and alignment over longer horizons.

Catalog record

CREATORTengfei Liu Yang Shi Xuanyu Zhu Jiafu Tang Liu Yang Qixun Wang Zhuoran Zhang, Yuqi Tang Fengxiang Wang Yuhao Dong Xinlong Chen Bozhou Li Bohan Zeng Yue Ding, Xiaohan Zhang Jialu Chen Haotian Wang Yuanxing Zhang Pengfei Wan Leye Wang
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE67850
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2605.26244

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

LongAV-Compass: Towards Unified Evaluation of Minute-Scale Audio-Visual Generation Across T2AV, I2AV, and V2AV Tengfei Liu Yang Shi Xuanyu Zhu Jiafu Tang Liu Yang Qixun Wang Zhuoran Zhang, Yuqi Tang Fengxiang Wang Yuhao Dong Xinlong Chen Bozhou Li Bohan Zeng Yue Ding, Xiaohan Zhang Jialu Chen Haotian Wang Yuanxing Zhang Pengfei Wan Leye Wang Abstract Audio-visual generation is rapidly advancing from short clips to minute-long content, while existing evaluation protocols remain largely confined to short-form settings. Existing benchmarks primarily focus on 5–10 second text-conditioned generation and rarely support unified evaluation across text, image, and video conditioning modalities. Moreover, they provide limited insight into how identity consistency, narrative coherence, and audio-visual alignment degrade over extended temporal horizons. To bridge this gap, we introduce LongAV-Compass , a systematic benchmark for minute-long audio-visual generation. LongAV-Compass contains 284 curated test cases spanning text-to-audio-video (T2AV), image-to-audio-video (I2AV), and video-to-audio-video (V2AV), organized by application scenario and generation complexity. The benchmark combines taxonomy-guided benchmark construction with a unified evaluation framework that integrates MLLM-assisted assessment with complementary perceptual and multimodal metrics, including DINO-v2, ArcFace, CLIP, and ImageBind. The framework evaluates more than 20 fine-grained dimensions covering within-segment quality, cross-segment consistency, global narrative coherence, semantic alignment, and audio-visual synchronization. Through experiments on 11 representative models together with human-alignment validation, LongAV-Compass provides a diagnostic testbed for analyzing the limitations of current systems in sustaining coherent, semantically aligned, and temporally consistent minute-scale audio-visual generation across diverse input modalities. Keywords: Audio-Visual Generation, Long Video Generation, Evaluation 1 Introduction Figure 1 : Overview of LongAV-Compass . The benchmark unifies T2AV, I2AV, and V2AV under shared taxonomy, event-level annotation, and a hierarchical evaluation framework, enabling diagnosis of long-range audio-visual failures beyond flat leaderboard comparison. Recent advances in video generation models are pushing audio-visual generation beyond short clips. Commercial and open-source systems increasingly support longer durations, richer prompting, and native or compositional audio generation, making minute-scale outputs relevant to applications such as vlogs, tutorials, product demonstrations, advertisements, and story-driven content. In this setting, success is no longer determined by producing a visually plausible 5-second clip. Instead, models must sustain subject identity, event continuity, scene transitions, and audio grounding over substantially longer temporal horizons. However, evaluation has not kept pace with this shift. Existing benchmarks for video and audio-visual generation remain largely focused on short-form settings, where a single clip is often sufficient to assess local visual quality or coarse semantic alignment. Benchmarks such as VBench 8 and EvalCrafter 13 have advanced standardized evaluation for video generation models, while recent audio-visual benchmarks such as VABench 7 and T2AV-Compass 2 further extend evaluation to synchronized audio generation. These benchmarks provide valuable tools for short-video assessment, but their design does not fully capture the challenges of long-form generation, where failures often emerge only across multiple events, larger temporal gaps, or prolonged audio-visual interactions. This gap leads to three key limitations. First, current benchmarks operate at a temporal scale that provides limited evidence about whether models can remain coherent over minute-long generation. Second, their coverage is often fragmented across input conditions, making it difficult to compare text-to-audio-video (T2AV), image-to-audio-video (I2AV), and video-to-audio-video (V2AV) systems under a unified protocol. Third, current evaluation offers limited diagnostic visibility into long-range degradation, such as cross-event identity drift, weak continuation quality, unstable scene transitions, and the decay of audio-visual synchronization as duration increases. Table 1 : Comparisons between LongAV-Compass and representative video and audio-visual generation benchmarks. LongAV-Compass focuses on two missing axes in prior evaluation: unified X2AV coverage across T2AV, I2AV, and V2AV, and longer sample duration. Here, V2A denotes generating audio for a given video, whereas V2AV evaluates video-conditioned audio-video continuation. The final column indicates whether the benchmark explicitly reports an average sample or video duration exceeding one minute. Benchmark #Samples T2V T2AV I2AV V2A V2AV Unified X2AV Avg. Video Duration > 1min MSVBench 22 276 ✓ ✗ ✗ ✗ ✗ ✗ ✗ AVGen-Bench 32 235 ✗ ✓ ✗ ✗ ✗ ✗ ✗ T2AV-Compass 2 500 ✗ ✓ ✗ ✗ ✗ ✗ ✗ VABench 7 1,299 ✗ ✓ ✓ ✗ ✗ ✗ ✗ PhyAVBench 28 337 ✗ ✓ ✓ ✓ ✗ ✗ ✗ VinTAGe-Bench 11 636 ✗ ✗ ✗ ✓ ✗ ✗ ✗ LongAV-Compass 284 ✗ ✓ ✓ ✗ ✓ ✓ ✓ As summarized in Table 1 , existing benchmarks typically cover only part of the X2AV task space or remain focused on short-form generation, leaving unified minute-scale audio-visual evaluation underexplored. To address these limitations, we introduce LongAV-Compass , a unified benchmark for minute-scale audio-visual generation. LongAV-Compass contains 284 curated test cases, including 128 T2AV examples, 115 I2AV examples, and 41 V2AV examples. The benchmark is organized according to a two-dimensional taxonomy of application scenario and generation complexity, covering Vlog, Content-Creator, Performance Ads, and Brand Ads. Each test case is annotated with both a global description and event-level structure, enabling evaluation of long-form narrative organization rather than isolated frames or short clips. Beyond dataset construction, LongAV-Compass provides a unified evaluation framework tailored to long-form audio-visual generation. The framework assesses more than 20 fine-grained dimensions spanning within-segment video quality, cross-segment consistency, global narrative coherence, long-audio quality, audio-visual synchronization, and input-conditioned semantic alignment. It follows an MLLM-centered evaluation protocol based on Gemini 3.1 Pro 4 , complemented by specialized perceptual and multimodal metrics including DINO-v2 17 and CLIP 18 . This hybrid design enables evaluation from complementary perspectives, including segment-level quality, cross-segment subject consistency, script following, semantic alignment, image anchoring, video continuation quality, and audio-visual synchronization. We further conduct a human-alignment study to validate the reliability of the resulting scores. Figure 1 illustrates the overall design of LongAV-Compass . It unifies T2AV, I2AV, and V2AV under a shared taxonomy, event-level annotation schema, and hierarchical evaluation framework, while still supporting task-specific diagnostics and leaderboards. Rather than serving as a simple extension of short-form leaderboards, LongAV-Compass is designed as a diagnostic benchmark for understanding long-form audio-visual generation. Through unified evaluation of 11 representative systems, it enables systematic analysis of model capabilities and failure modes, including long-range identity drift, brittle event transitions, conditioning-specific weaknesses, and unstable minute-scale audio continuity. Our contributions are summarized as follows: • We introduce LongAV-Compass , the first benchmark dedicated to minute-scale audio-visual generation across text, image, and video inputs, with 284 curated test cases organized by application scenario and generation complexity. • We design a unified evaluation framework for long-form audio-visual generation across T2AV, I2AV, and V2AV. The framework evaluates more than 20 dimensions and decomposes long-video assessment into three complementary perspectives: within-segment quality, cross-segment consistency, and global narrative coherence, together with audio-visual synchronization and input-conditioned semantic alignment. • We conduct a comprehensive evaluation of 11 representative generation systems under the proposed protocol. Beyond overall ranking, our analysis reveals the capabilities current models handle well and the failure modes they still exhibit, providing a systematic diagnosis of long-form audio-visual generation. 2 Related Work 2.1 Benchmarks on Short-Form Video Generation Progress in benchmarking video generation has been largely driven by short-form evaluation suites such as VBench 8 , EvalCrafter 13 , and FETV 14 . These benchmarks define systematic evaluation dimensions covering visual quality, motion realism, semantic alignment, and prompt following 9 ; 5 ; 23 , enabling more standardized comparisons among video generation models. However, their protocols are primarily designed for short text-conditioned clips,

Read the complete text