Learning Compact Video Representations for Efficient Long-form Video Understanding in Large Multimodal Models

Yuxiao Chen, Jue Wang, Zhikang Zhang, Jingru Yi, Xu Zhang, Yang Zou · 2026

Read Learning Compact Video Representations for Efficient Long-form Video Understanding in Large Multimodal Models by Yuxiao Chen, Jue Wang, Zhikang Zhang, Jingru Yi, Xu Zhang, Yang Zou (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2602-17869.txt.

Description

Yuxiao Chen and team introduce a compact video code schema for lean long-form video understanding in large multimodal models. Long-form videos spanning tens of minutes are now feasible to analyze. Yet video repeat-heavy nature challenges current models in two ways: fitting more frames within memory limits, and extracting telling information from vast input volumes.

The proposed end-to-end schema pairs an information-density-based tuning video sampler with an autoencoder-based space-time video squeezer, joined with a multi-input large language model. It tunes captures of essential information from clips of varying lengths and achieves high compression rates while keeping crucial telling information. Tests show good scores across benchmarks, shining in both long-form and standard video understanding tasks. The approach proves versatile and effective for long clips, where redundancy otherwise overwhelms models.

Catalog record

CREATORYuxiao Chen, Jue Wang, Zhikang Zhang, Jingru Yi, Xu Zhang, Yang Zou
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE41 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2602-17869

Browse the full library · About the Tower of Babel