Task-Aware KV Compression For Cost-Effective Long Video Understanding
Minghao Qin 1 Yan Shu 1,3 Peitian Zhang 1,4 Kun Lun 1,7 Huaying Yuan 1,4 Juenjie Zhou 1,5 Shitao Xiao 1 Bo Zhao 1,2 Zheng Liu 1,6† 1 Beijing Academy of Artificial Intelligence 2 Shanghai Jiao Tong University 3 University of Trento 4 Renmin University of China 5 Beijing University of Posts and Teleco · 2025
Read Task-Aware KV Compression For Cost-Effective Long Video Understanding by Minghao Qin 1 Yan Shu 1,3 Peitian Zhang 1,4 Kun Lun 1,7 Huaying Yuan 1,4 Juenjie Zhou 1,5 Shitao Xiao 1 Bo Zhao 1,2 Zheng Liu 1,6† 1 Beijing Academy of Artificial Intelligence 2 Shanghai Jiao Tong University 3 University of Trento 4 Renmin University of China 5 Beijing University of Posts and Teleco (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2506-21184.txt.
Introduction
Minghao Qin and co-authors cut the cost of understanding long videos with language models, in this 2025 paper. Long footage floods such models with cached keys and values, and squeezing that cache usually sheds detail. Their method, Video-X2L, stores the cache at two squeeze levels during the reading pass.
Fine copies keep detail; coarse copies stay compact. While answering, the system reloads fine copies only for the chunks a task truly needs. No retraining is required, and the method plugs into existing models. On four long-video benchmarks, it beat rival squeeze methods while saving much compute.
Catalog record
| CREATOR | Minghao Qin 1 Yan Shu 1,3 Peitian Zhang 1,4 Kun Lun 1,7 Huaying Yuan 1,4 Juenjie Zhou 1,5 Shitao Xiao 1 Bo Zhao 1,2 Zheng Liu 1,6† 1 Beijing Academy of Artificial Intelligence 2 Shanghai Jiao Tong University 3 University of Trento 4 Renmin University of China 5 Beijing University of Posts and Teleco |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 39 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2506-21184
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Task-Aware KV Compression For Cost-Effective Long Video Understanding Minghao Qin 1 Yan Shu 1,3 Peitian Zhang 1,4 Kun Lun 1,7 Huaying Yuan 1,4 Juenjie Zhou 1,5 Shitao Xiao 1 Bo Zhao 1,2 Zheng Liu 1,6† 1 Beijing Academy of Artificial Intelligence 2 Shanghai Jiao Tong University 3 University of Trento 4 Renmin University of China 5 Beijing University of Posts and Telecommunications 6 Hong Kong Polytechnic University 7 Institute of Automation, CAS, Beijing, China Abstract Long-video understanding (LVU) remains a severe challenge for existing multimodal large language models (MLLMs), primarily due to the prohibitive computational cost. Recent approaches have explored KV compression to mitigate this issue, but they often suffer from significant information loss at high compression ratios. In this paper, we introduce Video-X 2 L , which flexibly preserves critical video information for each LVU task. Video-X 2 L involves two key operations. The first one is called bi-level KV compression . During the MLLM’s pre-filling stage, Video-X 2 L generates two types of compressed KVs: low-compression KVs (L-KVs) to capture fine-grained video details and high-compression KVs (H-KVs) to offer compact video representations. The second one is called selective KV re-loading . During the MLLM’s decoding stage, Video-X 2 L selectively re-loads L-KVs for the most critical video chunks while using H-KVs for other less important ones. This allows the MLLM to fully utilize task-specific information while maintaining the overall compactness. Video-X 2 L is simple yet effective: it is free from additional training and directly compatible with existing KV-compressible MLLMs. We evaluate Video-X 2 L with a variety of popular LVU benchmarks, including VideoMME, MLVU, LongVideoBench, and VNBench. Our experiment result shows that Video-X 2 L outperforms existing KV-compression methods by a huge advantage while substantially saving the computation cost. Our source code has been made publicly available at the this repository 1 1 1 https://github.com/UnableToUseGit/VideoX22L . Figure 1: The framework of Video-X 2 L. First, it performs bi-level KV compression, which compresses KV cache into L-KVs (for fine-grained details) and H-KVs (for abstract representations). Second, it make selective use of the compressed KVs based on their importance to each task. The final answer is generated using the selectively re-loaded KVs of hybrid compression ratios. 1 Introduction Multimodal Large Language Models (MLLMs) have made substantial progress in addressing tasks that require perception and reasoning over visual data reid2024gemini ; gpt4o ; liu2023visual_llava . Recently, there has been growing interest in extending MLLMs to video-related scenarios videollama ; videollava ; maaz2023videochatgpt ; li2023videochat , given their strong potential in modeling the spatial-temporal information within videos videollama ; videollava ; maaz2023videochatgpt ; li2023videochat . However, existing MLLMs face significant challenges in handling long videos, primarily due to the prohibitive computational and memory costs. In particular, long videos consist of numerous frames, each producing a huge amount of visual tokens. As a result, the input often exceeds the model’s context window, making it computationally expensive and impractical for real-world applications. Even with extended context lengths, processing such a large number of visual tokens remains inefficient and resource-intensive. To alleviate these issues, various compression strategies have been explored to enable cost-effective processing of long videos. One common approach is token reduction, which removes redundant tokens before they are input into MLLMs malmm2024 ; llamavid ; maaz2023videochatgpt ; moviechat2023 . Recently, KV compression has been introduced as a more promising strategy, which leverages MLLMs’ inherent sparsification capability to generate compact representations for long videos ye2024voco ; shu2024video ; liu2025video . However, KV compression still suffers from severe information loss especially under high compression ratios, limiting its ability to support tasks that require fine-grained analysis of long videos. In this paper, we propose a novel framework for long-video understanding, called Video-X 2 L. This method introduces two key operations to enhance the generation and utilization of compressed KVs (shown as Figure 1 ). \bullet Bi-level KV compression . Video-XL geneartes two types of compressed KVs during the pre-filling stage of MLLM: low-compression KVs (L-KVs) and high-compression KVs (H-KVs). Specifically, the input video is first partitioned into a series of chunks. While encoding each chunk, the original KVs from MLLMs are compressed as L-KVs using a small compression ratio, which preserves fine-grained information about the video. Meanwhile, the KVs are also compressed as H-KVs using a large compression ratio, resulting in highly abstract representations for the corresponding chunk. Both forms of KVs are cached in the main memory and reused by downstream tasks, thereby enabling the saving of computation cost. \bullet Selective KV re-loading . Video-XL makes selective use of the compressed KVs during the decoding stage of MLLM. Given a specific LVU task, it first estimates the importance of each video chunk using a relevance oracle (e.g., a vision-language model, like CLIP, or simply the vanilla attention score from the MLLM). Based on this result, it recalls L-KVs for the most critical video chunks and H-KVs for other less important ones. The recalled KVs are re-ordered based on their relative position and re-loaded to the GPU memory. The decoding step is performed on top of the KVs with hybrid compression ratios. While the majority of video chunks are represented by H-KVs, the most critical ones are highlighted by L-KVs. This allows Video-X 2 L to generate high-quality answers while maintaining strong cost-effectiveness. Additionally, Video-X 2 L can be seamlessly integrated with existing KV-compressible MLLMs and arbitrary relevance oracles in a training-free way, making it extremely easy to be adopted in practice. We perform comprehensive experiments for Video-X 2 L using a variety of popular LVU benchmarks, including VideoMME videomme , MLVU zhou2024mlvu , LongVideoBench wu2024longvideobench , and VNBench vnbench . Our results show that Video-X 2 L significantly enhances MLLM performance in long-video understanding across diverse scenarios. At the same time, it achieves substantial reductions in memory usage and notable improvements in inference speed, which enables more efficient processing of LVU tasks. Finally, the empirical advantage become more pronounced with higher compression ratios or longer video inputs, indicating Video-X 2 L’s potential in addressing the challenges posed by extra-long videos. To summarize, the contributions of this paper are highlighted as follows. • We propose a novel KV-compression framework Video-X 2 L for LVU tasks for efficient and high-quality processing of LVU tasks. • We introduce two key oeprations: bi-level KV compression and selective KV re-loading, to improve the generation and utilization of compressed KVs. • We perform comprehensive experiment, verifying the effectiveness and efficiency of Video-X 2 L in addressing the challenges associated with long-video understanding. 2 Related Work Multimodal Large Language Models. Multimodal Large Language Models (MLLMs) are built upon Large Language Models (LLMs). They employ visual encoders to extract visual features and then utilize connectors to align these features with textual features, thereby enabling the model to effectively understand visual content (e.g., alayrac2022flamingo ; li2023blip2 ; liu2023llava ; gpt4o ; reid2024gemini ; zhu2023minigpt ). Over the past few years, MLLMs have achieved rapid development in both commercial and open-source domains. Commercial models, such as GPT-4o gpt4o and Gemini reid2024gemini , have long held a leading position in the field of multimodal understanding. However, recently, several excellent open-source MLLMs, including DeepSeek-VL2 wu2024deepseek , Qwen2.5-VL bai2025qwen2 and InternVL3 zhu2025internvl3 , have emerged, demonstrating performance