Density-aware Soft Context Compression with Semi-Dynamic Compression Ratio

Yijiong Yu, Shuai Yuan, Jie Zheng, Huazheng Wang, Ji Pei · 2026

Read Density-aware Soft Context Compression with Semi-Dynamic Compression Ratio by Yijiong Yu, Shuai Yuan, Jie Zheng, Huazheng Wang, Ji Pei (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2603-25926.txt.

Description

Yu and co-authors present a context compression method that adapts to how dense a text is. Soft compression squeezes long LLM context into fewer latent tokens. Fixed ratios waste effort, since real text varies in information density. Fully dynamic ratios prove hard for models to handle, the authors find. Their selector predicts a target ratio from density, then snaps it to a fixed menu of ratios. It trains jointly with the compressor on synthetic data. Across tests, the method beats static baselines, they report.

Catalog record

CREATORYijiong Yu, Shuai Yuan, Jie Zheng, Huazheng Wang, Ji Pei
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE29081
LICENSEhttp://creativecommons.org/licenses/by-sa/4.0/
RIGHTSCC BY SA
SOURCEarXiv

Source: https://arxiv.org/abs/2603.25926

Browse the full library · About the Tower of Babel