Density-aware Soft Context Compression with Semi-Dynamic Compression Ratio
Yijiong Yu, Shuai Yuan, Jie Zheng, Huazheng Wang, Ji Pei · 2026
Read Density-aware Soft Context Compression with Semi-Dynamic Compression Ratio by Yijiong Yu, Shuai Yuan, Jie Zheng, Huazheng Wang, Ji Pei (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2603-25926.txt.
Description
Yu and co-authors present a context compression method that adapts to how dense a text is. Soft compression squeezes long LLM context into fewer latent tokens. Fixed ratios waste effort, since real text varies in information density. Fully dynamic ratios prove hard for models to handle, the authors find. Their selector predicts a target ratio from density, then snaps it to a fixed menu of ratios. It trains jointly with the compressor on synthetic data. Across tests, the method beats static baselines, they report.
Catalog record
| CREATOR | Yijiong Yu, Shuai Yuan, Jie Zheng, Huazheng Wang, Ji Pei |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 29081 |
| LICENSE | http://creativecommons.org/licenses/by-sa/4.0/ |
| RIGHTS | CC BY SA |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2603.25926