Fast Autoregressive Video Diffusion and World Models with Temporal Cache Compression and Sparse Attention
Dvir Samuel, Issar Tzachor, Matan Levy, Micahel Green, Gal Chechik, Rami Ben-Ari · 2026
Read Fast Autoregressive Video Diffusion and World Models with Temporal Cache Compression and Sparse Attention by Dvir Samuel, Issar Tzachor, Matan Levy, Micahel Green, Gal Chechik, Rami Ben-Ari (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2602-01801.txt.
Description
Dvir Samuel and colleagues speed up video diffusion, whose attention layers bottleneck inference as the KV cache grows. They find three redundancies: duplicate keys across frames, slowly evolving queries, and cross attention over long prompts where few tokens matter per frame. Their FAST-AR combines TempCache compressing the cache by temporal correspondence and AnnCA picking frame tokens by nearest neighbor matching. AnnSA sparsifies self attention to semantically matched keys. Tests show five to tenfold end to end speedups with near identical visual quality, stable throughput and memory over long rollouts.
Catalog record
| CREATOR | Dvir Samuel, Issar Tzachor, Matan Levy, Micahel Green, Gal Chechik, Rami Ben-Ari |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 49144 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2602.01801