MonarchRT: Efficient Attention for Real-Time Video Generation

Krish Agarwal 1, Zhuoming Chen 1, Cheng Luo, Yongqi Chen 3, Haizhong Zheng 1, Xun Huang 3, Atri Rudra 2, Beidi Chen 1 · 2026

Read MonarchRT: Efficient Attention for Real-Time Video Generation by Krish Agarwal 1, Zhuoming Chen 1, Cheng Luo, Yongqi Chen 3, Haizhong Zheng 1, Xun Huang 3, Atri Rudra 2, Beidi Chen 1 (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2602-12271.txt.

Description

Real-time video generation with diffusion transformers chokes on the quadratic cost of three-dimensional self-attention, especially in few-step, autoregressive modes where errors build up. Krish Agarwal led this work with seven co-authors, finding that video attention is not reliably sparse. It mixes strong periodic structure from position with sparse semantic links and dense mixing, beyond even oracle top-k capacity.

Their answer, MonarchRT, factorises attention with Monarch matrices in aligned blocks, extended with a tiled form, keeping expressiveness while cutting cost. Fine-tuning and custom Triton kernels remove the overhead. On Self-Forcing, it reaches 95 percent attention sparsity with no quality loss, beating FlashAttention kernels by 1.4 to 11.8 times and reaching true real-time video at 16 frames per second on one RTX 5090.

Catalog record

CREATORKrish Agarwal 1, Zhuoming Chen 1, Cheng Luo, Yongqi Chen 3, Haizhong Zheng 1, Xun Huang 3, Atri Rudra 2, Beidi Chen 1
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE73 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2602-12271

Browse the full library · About the Tower of Babel