Dynamic Differential Linear Attention: Enhancing Linear Diffusion Transformer for High-Quality Image Generation
Boyuan Cao 1 XingBo Yao 2 Chenhui Wang 1 Jiaxin Ye 1 Yujie Wei 1 Hongming Shan 1 1 Fudan University 2 Hong Kong University of Science and Technology (Guangzhou) {caoby23, chenhuiwang21, jxye22, yjwei22}@m.fudan.edu.cn, [email protected] [email protected] · 2026
Read Dynamic Differential Linear Attention: Enhancing Linear Diffusion Transformer for High-Quality Image Generation by Boyuan Cao 1 XingBo Yao 2 Chenhui Wang 1 Jiaxin Ye 1 Yujie Wei 1 Hongming Shan 1 1 Fudan University 2 Hong Kong University of Science and Technology (Guangzhou) {caoby23, chenhuiwang21, jxye22, yjwei22}@m.fudan.edu.cn, [email protected] [email protected] (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2601-13683.txt.
Description
Boyuan Cao and co-authors aim to cut the cost of making images with diffusion transformers in 2026. These models, called DiTs, make high-fidelity images that look sharp and true. Their self-attention step, though, grows in cost with the square of the input size. That quadratic cost blocks use at large scales.
The team builds a linear form of attention with a dynamic differential twist. Linear cost grows in a straight line, not a square, with size. Keeping image quality high while costs drop is the core test. Cheaper, finer image models would serve more users on less hardware.
Catalog record
| CREATOR | Boyuan Cao 1 XingBo Yao 2 Chenhui Wang 1 Jiaxin Ye 1 Yujie Wei 1 Hongming Shan 1 1 Fudan University 2 Hong Kong University of Science and Technology (Guangzhou) {caoby23, chenhuiwang21, jxye22, yjwei22}@m.fudan.edu.cn, [email protected] [email protected] |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 45 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2601-13683