Untwisting RoPE: Frequency Control for Shared Attention in DiTs Journal: TOG
Aryan Mikaeili, Or Patashnik, Andrea Tagliasacchi, Daniel Cohen-Or, Ali Mahdavi-Amiri · 2026
Read Untwisting RoPE: Frequency Control for Shared Attention in DiTs Journal: TOG by Aryan Mikaeili, Or Patashnik, Andrea Tagliasacchi, Daniel Cohen-Or, Ali Mahdavi-Amiri (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2602-05013.txt.
Description
Style transfer with image generators has a stubborn failure mode: the model copies the reference picture instead of borrowing only its style. Aryan Mikaeili and 4 co-authors trace that failure to positional encoding in this 2026 graphics paper. Their focus is RoPE, the rotary scheme that tells transformer models where each token sits.
Analysis shows RoPE splits into frequency bands with different sensitivity to position. The high-frequency bands dominate attention, forcing the model to look at spatially matching spots in the reference image, which causes the copying. By selectively adjusting the bands, attention can follow meaning rather than exact position. Applied to modern diffusion transformers, the method restores stable shared attention and gives users control over the balance between style transfer and content copying.
Catalog record
| CREATOR | Aryan Mikaeili, Or Patashnik, Andrea Tagliasacchi, Daniel Cohen-Or, Ali Mahdavi-Amiri |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 43 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2602-05013