Untwisting RoPE: Frequency Control for Shared Attention in DiTs Journal: TOG

Aryan Mikaeili, Or Patashnik, Andrea Tagliasacchi, Daniel Cohen-Or, Ali Mahdavi-Amiri · 2026

Read Untwisting RoPE: Frequency Control for Shared Attention in DiTs Journal: TOG by Aryan Mikaeili, Or Patashnik, Andrea Tagliasacchi, Daniel Cohen-Or, Ali Mahdavi-Amiri (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2602-05013.txt.

Description

Style transfer with image generators has a stubborn failure mode: the model copies the reference picture instead of borrowing only its style. Aryan Mikaeili and 4 co-authors trace that failure to positional encoding in this 2026 graphics paper. Their focus is RoPE, the rotary scheme that tells transformer models where each token sits.

Analysis shows RoPE splits into frequency bands with different sensitivity to position. The high-frequency bands dominate attention, forcing the model to look at spatially matching spots in the reference image, which causes the copying. By selectively adjusting the bands, attention can follow meaning rather than exact position. Applied to modern diffusion transformers, the method restores stable shared attention and gives users control over the balance between style transfer and content copying.

Catalog record

CREATORAryan Mikaeili, Or Patashnik, Andrea Tagliasacchi, Daniel Cohen-Or, Ali Mahdavi-Amiri
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE43 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2602-05013

Browse the full library · About the Tower of Babel