RefAM : Attention Magnets for Zero-Shot Referral Segmentation
Anna Kukleva, Enis Simsar, Alessio Tonioni, Muhammad Ferjad Naeem, Federico Tombari, Jan Eric Lenssen, Bernt Schiele · 2025
Read RefAM : Attention Magnets for Zero-Shot Referral Segmentation by Anna Kukleva, Enis Simsar, Alessio Tonioni, Muhammad Ferjad Naeem, Federico Tombari, Jan Eric Lenssen, Bernt Schiele (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2509-22650.txt.
Description
Anna Kukleva and 6 co-authors find that filler words in a prompt act like magnets for attention. Referring segmentation asks a model to outline the object a text phrase points to. Most methods fine-tune or chain several models. The team instead reuses attention scores inside diffusion transformers, with no training and no architecture change. Stop words soak up surplus attention, and filtering them cuts noise.
Deeper layers grow global attention sinks that blur the maps. Suppressing or redirecting those sinks sharpens the grounding. Appending stop words splits background activity into small clusters, which sharpens maps further. Their framework, RefAM, led training-free methods on image and video benchmarks.
Catalog record
| CREATOR | Anna Kukleva, Enis Simsar, Alessio Tonioni, Muhammad Ferjad Naeem, Federico Tombari, Jan Eric Lenssen, Bernt Schiele |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 54 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2509-22650