When Does Content-Based Routing Work? Representation Requirements for Selective Attention in Hybrid Sequence Models
Abhinaba Basu · 2026
Read When Does Content-Based Routing Work? Representation Requirements for Selective Attention in Hybrid Sequence Models by Abhinaba Basu (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2603-20997.txt.
Description
Basu asks when content-based routing can work in hybrid sequence models. Routing picks which tokens deserve costly attention, but he argues it needs pairwise token comparison, and that need cannot be escaped. Across more than twenty experiments and fifteen routing methods, every system with high routing precision uses pairwise comparison. Methods that skip it, such as recurrent models, memory banks, and bandits, fail badly. Two ingredients are required: per-token representations with bidirectional context, plus pairwise comparison. One design reaches near perfect routing at linear cost.
Catalog record
| CREATOR | Abhinaba Basu |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 30239 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2603.20997