ReAttn: Improving Attention-based Re-ranking via Attention Re-weighting

Yuxing Tian, Fengran Mo, Weixu Zhang, Yiyan Qi, Jian-Yun Nie · 2026

Read ReAttn: Improving Attention-based Re-ranking via Attention Re-weighting by Yuxing Tian, Fengran Mo, Weixu Zhang, Yiyan Qi, Jian-Yun Nie (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2602-19969.txt.

Description

Attention-based re-ranking scores documents from attention weights, offering an efficient, readable alternative to text-generating ranking. A 2026 arXiv paper by Yuxing Tian and four co-authors shows two flaws in that approach and proposes a fix called ReAttn.

First, attention concentrates on a few tokens in a few documents, leaving others hard to tell apart. Second, attention overweights phrases that merely resemble the query, so word-level lookalikes that are irrelevant rank too highly. ReAttn first weights attention by cross-document inverse frequency, lowering the weight on query-overlapping tokens that recur across candidates and easing word-level bias. It then applies entropy-based smoothing to spread attention more evenly across informative tokens. Both adjustments act on existing weights with no extra training, and experiments confirm the method works.

Catalog record

CREATORYuxing Tian, Fengran Mo, Weixu Zhang, Yiyan Qi, Jian-Yun Nie
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE40 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2602-19969

Browse the full library · About the Tower of Babel