Less Is More: Training-Free Sparse Attention with Global Locality for Efficient Reasoning
Lijie Yang, Zhihao Zhang, Arti Jain, Shijie Cao, Baihong Yuan, Yiwei Chen, [0.2em] Zhihao Jia, Ravi Netravali, Princeton University · 2025
Read Less Is More: Training-Free Sparse Attention with Global Locality for Efficient Reasoning by Lijie Yang, Zhihao Zhang, Arti Jain, Shijie Cao, Baihong Yuan, Yiwei Chen, [0.2em] Zhihao Jia, Ravi Netravali, Princeton University (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2508-07101.txt.
Description
Lijie Yang and co-authors make reasoning models faster by watching fewer words at once, in a 2025 paper. Long reasoning chains cost time and memory. Sparse attention trims that cost by skipping less relevant tokens. Old trimming methods often hurt accuracy. Errors pile up over long outputs, or the model needs retraining. Their method, LessIsMore, needs no retraining.
The key insight is that key tokens stay stable. The same tokens matter across attention heads and across steps. So the method picks one shared token set for all heads. A window of recent tokens is always kept. The set is reused across layers for consistency. Across model families and hard reasoning tests, accuracy held or improved. Decoding sped up by as much as one point six times. The attention step itself ran up to one point seven times faster.
Catalog record
| CREATOR | Lijie Yang, Zhihao Zhang, Arti Jain, Shijie Cao, Baihong Yuan, Yiwei Chen, [0.2em] Zhihao Jia, Ravi Netravali, Princeton University |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 45 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2508-07101