Double-P: Hierarchical Top-P Sparse Attention for Long-Context LLMs

Wentao Ni, Kangqi Zhang, Zhongming Yu, Oren Nelson, Mingu Lee, Hong Cai, Fatih Porikli, Jongryool Kim, Zhijian Liu, Jishen Zhao · 2026

Read Double-P: Hierarchical Top-P Sparse Attention for Long-Context LLMs by Wentao Ni, Kangqi Zhang, Zhongming Yu, Oren Nelson, Mingu Lee, Hong Cai, Fatih Porikli, Jongryool Kim, Zhijian Liu, Jishen Zhao (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2602-05191.txt.

Description

Long documents make attention expensive, because every new token looks back at a growing cache. Wentao Ni and 9 co-authors present Double-P in this 2026 machine learning paper, a sparse attention scheme that keeps only the tokens carrying most of the attention mass. Fixed-size top-k budgets cannot adapt to heads and layers whose attention spreads very differently.

Double-P works in two stages. A coarse pass estimates the important mass at cluster level, using size-weighted cluster centers, and a second pass spends token-level computation only where the estimate says it is needed. That hierarchy jointly trims selection overhead and the sparse attention computation itself. Across long-context benchmarks, accuracy stayed within a hair of full attention while attention overhead fell by up to about 1.8 times and end-to-end decoding ran up to about 1.3 times faster than leading fixed-budget sparse methods.

Catalog record

CREATORWentao Ni, Kangqi Zhang, Zhongming Yu, Oren Nelson, Mingu Lee, Hong Cai, Fatih Porikli, Jongryool Kim, Zhijian Liu, Jishen Zhao
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE38 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2602-05191

Browse the full library · About the Tower of Babel