Dynamic Superblock Pruning for Fast Learned Sparse Retrieval Conference: Proceedings of the 48th International ACM SIGIR Conference on Research and Development in Information Retrieval; July 13–18, 2025; Padua, Italy Proceedings of the 48th International ACM SIGIR Conference on Research and Developm

Parker Carlson, Wentai Xie, Shanxiu He, Tao Yang · 2025

Read Dynamic Superblock Pruning for Fast Learned Sparse Retrieval Conference: Proceedings of the 48th International ACM SIGIR Conference on Research and Development in Information Retrieval; July 13–18, 2025; Padua, Italy Proceedings of the 48th International ACM SIGIR Conference on Research and Developm by Parker Carlson, Wentai Xie, Shanxiu He, Tao Yang (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2504-17045.txt.

Description

Parker Carlson, Wentai Xie, Shanxiu He, and Tao Yang present a faster search method in a 2025 paper. Learned sparse search scores each text by a set of weighted terms, which can be slow on large sets. Their method groups texts into blocks, then groups blocks into superblocks for a quick first pass.

The first pass can skip whole superblocks that cannot reach the top results. This builds on older block and cluster tricks but acts earlier, at a larger scale. The method can run in a safe mode or a fast near-exact mode. Tests on a standard passage set beat the best prior methods by a clear margin on a single CPU thread.

Catalog record

CREATORParker Carlson, Wentai Xie, Shanxiu He, Tao Yang
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE26 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2504-17045

Browse the full library · About the Tower of Babel