Exploring ℓ 0 \ell_{0} Sparsification for Inference-free Sparse Retrievers

Xinjie Shen, Zhichao Geng, Yang Yang · 2025

Read Exploring ℓ 0 \ell_{0} Sparsification for Inference-free Sparse Retrievers by Xinjie Shen, Zhichao Geng, Yang Yang (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2504-14839.txt.

Introduction

Xinjie Shen, Zhichao Geng, and Yang Yang speed up sparse search tools that skip live query work in a 2025 paper. In this setup, files are coded once at index time and queries need no model run. Old tools thin the code with a rule built for paired coding, which fits this lopsided setup poorly.

The team tries a thinning route drawn from a count of live weights in the code. On a broad test set of new fields, their route tops all like tools and nears the best paired tools. They also chart the trade-off of hit rate and speed, to show use for real search work.

Catalog record

CREATORXinjie Shen, Zhichao Geng, Yang Yang
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE22 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2504-14839

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Exploring ℓ 0 \ell_{0} Sparsification for Inference-free Sparse Retrievers Xinjie Shen, Zhichao Geng, Yang Yang Abstract . With increasing demands for efficiency, information retrieval has developed a branch of sparse retrieval, further advancing towards inference-free retrieval where the documents are encoded during indexing time and there is no model-inference for queries. Existing sparse retrieval models rely on FLOPS regularization for sparsification, while this mechanism was originally designed for Siamese encoders, it is considered to be suboptimal in inference-free scenarios which is asymmetric. Previous attempts to adapt FLOPS for inference-free scenarios have been limited to rule-based methods, leaving the potential of sparsification approaches for inference-free retrieval models largely unexplored. In this paper, we explore ℓ 0 \ell_{0} inspired sparsification manner for inference-free retrievers. Through comprehensive out-of-domain evaluation on the BEIR benchmark, our method achieves state-of-the-art performance among inference-free sparse retrieval models and is comparable to leading Siamese sparse retrieval models. Furthermore, we provide insights into the trade-off between retrieval effectiveness and computational efficiency, demonstrating practical value for real-world applications. Conference: Proceedings of the 48th International ACM SIGIR Conference on Research and Development in Information Retrieval; July 13–18, 2025; Padua, Italy Proceedings of the 48th International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR ’25), July 13–18, 2025, Padua, Italy DOI: 10.1145/3726302.3730192 ISBN: 979-8-4007-1592-1/2025/07 CCS: Information systems Information retrieval CCS: Information systems Document representation † † cc-license: by 1. Introduction Information retrieval systems have evolved significantly over the past decades, with sparse retrieval remaining a cornerstone branch due to its efficiency, scalability, and reasonable search relevance ( Lassance and Clinchant, 2022 ) . While dense retrievers such as ColBERTv2 ( Santhanam et al., 2022 ) and Contriever ( Izacard et al., 2021 ) learn continuous semantic representations of queries and documents through neural networks, sparse retrievers focus on capturing lexical matches through token-based representations ( Dai and Callan, 2020a ; MacAvaney et al., 2020a ; Zhao et al., 2021 ) . To further enhance retrieval efficiency, inference-free retrieval methods ( Geng et al., 2024 ; Lassance et al., 2024 ; Nogueira et al., 2019 ; Dai and Callan, 2020b ) have emerged as a promising solution to the computational challenges in modern search systems ( Banon et al., 2025 ) . These approaches pre-compute document representations while degenerating query-time inference into lightweight methods such as term matching ( Robertson and Jones, 1976 ; Robertson and Zaragoza, 2009 ) or tokenization ( Formal et al., 2021b ; Formal et al., 2021a ; Formal et al., 2022a ; Lassance et al., 2024 ) . This design achieves superior efficiency while maintaining competitive performance. The efficiency of sparse retrieval models is largely determined by their sparsification capability. A key design element in Siamese sparse retriever models is the FLOPS (Floating Point Operations) regularization ( Paria et al., 2020 ) , which penalizes non-zero elements in sparse representations using their squared average. However, this approach is less suited for inference-free models due to their asymmetric architecture, which differs from the original Siamese design in models like SPLADE. Moreover, FLOPS continually penalizes the scale of token weights but pays less attention to tokens’ \ell_{0} . The encoded document length could lower to zero with larger weight on FLOPS, In this paper, we propose a combination of two novel approaches designed for sparse inference-free models: \ell_{0} mask FLOPS and \ell_{0} approximation activation. These techniques introduce selective sparsification by applying regularization only to document side representations exceeding desired sparsity thresholds, while allowing already-sparse representations to optimize unrestrictedly for the ranking objective. Through extensive out-of-domain experiments on BEIR, we demonstrate that we can achieve superior performance to representative dense retriever models and inference-free models, and become comparable with state-of-the-art sparse retriever models, while maintaining good computational efficiency. The key contributions of our work include: • Two novel sparsification techniques, \ell_{0} mask loss and \ell_{0} approximation activation, designed for inference-free retrievers. • Strong empirical results on BEIR benchmark, achieving comparable performance to state-of-the-art sparse retrievers. • Comprehensive analysis of efficiency-performance trade-offs, providing practical insights for real-world scenarios. 2. Related Work 2.1. Sparse Retrievers Compared to dense retrievers, like ColBERTv2 ( Santhanam et al., 2022 ) , Contriever ( Izacard et al., 2021 ) and TAS-B ( Hofstätter et al., 2021 ) , which learn continuous semantic representations of both queries and documents utilizing neural networks, sparse representations focus on capture lexical matches through discrete token-based representations ( Dai and Callan, 2020a ; Bai et al., 2020 ; MacAvaney et al., 2020a ; Zhao et al., 2021 ) . Specifically, sparse representations typically represent documents and queries as high-dimensional sparse vectors where each dimension corresponds to a specific term in the vocabulary, and the value indicates the importance of that term. The SPLADE series models ( Formal et al., 2021b ; Formal et al., 2021a ; Formal et al., 2022a ; Lassance et al., 2024 ) have achieved great success as representative sparse retriever models, utilizing BERT’s masked language model head and pooling to generate vocabulary-sized sparse representations. However, query encoding remains an efficiency bottleneck in online practice scenarios, requiring further development and variants for adaptation. 2.2. Inference-free Retrievers Inference-free retriever models, such as BM25 ( Robertson and Jones, 1976 ; Robertson and Zaragoza, 2009 ) , DEEPCT ( Dai and Callan, 2020b ) , Doc2Query ( Nogueira et al., 2019 ) , DeepImpact ( Mallia et al., 2021 ) , EPIC ( MacAvaney et al., 2020b ) , SPLADE-v3-Doc ( Lassance et al., 2024 ) and SPLADE-doc-distill ( Geng et al., 2024 ; Formal et al., 2021a ) , can directly retrieve relevant documents without requiring computationally expensive neural inference on the query side. These models typically rely on pre-computed document representations offline, and simple operations for obtaining query representations online, making them particularly attractive for real-world applications ( Busch et al., 2012 ; Dempster et al., 2018 ) where latency and computational resources are critical concerns due to huge query volumes. Unlike inference-required models that need to encode queries through neural networks during retrieval, inference-free retrievers use simple operations like term matching ( Robertson and Jones, 1976 ; Robertson and Zaragoza, 2009 ) or tokenization ( Formal et al., 2021b ; Formal et al., 2021a ; Formal et al., 2022a ; Lassance et al., 2024 ) to avoid huge computational costs, while offering lower but reasonable throughput and latency. 3. Preliminary Our work is built upon the representative sparse inference-free retriever, SPLADE-doc-distill . This model predicts token importance across the vocabulary space using BERT’s masked language model head. The model creates sparse 30,522-dimensional vectors using max pooling and ReLU activation. It operates asymmetrically, processing only documents. Moreover, we adopt the IDF-aware technique proposed by ( Geng et al., 2024 ) . For an input document with token weight ( w_{i,j} ) at position ( i ), the sparse representation of token j is: (1) w_{j}=\text{IDF}_{j}\cdot\max_{i}\log(1+\sigma(w_{i,j})), where \text{IDF}_{j} is the IDF value of token j and \sigma(\cdot) is the ReLU function. For query representation, it employs an inference-free approach to generate a Bag-of-Words-like sparse vector, where the weight w_{j} of token j in the query representation is set to 1 only if token j is present in the query; otherwise, w_{j} is set to 0. IDF values are

Read the complete text