Effective Inference-Free Retrieval for Learned Sparse Representations Conference: ; July 13–18, 2025; Padua, Italy Conference: Proceedings of the 48th International ACM SIGIR Conference on Research and Development in Information Retrieval; July 13–18, 2025; Padua, Italy Proceedings of the 48th Inter

Franco Maria Nardini, Thong Nguyen, Cosimo Rulli, Rossano Venturini, Andrew Yates · 2025

Read Effective Inference-Free Retrieval for Learned Sparse Representations Conference: ; July 13–18, 2025; Padua, Italy Conference: Proceedings of the 48th International ACM SIGIR Conference on Research and Development in Information Retrieval; July 13–18, 2025; Padua, Italy Proceedings of the 48th Inter by Franco Maria Nardini, Thong Nguyen, Cosimo Rulli, Rossano Venturini, Andrew Yates (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2505-01452.txt.

Introduction

Franco Maria Nardini and colleagues remove the neural network from the query path of learned sparse search, replacing it with a table lookup. Learned sparse retrieval encodes text as weighted word bags and answers from inverted indexes. Training usually regularizes hard for sparsity, but newer engines changed the trade-off. Writing in twenty twenty-five, the team first shows regularization can be relaxed, yielding stronger encoders. The bottleneck then moves: encoding each query with a network now dominates the cost. Their method learns a fixed score for every token at training time, so query encoding becomes a seamless lookup.

The result set state-of-the-art effectiveness in their tests, both in-domain and out-of-domain. Gains over a strong baseline appeared on two standard benchmarks. Speed and quality, usually traded against each other, improved together.

Catalog record

CREATORFranco Maria Nardini, Thong Nguyen, Cosimo Rulli, Rossano Venturini, Andrew Yates
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE25 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2505-01452

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Effective Inference-Free Retrieval for Learned Sparse Representations Conference: ; July 13–18, 2025; Padua, Italy Conference: Proceedings of the 48th International ACM SIGIR Conference on Research and Development in Information Retrieval; July 13–18, 2025; Padua, Italy Proceedings of the 48th International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR ’25), July 13–18, 2025, Padua, Italy DOI: 10.1145/3726302.3730185 ISBN: 979-8-4007-1592-1/2025/07 CCS: Information systems Retrieval models and ranking Franco Maria Nardini, Thong Nguyen, Cosimo Rulli, Rossano Venturini, Andrew Yates Abstract . Learned Sparse Retrieval (LSR) is an effective IR approach that exploits pre-trained language models for encoding text into a learned bag of words. Several efforts in the literature have shown that sparsity is key to enabling a good trade-off between the efficiency and effectiveness of the query processor. To induce the right degree of sparsity, researchers typically use regularization techniques when training LSR models. Recently, new efficient—inverted index-based—retrieval engines have been proposed, leading to a natural question: has the role of regularization changed in training LSR models? In this paper, we conduct an extended evaluation of regularization approaches for LSR where we discuss their effectiveness, efficiency, and out-of-domain generalization capabilities. We first show that regularization can be relaxed to produce more effective LSR encoders. We also show that query encoding is now the bottleneck limiting the overall query processor performance. To remove this bottleneck, we advance the state-of-the-art of inference-free LSR by proposing Learned Inference-free Retrieval ( Li-Lsr ). At training time, Li-Lsr learns a score for each token, casting the query encoding step into a seamless table lookup. Our approach yields state-of-the-art effectiveness for both in-domain and out-of-domain evaluation, surpassing Splade -v3-Doc by 1 1 point of mRR@10 on MsMarco and 1.8 1.8 points of nDCG@10 on Beir . † † cc-license: by 1. Introduction Pretrained Language Models (PLM) have impacted IR in many ways ( Lin et al. 2021 ) . Indeed, the possibility of producing vector representations, i.e., embeddings , of the input text by grasping semantic and contextual information is one of the most important ( Karpukhin et al. 2020 ; Xiong et al. 2021 ; Khattab and Zaharia 2020 ) . Learning representations allows the problem of retrieving relevant documents for a query to be cast into the problem of finding the closest document embeddings w.r.t. the input query embedding according to a specific similarity metric, which is often dot product or cosine. Learned sparse representations (LSR) ( MacAvaney et al. 2020 ; Formal et al. 2021b ; Formal et al. 2021a ; Formal et al. 2022 ; Lassance and Clinchant 2022 ; Lassance et al. 2024 ; Zhao et al. 2021 ) map queries and documents into n -dimensional vectors where dimensions are grounded on the vocabulary of the collection. When a coordinate is nonzero in a sparse embedding, that indicates that the corresponding term is semantically relevant to the input. LSR models have shown to be competitive with dense retrieval models ( Lin et al. 2021 ; Karpukhin et al. 2020 ; Xiong et al. 2021 ; Reimers and Gurevych 2019 ; Khattab and Zaharia 2020 ; Santhanam et al. 2022 ; Daliri et al. 2023 ) and to effectively generalize to out-of-domain datasets ( Bruch et al. 2023 ; Lassance et al. 2024 ) , while being interpretable by design. Moreover, sparse embeddings retain some of the benefits of classical lexical models such as BM25 ( Robertson et al. 1994 ) , including the possibility to use retrieval machinery based on inverted indexes. Despite the natural link that can be drawn between learned sparse representations and inverted indexes, their matching has only recently been fully realized. In fact, learned sparse representations show important differences w.r.t. normal query/document text, such as longer queries, and the non-Zipfian distribution of terms ( Mackenzie et al. 2021 ) , which results in increased per-query latency. Several efforts have investigated how this performance gap can be closed when using inverted index-based methods. In this line, Bruch et al. recently introduced Seismic ( Bruch et al. 2024a ; Bruch et al. 2024b ; Bruch et al. 2025 ) , a novel inverted index-based approximate nearest neighbors retrieval, which is shown to outperform state-of-the-art approaches for retrieval with learned sparse representations. Seismic is inherently built to work with Learned Sparse Representations and its properties, such as longer queries, longer documents, and non-Zipfian distribution. In this paper, we explore the role of term expansion in Learned Sparse Retrieval, by proposing an extended evaluation of regularization approaches for LSR on both in- and out-of-domain benchmarks. We show that regularization can be relaxed in LSR, producing more effective sparse embeddings while still allowing for reduced retrieval times, thanks to modern retrieval engines. In this setting, query encoding can be a significant bottleneck on CPU, limiting the performance of the query processor. We thus advance the state of the art by introducing Learned Inference-free Retrieval ( Li-Lsr ), a new technique that learns a score for each token, casting the query encoding step to a fast table lookup operation. In summary, the contributions of this paper are the following: • We present an in-depth evaluation of regularization techniques for LSR, showing its pivotal role in in- and out-of-domain effectiveness. In particular, our setup shows how relaxing regularization improves retrieval quality in both scenarios. • We evaluate the consequences of relaxing regularization from an efficiency perspective. The analysis shows that when leveraging modern retrieval engines, even longer (i.e., less sparse) representations can be retrieved efficiently. • We propose Learned Inference-free Retrieval ( Li-Lsr ), a novel inference-free approach that allows the replacement of the query encoder with a fast lookup table. Li-Lsr learns the optimal score for each term at training time, enabling both effective and efficient query encoding. Li-Lsr outperforms state-of-the-art inference-free retrievers such as Splade -v3-Doc, with a margin of 1 point of mRR on MsMarco and 1.8 points of nDCG@10 on Beir . 2. Methodology In this section, we define the methodology used to assess the impact of term expansion in modern LSR. Moreover, we introduce Li-Lsr , a novel approach to the inference-free paradigm that learns the importance score of each term at training time. 2.1. Exploring Term Expansion in LSR Unlike naïve term-matching engines, which exploit terms that appear in the input text, learned sparse representations are trained to activate relevant terms regardless of whether they appear in the input tokens. In fact, LSR can encompass term expansion on both the query and the document, i.e., adding terms to the representation to improve its generalization power. Term expansion in LSR encoders is enabled and controlled by regularization terms applied to the loss function, which control the sparsity in the representations, thus enabling a practical—yet effective—use of the obtained embeddings. In this section, we define a methodology that aims to assess the effectiveness of LSR as a function of the regularization applied during training. Equation 1 introduces a general loss function \mathcal{L} used for learning sparse representations. (1) \mathcal{L}\ =\ \mathcal{L}_{\text{rank}}+\lambda_{q}\,\mathcal{L}^{Q}_{\text{reg}}\ +\ \lambda_{d}\,\mathcal{L}^{D}_{\text{reg}} In state-of-the-art methods, \mathcal{L}_{\text{rank}} is typically a distillation loss ( Formal et al. 2022 ; Lassance et al. 2024 ; Lassance and Clinchant 2022 ) , encouraging the student model to align its scores with a

Read the complete text