Latent Terms: Dense Retrievers Contain Trivially Extractable BM25-ready Zipfian Vocabularies
Benjamin Clavié, Sean Lee, Aamir Shakir, Makoto P. Kato · 2026
Read Latent Terms: Dense Retrievers Contain Trivially Extractable BM25-ready Zipfian Vocabularies by Benjamin Clavié, Sean Lee, Aamir Shakir, Makoto P. Kato (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-29384.txt.
Description
Benjamin Clavié presents Latent Terms, showing that retrievers learn forms that split into sparse parts. Trained on frozen retrievers, sparse autoencoders with no tweaks for retrieval extract a vocab with Zipfian stats. That vocab is ready for sparse retrieval scoring via BM25. This enables sparse retrieval with no learned expansion goal or sparse supervision, and applies to any retriever. Latent Terms matches or beats its base model scoring and SPLADE variants. It beats its base model by a lot on LIMIT, a task to highlight failures.
Catalog record
| CREATOR | Benjamin Clavié, Sean Lee, Aamir Shakir, Makoto P. Kato |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 42292 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2605.29384