Training for Compositional Sensitivity Reduces Dense Retrieval Generalization

Radoslav Ralev, Aditeya Baral, Iliya Zhechev, Jen Agarwal & Srijith Rajamohan · 2026

Read Training for Compositional Sensitivity Reduces Dense Retrieval Generalization by Radoslav Ralev, Aditeya Baral, Iliya Zhechev, Jen Agarwal & Srijith Rajamohan (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2604-16351.txt.

Introduction

Ralev, Baral, Zhechev, Agarwal, and Rajamohan show that training for compositional sensitivity hurts dense retrieval. Dense retrieval squeezes texts into single vectors ranked by cosine similarity. Small edits like negation or role swaps flip meaning while keeping high similarity. Across four dual-encoder backbones, structure-targeted negatives cut zero-shot NanoBEIR retrieval. Drops run 8 to 9 percent on small backbones and up to 40 percent on medium ones, while pooled-space gaps improve only partly. Verifiers over token maps then separate structural near-misses better.

Catalog record

CREATORRadoslav Ralev, Aditeya Baral, Iliya Zhechev, Jen Agarwal & Srijith Rajamohan
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE25835
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2604.16351

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Training for Compositional Sensitivity Reduces Dense Retrieval Generalization Radoslav Ralev, Aditeya Baral, Iliya Zhechev, Jen Agarwal & Srijith Rajamohan Abstract Dense retrieval compresses texts into single embeddings ranked by cosine similarity. While efficient for recall, this interface is brittle for identity-level matching: minimal compositional edits (negation, role swaps) flip meaning yet retain high similarity. Motivated by geometric results for unit-sphere cosine spaces ( 9 ) , we test this retrieval-composition tension in text-only retrieval. Across four dual-encoder backbones, adding structure-targeted negatives consistently reduces zero-shot NanoBEIR retrieval (8--9% mean nDCG@10 drop on small backbones; up to 40% on medium ones), while only partially improving pooled-space separation. Treating pooled cosine as a recall interface, we then benchmark verifiers scoring token--token cosine maps. MaxSim (late interaction) excels at reranking but fails to reject structural near-misses, whereas a small Transformer over similarity maps reliably separates near-misses under end-to-end training. 1 1 1 Code and datasets are available at https://github.com/radoslavralev/limitations-text-retrieval 1 Introduction The dominant dual-encoder paradigm compresses texts into fixed vectors for efficient maximum inner product search (MIPS) retrieval ( 16 ; 10 ) . While effective for fuzzy topical matching, this architecture suffers a fundamental “resolution loss” regarding composition. Because the embedding function compresses variable-length reasoning into a single point, it often treats sentences as commutative bags-of-words, struggling to distinguish structural near-misses (e.g.,“the dog bit the man” vs. “the man bit the dog”) ( 19 ) . Recent theory suggests this is geometrically inevitable: 9 argue that unit-sphere cosine spaces force conceptual clusters into linear superposition, a geometry hostile to non-commutative structures like negation or order. This implies a retrieval–composition tension : forcing compositional sensitivity into a single vector degrades broad topical generalization. Contributions. We investigate this tension in text-only retrieval. We show that training with structure-targeted hard negatives creates a zero-sum game: the model rejects specific permutations but suffers significant degradation in out-of-domain retrieval (NanoBEIR). We argue that identity-sensitive matching should instead be treated as a distinct verification task. We benchmark lightweight verifiers on token–token similarity maps, finding that while MaxSim excels at relevance, true identity preservation requires learned verifiers that detect topological patterns in the map. 2 Single‑vector cosine is a bottleneck for identity Under unit-norm pooled embeddings and cosine scoring, a single inner product must simultaneously encode topical similarity and compositional distinctions. Previous work asserts that nontrivial content grouping pressures the representation toward (approximately) additive superposition ( 9 ) , which is commutative and tends to erase binding/order information. This predicts brittleness: there exist minimally edited near-misses (binding swaps, role reversals, scoped negation flips) that cannot be uniformly separated from paraphrases by a fixed cosine margin under the pooled-cosine bottleneck. We include the formal assumptions and an expanded statement in Appendix B . We adopt the standard two-stage setup. Stage 1: retrieve top- K candidates using ANN over pooled cosine keys. Stage 2: verify candidates using token interactions. Given token embeddings for query q and candidate c , we form the token similarity map M_{ij}(q,c)=\cos(q_{i},c_{j}) . A verifier F(q,c) consumes M (optionally with positional bias) and outputs a scalar used to rerank or gate candidates. We study a spectrum from simple reductions (global average; MaxSim/late interaction) to small learned pattern recognizers over M (tiny CNN / tiny Transformer). Full definitions (including alignment-biased variants and architectures) are in Appendix C . 3 Experiments Our analysis predicts a retrieval–composition tension for pooled-cosine dual encoders: allocating representational margin to reject meaning-changing near-misses can reduce the margin available for coarse content grouping. We test: (i) whether structure-targeted hard negatives degrade out-of-domain retrieval, and (ii) what verifier capacity is required to reject structural near-misses. For more information on dataset generation see Appendix D.1 . 3.1 Do composition-sensitive negatives hurt retrieval? We fine-tune dual encoders on NQ triplets using SentenceTransformers’ MultipleNegativesRankingLoss. We compare: Model A (baseline) trained on standard NQ supervision, and Model B (structured) trained on the mixed dataset described in § D.1 (standard + structural negatives). To compare across backbones under a fixed compute budget, we fix wall-clock training time per backbone and set steps based on measured throughput (details in Appendix). We evaluate zero-shot retrieval on NanoBEIR using nDCG@10 and Acc@1 (mean across datasets). Table 1 summarizes mean results across four backbones. Table 1: Mean NanoBEIR retrieval performance (nDCG@10 and Acc@1). Model A: standard fine-tuning. Model B: + structured negatives. nDCG@10 Acc@1 Backbone Model A Model B \Delta (% drop) Model A Model B \Delta (% drop) MiniLM-L6 0.439 \pm 0.000 0.401 \pm 0.001 -0.038 (-8.7%) 0.393 \pm 0.002 0.346 \pm 0.004 -0.047 (-12.0%) MiniLM-L12 0.467 \pm 0.001 0.424 \pm 0.005 -0.043 (-9.2%) 0.424 \pm 0.003 0.369 \pm 0.010 -0.055 (-13.0%) GTE-Small 0.481 \pm 0.002 0.442 \pm 0.006 -0.039 (-8.1%) 0.444 \pm 0.001 0.389 \pm 0.004 -0.055 (-12.4%) GTE-ModernBERT-base 0.543 \pm 0.001 0.324 \pm 0.018 -0.219 (-40.3%) 0.493 \pm 0.005 0.275 \pm 0.015 -0.218 (-44.2%) Results. Across all backbones and metrics, training with structural hard negatives ( Model B ) reduces NanoBEIR performance relative to the NQ -only baseline ( Model A ). On MiniLM-L6/L12 and gte-small , mean nDCG@10 drops by 8–9% and Acc@1 drops by 12–13%. On gte-modernbert-base , the drop is much larger (40% nDCG@10; 44% Acc@1). This supports the predicted tension: under a single pooled embedding with cosine scoring, allocating margin to reject lexically overlapping meaning-changes competes with broad topical grouping. Does the retrieval drop buy identity sensitivity in pooled space? To measure what compositional sensitivity is obtained within the pooled space , we plot cosine-similarity distributions between an original sentence s and a minimally perturbed near-miss \tilde{s} (negation, binding/order, spatial flips). Lower cosine is better: all perturbations are non-identical by construction. Fig. 1 overlays these distributions with 10k held-out NQ positives and negatives. Figure 1: Cosine-similarity distributions between an anchor sentence and a minimally edited near-miss under pooled embeddings. We compare Model A vs. Model B for three perturbation families (negation, binding/order, spatial) and overlay NQ positives/negatives for reference (10k pairs each). Lower is better for near-miss distributions. Two patterns stand out. First, NQ -only fine-tuning (Model A) leaves identity-breaking edits highly similar to the anchor: negation and binding remain near the positive regime, and spatial flips are nearly saturated. Second, introducing structural negatives (Model B) produces non-uniform improvements: while it significantly reduces similarity for negation and spatial flips, the gains for binding are less definitive. Despite a lower mean, binding lacks a distinct cluster to separate it from other categories. Thus, while structure-targeted negatives improve sensitivity for specific perturbation classes, they fail to establish a consistent identity margin in pooled cosine space, underscoring the continued necessity of token-interaction verification. Takeaway: structural negatives partially lower cosine for some edits but reliably hurt out‑of‑domain retrieval. 3.2 How small can the verifier be? We evaluate the verifier family \{F_{k}\} from § C.2 operating over token–token cosine maps M(q,c) . We compare: (i) Frozen encoder , where we train only the verifier, and (ii) End-to-end , where we train encoder and verifier jointly. All methods share the same stage-1 candidate generation via pooled cosine; only the stage-2 verifier differs. Evaluation 1: reranking on NanoBEIR. Fig. 2 reports NanoBEIR metrics after reranking the top- K candidates with each verifier. In the frozen regime, late interaction F_{1} (MaxSim) is the strongest and most consistent

Read the complete text