Paraphrase Brittleness in Production Retrieval-Augmented Commercial Recommendation: Reproducibility Below the Rerun-Stability Baseline
Will Jack, Noah Lehman, Keller Maloney, Sarah Xu · 2026
Read Paraphrase Brittleness in Production Retrieval-Augmented Commercial Recommendation: Reproducibility Below the Rerun-Stability Baseline by Will Jack, Noah Lehman, Keller Maloney, Sarah Xu (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-27440.txt.
Introduction
Will Jack, Noah Lehman, Keller Maloney, and Sarah Xu show buyer phrasing drives brand recommendations more than buyer intent. Across thousands of paraphrase and rerun trials, paraphrase pairs overlap only 14 to 29 percent, far below the 50 to 61 percent same-prompt baseline. More reasoning effort barely narrows the gap, so tracking brand mentions prompt by prompt is structurally unstable.
Catalog record
| CREATOR | Will Jack, Noah Lehman, Keller Maloney, Sarah Xu |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 38208 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2605.27440
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Paraphrase Brittleness in Production Retrieval-Augmented Commercial Recommendation: Reproducibility Below the Rerun-Stability Baseline Will Jack, Noah Lehman, Keller Maloney, Sarah Xu Abstract Small changes to how a buyer phrases a question — “best CRM” vs “top CRM” vs “best CRM for a SaaS startup” — produce substantially different brand recommendations from AI assistants. Across ∼ \sim 6,000 paraphrase runs and ∼ \sim 6,000 same-prompt rerun controls on OpenAI and Anthropic models, the recommendation-set similarity (Jaccard) between two paraphrases of the same underlying buying intent is 0.288 for cosmetic rewordings (clustered 95% CI [0.215, 0.361]) and 0.135 for constraint-adding rewordings ([0.098, 0.175], pooling region/language and specificity-ladder axes) — both far below the 0.50–0.61 same-prompt rerun baseline. The prompt string, not the underlying buyer intent, is the dominant input to which brands surface. Increasing reasoning effort does not narrow the gap (bounded by ± 0.05 \pm 0.05 ). This is a direct challenge to an increasingly popular AEO/GEO practice. Tracking a brand’s “AI visibility” by counting brand mentions over a fixed set of prompts produces a metric whose dominant source of variance is which paraphrase the tracker happens to issue, not the model’s behavior toward the brand: the same buyer intent in two natural paraphrases produces recommendation sets that overlap 14–29% in Jaccard versus 50–61% for same-prompt reruns. Sampling more paraphrases per intent reduces the artifact in principle, and efficient multi-prompt evaluation methods exist in the academic literature, but the natural buyer-phrasing space is much larger than the benchmark-scale prompt sets those methods have been validated on, and far beyond what any commercial tracker issues per brand-intent combination. Prompt-by-prompt mention tracking is therefore structurally unstable as a unit of measurement; meaningful improvement likely requires a different unit rather than a larger prompt set. 1 Introduction A growing class of commercial monitoring tools (Profound, Otterly, LLM Pulse, HubSpot AEO Grader, Authoritas, Semrush, and others) report brand share-of-voice in AI assistants by issuing a small, fixed set of tracking prompts at regular intervals and counting brand mentions in the model’s response. These tools are the most visible products of an AEO / GEO industry that frames “marketing to AI” as analogous to search-engine optimization — the brand-side problem reduced to a discoverability metric tracked on a small fixed set of category queries. Customers buy these tools to answer a single question: is my brand’s AI-mediated visibility going up or down? The product-design assumption is that the tracking prompt is a label for an underlying buying intent — “best CRM software” stands in for the cohort of buyers about to ask about a CRM — and that the measurement is meaningful insofar as the prompt is a representative member of the intent’s natural paraphrase distribution. Under this assumption, week-over-week movement on the metric reflects either (i) a real shift in the model’s recommendation distribution or (ii) bounded run-to-run sampling noise. We test that assumption empirically. It fails, and the failure mode points at a deeper conceptual error in the AEO/GEO framing: AI assistants are recommendation engines, and buyer phrasing distributes naturally, so tracking against any one phrasing string is a brittle proxy for “how AI thinks about my brand.” We test the assumption empirically by separately measuring (a) the rerun-stability baseline — the Jaccard similarity between two consensus-recommendation sets produced by the same prompt run twice within a single day at N{=}30 — and (b) the paraphrase brittleness — the Jaccard similarity between consensus-recommendation sets produced by two natural-language paraphrases of the same underlying intent. The gap between (a) and (b) is the signal-to-noise ratio of the monitoring methodology: if paraphrase Jaccard sits well below rerun Jaccard, then variation in the tracking metric is dominated by prompt-choice artifact rather than by run-to-run noise, and weekly trend movements are unidentifiable from the tracking prompt itself. A natural expectation from the LLM reasoning literature would predict that higher reasoning effort closes the gap. Guo et al. 2025 reports steep accuracy gains on the DeepSeek-R1 family as the thinking-token budget grows on math/competition tasks. Wang et al. 2023 ’s self-consistency, while mechanistically different (multi-sample majority voting at inference, not single-sample effort scaling), establishes the broader prior that more reasoning reduces variance on chain-of-thought tasks. If those gains extrapolate from single-correct-answer reasoning to multi-plausible-answer commercial recommendation, increasing reasoning effort should reduce variance there too. We test this by running each model at low and high single-sample reasoning effort across both the rerun-stability and the paraphrase-brittleness conditions, and find that the extrapolation fails: reasoning effort produces no meaningful effect on either reproducibility measure. 1.1 Contributions This paper makes three contributions. 1. Calibrated rerun-stability baseline for commercial recommendation. Across 50 prompts \times 4 cells \times N{=}30 same-prompt reruns within a single day (6,000 runs), we measure within-cell consensus-recommendation Jaccard at 0.50–0.61 and within-cell retrieval-pool Jaccard at 0.40–0.74. The baseline anchors the magnitude of “real” effects in any companion measurement. 2. Paraphrase-brittleness measurement against the baseline. Across five paraphrase axes (synonym swap, structural rewrite, modifier substitution, region/language, specificity ladder), \sim 20 base prompts each carrying \sim 5 paraphrase variants (one variant per axis on average) \times 3 cells \times N{=}20 ( \approx 6,000 runs total), we measure cross-paraphrase consensus-recommendation Jaccard at 0.288 (clustered 95% CI [0.215, 0.361]) for cosmetic rewordings and 0.135 (clustered 95% CI [0.098, 0.175]) for the constraint-adding pool, with the specificity-ladder subset at 0.133 [0.090, 0.185] and the region/language subset at 0.138 [0.080, 0.201]. The cosmetic-paraphrase number sits 21–32 percentage points below the rerun baseline; the constraint-adding number sits 37–48 percentage points below. Even the upper bound of the cosmetic CI (0.361) is 14 pp below the lower bound of the rerun baseline (0.50); the brittleness claim survives clustered uncertainty. 3. Single-sample reasoning effort does not reduce variance in commercial recommendation. The effect of low \to high single-sample reasoning effort on rerun stability is between -0.015 and +0.005 Jaccard; on paraphrase robustness it is bounded by \pm 0.05 . Both effects are small relative to the rerun-vs-paraphrase gap and, in the case of rerun stability, are not consistently positive. The broader prior from inference-time scaling [ 9 ] does not transfer from single-correct-answer reasoning tasks to multi-plausible-answer commercial recommendation, consistent with Meincke et al. 2025 ’s finding that chain-of-thought returns are flattening on frontier instruction-tuned models. 2 Background Prompt sensitivity in LLM outputs. Sclar et al. 2024 demonstrated that cosmetic formatting changes alone can swing LLaMA-2-13B accuracy by up to 76 points on standard benchmarks, an effect orders of magnitude larger than typical run-to-run sampling. Mizrahi et al. 2024 , in a 6.5M-instance multi-prompt study, showed that instruction paraphrases can reorder model rankings on benchmark leaderboards; their conclusion is that single-prompt evaluation is structurally brittle. Gan and Ng 2019 established the pre-LLM baseline: SQuAD-trained QA models drop sharply when test-time questions are paraphrased. Chatterjee et al. 2024 introduced POSIX, a quantitative prompt-sensitivity index spanning intent-preserving paraphrases, character perturbations, and spelling errors, situating prompt-sensitivity measurement on a common footing across tasks. LLM reproducibility under fixed prompts. Atil et al. 2024 documented that commercial LLM APIs produce non-trivial divergence across repeated identical prompts even at temperature zero. Yuan et al. 2025 traced residual nondeterminism to floating-point non-associativity and BF16 precision effects in transformer inference, establishing a numerical floor below which determinism is