Relevance to Utility: Process-Supervised Rewrite for RAG

Jaeyoung Kim, Jongho Kim, Seung-won Hwang, Seoho Song, Young-In Song, IPAI, Seoul National University, Naver Corp {jae.young, jongh97, seungwonh}@snu.ac.kr{song.seoho, song.youngin}@navercorp.com · 2025

Read Relevance to Utility: Process-Supervised Rewrite for RAG by Jaeyoung Kim, Jongho Kim, Seung-won Hwang, Seoho Song, Young-In Song, IPAI, Seoul National University, Naver Corp {jae.young, jongh97, seungwonh}@snu.ac.kr{song.seoho, song.youngin}@navercorp.com (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2509-15577.txt.

Introduction

Jaeyoung Kim leads a team that rewrites fetched text so a language model can use it better. Search backed systems fetch pages, then write answers from them. Fetched pages may be on topic yet lack what the answer needs. Past rewrite tools miss this point: they chase topic match, not true usefulness for the answer.

Their method, R2U, learns what usefulness means by watching rewriting and answering together. It scales that watch to build firm training signals. A rewrite is scored by how much it lifts the answer's quality. Those scores then tune the rewriter in two ways. On many open question sets, the method beats strong rivals. The gain comes from grading use, not just match.

Catalog record

CREATORJaeyoung Kim, Jongho Kim, Seung-won Hwang, Seoho Song, Young-In Song, IPAI, Seoul National University, Naver Corp {jae.young, jongh97, seungwonh}@snu.ac.kr{song.seoho, song.youngin}@navercorp.com
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE44 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2509-15577

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Relevance to Utility: Process-Supervised Rewrite for RAG Jaeyoung Kim, Jongho Kim, Seung-won Hwang, Seoho Song, Young-In Song, IPAI, Seoul National University, Naver Corp {jae.young, jongh97, seungwonh}@snu.ac.kr{song.seoho, song.youngin}@navercorp.com Abstract Retrieval-Augmented Generation systems often suffer from a gap between optimizing retrieval relevance and generative utility: retrieved documents may be topically relevant but still lack the content needed for effective reasoning during generation. While existing ‘‘bridge’’ modules attempt to rewrite the retrieved text for better generation, we show how they fail to capture true document utility. In this work, we propose R2U, with a key distinction of directly optimizing to maximize the probability of generating a correct answer through process supervision. As such direct observation is expensive, we also propose approximating an efficient distillation pipeline by scaling the supervision from LLMs, which helps the smaller rewriter model generalize better. We evaluate our method across multiple open-domain question-answering benchmarks. The empirical results demonstrate consistent improvements over strong bridging baselines. 1 1 1 Source code: https://anonymous.4open.science/r/R2U-D547/README.md . 1 Introduction Despite the impressive capabilities of large language models Achiam et al. 2023 ; Bai et al. 2023 , they often struggle with answering queries unseen from training. Retrieval-augmented generation (RAG) offers an effective solution by using a retriever to gather unseen yet relevant documents and a generator to synthesize them into a final response. Figure 1: Accuracy score gap of extractive vs abstractive answer types with different bridging models. For Naive RAG, the top-10 documents retrieved with Bing Search are passed to the generator. BGE-Reranker-Large Xiao et al. 2023 is used for the rewriter baseline. However, a persistent gap remains between retrieval relevance and generative utility: improving retriever performance does not necessarily yield more accurate answers Ke et al. 2024 . To bridge such a gap, bridge modules between retriever (R) and generation (G) have been proposed, which rerank or summarize the retrieved documents to better support generation. Examples include reranking methods Pradeep et al. 2023 ; Yang et al. 2025 ; Xiao et al. 2023 constrained to reorder documents without modifying their content, and query-focused summarization (QFS) Xu et al. 2023 ; Li et al. 2024 ; Yoon et al. 2024 ; Hwang et al. 2024 ; Edge et al. 2024 ; Chen et al. 2025 allowed to prune or enrich content selectively. We abstract all these as rewriting of transformations from retrieved documents d into d^{\prime} , considering q as well, towards optimizing document utility . Our key claim is that existing rewriting modules fail to capture true document utility, defined as its ability to support a generator for accurate answer generation . Existing work relies only on queries or retrieved documents as a proxy of such utility, but we argue that direct observation of answer generation is critical for capturing such utility. To illustrate, one well-known bias of search engines is ranking documents that lexically match queries high. Existing rewritings may improve extractive answers that can be directly taken from the documents, but may fail to help answer generation requiring reasoning beyond direct extraction. To demonstrate these limitations, we divided the MS MARCO dataset into extractive and abstractive answers, depending on whether the ground truth answer lexically matches within d or not. As shown in Figure 1 , the well-known rewriting module, BGE-Reranker Xiao et al. 2023 , boost extractive performance but harm performance in abstractive generation, revealing their limited generalization when deeper reasoning is required 2 2 2 Other baseline results are in Table 5 . . As direct observation is costly, existing work observes answer correctness Yoon et al. 2024 ; Hwang et al. 2024 , by rewriting first, then using answer correctness as a proxy observation. This risks spurious success, where the right answer is reached through flawed intermediate reasoning. In contrast, we propose a novel bridging with direct observation, denoted as r elevance to u tility (R2U). We observe that sufficiently robust LLMs, during reasoning, can capture the desired process of how documents should be rewritten to support the answer. With this process, we formally derive how the reasoning traces of robust LLMs approximate the ideal rewriter with true utility. However, such direct process observation is often too expensive at inference time. Therefore, we distill the direct observation of LLM, into a smaller language model (SLM) as a rewriter model. Beyond pushing inference cost to training time, this distillation enables more robust training of SLMs by scaling observations from LLMs, for robustifying SLM supervision. Specifically, in addition to observing LLM’s rewriting processes for imitation, we augment by reordering or omitting certain documents to help the SLM generalize better and mitigate positional bias. We further train the SLM with preference learning, aligning its rewriting toward the generator’s performance. We evaluate our approach on multiple question-answering tasks covering diverse query types. The results show that R2U improves the average F1 score by 8.8%pt over the naive RAG baseline and outperforms the best existing baselines by 7.8%pt, consistently enhancing performance across tasks. 2 Proposed Bridging Objective In this section, we first formalize the role of bridge modules in RAG system, outlining where existing approaches fall short (§ 2.1 ). We then present our formulation of the bridging objective and how our objective captures true document utility. (§ 2.2 –§ 3.2 ). 2.1 Motivation: Existing Bridging Models In RAG system, the generator \phi produces an answer a given query q and retrieved documents d as follows: a\sim P_{\phi}(a|q,d) (1) Previous bridging works attempted to rewrite the documents to improve their utility, typically by reordering the documents Pradeep et al. 2023 ; Xiao et al. 2023 , filtering out irrelevant contents Yoon et al. 2024 ; Li et al. 2024 , or reconstructing the inputs Chen et al. 2025 using q and d : a\sim P_{\phi}(a|q,d^{\prime}),~\textrm{where}~d^{\prime}\sim P_{\theta}(d^{\prime}|q,d) (2) Here, \theta denotes either a fine-tuned or off-the-shelf rewriting module, and d^{\prime} represents the rewritten version of the documents. We argue that such bridges, P_{\theta}(d^{\prime}|q,d) , cannot capture the true definition of document utility P_{\phi}(a\mid q,d^{\prime}) , which measures whether the rewritten document effectively supports the process of answer generation of a . 2.2 Proposed: Process Supervision The key objective of R2U is to learn a bridge model that rewrites documents to maximize their true utility , i.e., probability of generating a correct answer, formalized as max_{d^{\prime}}P(a|q,d^{\prime}) . However, this notion of true utility is inherently unobservable. To optimize it directly, one would need an ideal rewriter that samples from P_{\theta}(d^{\prime}|q,d,a) , i.e., a rewriter that conditions on the target answer a during rewriting. Training a model with direct access to a constitutes a shortcut that the rewriter could simply encode the answer instead of genuinely learning to support answer generation. Thus, the challenge is how to approximate this ideal but unobservable distribution using practically observable signals, without shortcuts. Our contribution is twofold. To address the above issue, we provide a derivation (§ 2.2.1 ) that shows how the ideal rewrite distribution, otherwise unobservable, can be grounded in observable quantities obtained from LLM reasoning traces toward the answer. A key prerequisite for this derivation is that rewritten documents d^{\prime} appear explicitly in these traces, allowing the distribution to be anchored in actual observations. Therefore, our second contribution is to demonstrate that sufficiently robust generators, e.g., 70B

Read the complete text