Towards Transparent RAG: Fostering Evidence Traceability in LLM Generation via Reinforcement Learning
Jingyi Ren, Yekun Xu, Xiaolong Wang, Weitao Li, Ante Wang, Weizhi Ma, Yang Liu · 2025
Read Towards Transparent RAG: Fostering Evidence Traceability in LLM Generation via Reinforcement Learning by Jingyi Ren, Yekun Xu, Xiaolong Wang, Weitao Li, Ante Wang, Weizhi Ma, Yang Liu (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2505-13258.txt.
Introduction
Jingyi Ren and co-authors introduced TRACE in 2025, a method to make language model answers show their evidence. Systems that search documents before answering often give replies with no clear path back to the sources. That hides errors and wastes the context that was retrieved.
TRACE trains models with reinforcement learning to give structured answers with explicit citations. Rewards cover evidence relevance, format, and accuracy, and a merging strategy keeps training steady across those signals. On three hard question sets, TRACE raised accuracy by ten to thirty percent and matched top commercial models. Tests on new tasks showed the skill carried over. Code was made public with the study.
Catalog record
| CREATOR | Jingyi Ren, Yekun Xu, Xiaolong Wang, Weitao Li, Ante Wang, Weizhi Ma, Yang Liu |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 46 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2505-13258
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Towards Transparent RAG: Fostering Evidence Traceability in LLM Generation via Reinforcement Learning Jingyi Ren, Yekun Xu, Xiaolong Wang, Weitao Li, Ante Wang, Weizhi Ma, Yang Liu Abstract . Retrieval-Augmented Generation (RAG) delivers substantial value in knowledge-intensive applications. However, its generated responses often lack transparent reasoning paths that trace back to source evidence from retrieved documents. This opacity not only compromises the interpretability of the output but also limits the model’s ability to fully exploit the provided context. To address this, we propose TRACE ( T ransparent RA G with eviden CE tracing), a framework designed to enhance evidence traceability in Large Language Models (LLMs) through reinforcement learning (RL). TRACE guides LLMs to produce structured outputs with explicit evidence citations by prompting and rewarding evidence relevance and proper formatting, alongside accuracy, to optimize structured traceability. To ensure training stability with multiple reward signals, we further introduce an adaptive strategy for merging rewards and adopt a stabilized KL-divergence estimator. Experiments on three multi-hop QA datasets using Qwen2.5-7B-Instruct and Llama-3.1-8B-Instruct show that TRACE achieves both transparent, evidence-attributed outputs and accuracy improvements of 10–30%. The resulting performance is comparable to advanced commercial LLMs (e.g., OpenAI o1, DeepSeek-R1). Further analyses demonstrate strong generalization capabilities to unseen tasks. Our code is publicly available now 1 1 1 https://github.com/ren258/TRACE . 1. Introduction Retrieval-Augmented Generation (RAG) ( Lewis et al., 2020 ) has become a powerful paradigm for enhancing large language models (LLMs) with non-parametric knowledge ( Gao et al., 2023b ; Gupta et al., 2024 ; Zhou et al., 2025 ) , particularly in knowledge-intensive domains such as medicine ( Li et al., 2024b ) , law ( Wiratunga et al., 2024 ) , and finance ( Setty et al., 2024 ) . Beyond accuracy gains from retrieved context ( Yu, 2022 ) , practical deployment further requires transparency, auditability, and decision traceability, so that one can verify which evidence supports which reasoning steps and the final answer ( Singh et al., 2024 ; Wei Jie et al., 2024 ) . Recent research emphasizes the active acquisition of information, often interleaving search and reasoning steps when initial documents are insufficient ( Gao et al., 2025 ; Li et al., 2025a ) . By dynamically expanding the context, these methods have achieved remarkable accuracy improvements on complex multi-hop QA tasks, particularly through Reinforcement Learning (RL) strategies like Group Relative Policy Optimization (GRPO) ( Shao et al., 2024 ; Jin et al., 2025 ) . However, despite their strong performance, most these approaches typically operate as black boxes regarding evidence utilization; they do not make explicit or verifiable which specific retrieved references are actually utilized during the intermediate reasoning that leads to the final answer ( Chen et al., 2025 ) . Another stream of work focuses on enhancing interpretability in a post-generation manner , either by appending citation tags to generated statements ( Nakano et al., 2021 ; Zhang et al., 2025 ) or establishing post-hoc links between generated content and retrieved materials ( Qian et al., 2025 ; Li et al., 2024c ) . While effective for auditability, these attribution-centric methods generally treat citation as a requirement for transparency rather than a mechanism for logic enforcement, leaving the potential of leveraging precise evidence selection to conversely enhance reasoning accuracy largely underexplored. In this work, we investigate how transparent attribution can enhance the performance of RAG generators given retrieved results. We propose TRACE ( T ransparent RA G with eviden CE tracing), a framework designed to unify reasoning accuracy and decision transparency via reinforcement learning. First, TRACE enforces a structured output protocol that explicitly includes the selected references, step-wise reasoning traces, and the final answer, rendering the entire evidence–reasoning–answer chain auditable. Second, to align the model with this objective, we implement task-specific reward functions encouraging correct answers, faithful citation usage, and well-formed decision traces; crucially, to promote holistic alignment, we introduce a bonus mechanism that assigns an amplified reward if and only if the response is perfect across all dimensions. Finally, we address the optimization challenges in constrained generation by substituting the default KL divergence estimator with a gradient-unbiased estimator. This modification mitigates the training instability issues identified in prior studies ( Tang and Munos, 2025 ; Yu et al., 2025 ) , ensuring robust convergence even under rigid formatting constraints. We implement TRACE on two representative open-source backbones: Qwen2.5-7B-Instruct ( Yang et al., 2024a ) and Llama3.1-8B-Instruct ( Grattafiori et al., 2024 ) . Extensive evaluations on three complex multi-hop QA benchmarks (HotpotQA ( Yang et al., 2018 ) , 2WikiMultiHopQA ( Ho et al., 2020 ) , and MuSiQue ( Trivedi et al., 2022 ) ), demonstrate that our method produces highly transparent outputs while delivering substantial accuracy improvements of 10–30% over backbones. Notably, our results indicate that open-source models trained with TRACE exhibit reasoning capabilities approaching those of advanced commercial models such as OpenAI o1 ( OpenAI, 2024b ) and DeepSeek-R1 ( Song et al., 2025a ) . Moreover, generalization experiments confirm that TRACE maintains superior performance across out-of-domain (OOD) test sets and diverse real-world retrieval scenarios, including both local and web-based settings. Finally, comprehensive ablation studies verify the necessity and effectiveness of each proposed component within our framework. Our main contributions are summarized as follows: • We propose TRACE , a reinforcement learning framework that synergizes evidence attribution with reasoning accuracy. We utilize RL to optimize the joint generation of evidence traces and answers, enabling the model to leverage grounding to enhance performance. • We design a customized optimization strategy specifically for RAG reasoning tasks, incorporating a holistic alignment bonus to resolve gradient conflicts and a gradient-unbiased KL estimator to mitigate the training instability often encountered in complex evidence processing. • We validate TRACE across three multi-hop benchmarks, demonstrating that it achieves 10–30% accuracy improvements, consistently surpassing all baselines of comparable scale and exhibiting comparable reasoning capabilities than top-tier commercial models. 2. Related Work 2.1. Retrieval-Augmented Generation for Complex Reasoning Retrieval-Augmented Generation (RAG) ( Lewis et al., 2020 ) fundamentally enhances LLMs by grounding generation in external evidence, thereby reducing parametric hallucinations and improving factual consistency on knowledge-intensive tasks ( Guu et al., 2020 ; Gao et al., 2023b ; Shuster et al., 2021 ; Chen et al., 2024 ) . However, applying RAG to complex multi-hop Question Answering (QA) tasks ( Yang et al., 2018 ; Ho et al., 2020 ; Trivedi et al., 2022 ) presents unique challenges, as these problems require synthesizing information from multiple disparate documents to derive the correct answer ( Press et al., 2023 ) . Unlike simple single-hop queries, multi-hop reasoning is highly sensitive to the quality of the retrieved context. As retrieval systems scale to include a larger number of top-ranked documents to ensure coverage, the inclusion of irrelevant or noisy context inevitably increases ( Petroni et al., 2020 ) . Prior studies indicate that such noise can severely mislead the generation process, causing models to hallucinate or misinterpret relationships between entities ( Creswell et al., 2023 ) . Furthermore, when faced with long and noisy contexts, LLMs often exhibit the “lost-in-the-middle” ( Liu et al., 2024 ) phenomenon or simply ignore the retrieved evidence in favor of their internal priors ( Yoran et al., 2024