Incorporating Q&A Nuggets into Retrieval-Augmented Generation
Laura Dietz ✉, Bryan Li, Gabrielle Liu, Jia-Huei Ju, Eugene Yang, Dawn Lawrie, William Walden, James Mayfield · 2026
Read Incorporating Q&A Nuggets into Retrieval-Augmented Generation by Laura Dietz ✉, Bryan Li, Gabrielle Liu, Jia-Huei Ju, Eugene Yang, Dawn Lawrie, William Walden, James Mayfield (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2601-13222.txt.
Introduction
Laura Dietz and co-authors fold judged facts into text that AI writes from sources. Their line of work joins auto judging into retrieval augmented generation, where a model writes with docs in hand. The aim is text that can be checked, fact by fact.
Their system is called Crucible. It builds a bank of question and answer bits, called nuggets, from the docs it pulls. Each nugget keeps a clear link to its source. The bank then guides how text is drawn out, picked, and put into a report. For teams that need sourced text, the route keeps claims tied to proof as the text is built, not just checked at the end.
Catalog record
| CREATOR | Laura Dietz ✉, Bryan Li, Gabrielle Liu, Jia-Huei Ju, Eugene Yang, Dawn Lawrie, William Walden, James Mayfield |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 18 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2601-13222
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Incorporating Q&A Nuggets into Retrieval-Augmented Generation Laura Dietz ✉, Bryan Li, Gabrielle Liu, Jia-Huei Ju, Eugene Yang, Dawn Lawrie, William Walden, James Mayfield Abstract RAGE systems integrate ideas from automatic evaluation (E) into Retrieval-augmented Generation (RAG). As one such example, we present Crucible , a Nugget-Augmented Generation System that preserves explicit citation provenance by constructing a bank of Q&A nuggets from retrieved documents and uses them to guide extraction, selection, and report generation. Reasoning on nuggets avoids repeated information through clear and interpretable Q&A semantics—instead of opaque cluster abstractions—while maintaining citation provenance throughout the entire generation process. Evaluated on the TREC NeuCLIR 2024 collection, our Crucible system substantially outperforms Ginger , a recent nugget-based RAG system, in nugget recall, density, and citation grounding. 1 1 1 Appendix at https://github.com/hltcoe/ecir26-crucible-system-appendix/ 1 Introduction Retrieval-Augmented Generation (RAG) has become the dominant framework for grounding LLM outputs in evidence [ 16 , 13 ] . At the same time, nugget-based evaluation methods have emerged as the standard for measuring relevance of long-form responses [ 18 , 10 ] . A nugget , i.e., short Q&A pair, fact, or claim, is a fine-grained reusable unit for assessing whether key information is covered. By encoding the information that must appear in a system answer, nuggets enable evaluation metrics such as “nugget recall” that directly quantify the amount of useful information given by the information system. We argue that nuggets are valuable not only for evaluation but also for guiding retrieval and generation, especially given that LLMs can produce high‑quality nuggets automatically [ 6 , 9 ] . Contributions. To support this argument, we present Crucible , a nugget-centric RAG system that automatically constructs its own nugget bank and uses it as a control signal throughout the pipeline. For example, nuggets form the basis for controlling redundancy without content-based clustering, while naturally preserving citation provenance. On the TREC NeuCLIR 2024 test collection, we provide a direct comparison to a recent nugget-oriented RAG system Ginger [ 14 ] . We find that Crucible decisively outperforms Ginger across several evaluation metrics, from nugget recall to citation support. 2 Related Work Summarize one‑document‑at‑a‑time. Early RAG systems either produce a summary per retrieved passage [ 16 ] or encode each passage separately [ 13 ] . This retains exact citation provenance, but the model must attend to many independent inputs, which often harms coherence and fluency. Joint representation. Later work uses a single joint representation to represent the retrieved document set. RealM [ 12 ] and EMDR2 [ 24 ] learn a unified encoding; xRAG [ 3 ] pushes this to the extreme by compressing all content into a single token representation. These approaches improve synthesis and fluency, yet the compact latent representation discards explicit links to source documents, making citation grounding difficult and the pipeline opaque to developers. A recommended remedy is to first hallucinates an answer and then retrieve supporting evidence, as in HyDE [ 11 ] , which raises lingering concerns about trustworthiness. Clustering‑based RAG. Ginger [ 14 ] is a nugget-based RAG system that defines nuggets as verbatim text spans copied directly from retrieved passages. These spans are first clustered into topical facets using BERTopic, after which the clusters are reranked to identify the most relevant facets. Summaries of the top clusters form the system’s output, which is then rewritten for fluency. Although the pipeline has access to evidence for generation, the clustering summarization step abstracts away from the original document extractions, impacting the faithful citation grounding. Agentic RAG. Agentic frameworks decompose the pipeline into subtasks and let a planner invoke them as needed [ 27 , 1 ] . WebGPT [ 19 ] embeds search instructions in the prompt, letting the LLM choose passages to cite. A prominent agentic system is GptResearcher [ 7 , 8 ] , which orchestrates query generation, retrieval, extraction, and report writing, with optional verification and trace‑back. 3 Nugget-first RAG Approach: Crucible Figure 1: For each generated nugget, Crucible extracts candidate sentences and adds the best k sentences to the final response. No content clustering is needed. Figure 1 illustrates the workflow of Crucible , which builds its pipeline around structured Q&A nuggets that guide retrieval, extraction, and assembly. This contrasts with Ginger , which operates on verbatim spans clustered into facets. Nugget ideation. We begin by retrieving an initial pool of documents; in our study we use the PLAID‑X dense retriever [ 28 ] . For each document we generate a short query-focused summary with an LLM, then prompt the same LLM (using LLaMA-3.3-70B-Instruct ) to produce Q&A nuggets conditioned on the summary and the user request. To reduce redundancy among nuggets, we first detect paraphrases with an LLM prompt, then successively merge most confident paraphrases until the desired number of canonical nuggets is obtained. Nugget ranking. The resulting nugget bank is reranked with a Support‑Vector Classifier (SVC) trained on 19 quality features, which utilize LLM judge prompts to measure how well each nugget addresses the information need (task statement, background, role, communication style, and scope), degree of vitality [ 22 ] , and several features from researchy questions [ 23 ] . This is combined with basic readability metrics to quantify reading level [ 2 ] and sentence complexity [ 17 ] . Using the number of paraphrases as an indicator for nugget quality, we fuse this feature-based ranking with a popularity-based ranking. Obtaining the top 20 nuggets for each request will form the nugget bank that drives the remainder of the response generation. ⬇ Given the following question and answer , please find the sections in the provided source document that support and validate the answer to the question . If the source document does not support the given answer , then set confidence to 0.0 and all fields to None . Provide the section from the document that supports the answer , with complete sentences directly from the document . Please include context around each supporting segment , making sure that there is enough context to support why the answer is a correct response to the question . Your response should include just the extracted text segment , and nothing else . Then condense the extracted text into one concise sentence that clearly demonstrates how the question is answered , without referring to the source document . - Emit ** each field exactly once **, in the order shown . - Do NOT repeat headers . Do NOT include any field more than once . - If a field is unknown , omit it entirely - do NOT write ‘ None ‘ or ‘ null ‘. Field name Description In: nugget_text Input question In: answer Valid answer or list of answers In: source_document Input document chunk In: title Title query In: background User background In: problem_statement User’s problem statement Out: extracted_text_segment Passage returned by the model Out: summary Single concise sentence drawn from the passage Out: reasoning Internal chain of thought (ignored downstream) Out: confidence Model reported confidence value (float) Figure 2: Prompt for scanning, extraction, and sentence candidate generation. Retrieval. Our base system uses documents of which nuggets were extracted. Additionally, the pipeline is evaluated with the top 100 of several retrieval models with their default