Re-Ranking Through an Attribution Lens for Citation Quality in Legal QA
Mohamed Hesham Elganayni, Selim Saleh · 2026
Read Re-Ranking Through an Attribution Lens for Citation Quality in Legal QA by Mohamed Hesham Elganayni, Selim Saleh (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2606-03728.txt.
Introduction
Elganayni and Saleh rerank legal search results by predicted usefulness for citation, not by similarity. Retrieval systems for legal questions fetch passages by semantic similarity. A language model then cites from that pool. Prior work assumed top-ranked passages get cited best. On the AQuAECHR benchmark, that assumption fails. Similarity does not track passage attribution, and similarity ranking does worse than random at finding gold citations.
Attribution scores from perturbation methods like C-LIME supply a new training signal. A light cross-encoder learns to predict those scores and reranks passages before generation. Two language models and five-fold cross-validation test the approach. Citation faithfulness and match to gold expert answers improve a great deal. Rerankers trained on different models converge past their raw attribution agreement. A shared relevance signal thus emerges, partly transferable across models. Same-model reranking still works best.
Catalog record
| CREATOR | Mohamed Hesham Elganayni, Selim Saleh |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 33 KB |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2606.03728
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Re-Ranking Through an Attribution Lens for Citation Quality in Legal QA Mohamed Hesham Elganayni, Selim Saleh Abstract Retrieval-augmented generation systems for legal question answering typically retrieve passages based on semantic similarity and provide them to a language model, which then generates cited answers. Prior work assumes that highly ranked passages are most likely to be usefully cited by the model. Perturbation-based attribution methods, such as C-LIME, have been used exclusively for post-hoc explanation. However, on the AQuAECHR benchmark, semantic similarity does not correlate with passage attribution. Within a retriever’s candidate pool, similarity-based ranking performs worse than random selection at surfacing gold citation paragraphs. To address this limitation, a lightweight cross-encoder is trained on continuous perturbation-based attribution scores to re-rank passages prior to generation. This approach is evaluated on the AQuAECHR benchmark, using two language models and five-fold cross-validation. The re-ranker substantially improves citation faithfulness and alignment with gold expert answers. Notably, two re-rankers trained independently on different models converge beyond their raw attribution agreement. This finding indicates that the cross-encoder reduces model-specific noise and produces a shared relevance signal that partially transfers across models, although same-model re-ranking remains more effective. These results demonstrate that perturbation-based attribution provides a practical, model-agnostic training signal for citation-aware retrieval. [[email protected], ] [[email protected], ] keywords retrieval-augmented generation ,legal question answering ,passage re-ranking ,perturbation-based attribution ,citation quality ,cross-encoder ,ECHR † † copyrightyear: 2026 † † copyright: Copyright for this paper by its authors. Use permitted under Creative Commons License Attribution 4.0 International (CC BY 4.0). † † venue: Proceedings of the Eighth International Workshop on Automated Semantic Analysis of Information in Legal Text (ASAIL 2026), 12 June 2026, Singapore † † address: Technical University of Munich, Munich, Germany 1 Introduction Large language models have demonstrated substantial legal reasoning capabilities, including passing bar examinations 6 and answering jurisdiction-specific questions grounded in case law. However, deployment in legal practice requires more than correct answers. Strict adherence to source text and precise jurisdictional reasoning are essential, and every statement must be traceable to authoritative sources. Retrieval-augmented generation 7 addresses this by retrieving relevant passages and prompting the model to cite them when generating answers. Nevertheless, citation quality remains a significant bottleneck. For example, on the AQuAECHR benchmark for European Court of Human Rights (ECHR) question answering, retrieve-then-generate systems achieve only modest citation faithfulness 23 , and broader evaluations indicate that even the strongest models provide complete citation support for only about half their statements 2 . A central assumption in these pipelines is that passages ranked highly by the retriever are those the language model will usefully cite. However, this assumption does not hold. On AQuAECHR, semantic similarity between a question and its retrieved passages is effectively uncorrelated with perturbation-based attribution scores, which measure the actual influence of each passage on the generated answer. Furthermore, within the retriever’s candidate pool, ranking by similarity performs worse than random selection at surfacing gold citation paragraphs. In specialized domains such as ECHR case law, standard retrievers frequently return topically related but doctrinally irrelevant precedents. Consequently, the generator often ignores the top-ranked evidence in favor of lower-ranked passages or post-rationalizes claims using parametric memory instead of the provided context 22 . This highlights that standard retrievers working with embedding-based similarity may be inadequate for identifying the exact paragraphs that a language model actually relies upon for citation. Perturbation-based attribution methods such as C-LIME and L-SHAP 11 can measure passage-level influence by observing how the generation changes when individual passages are removed. These methods distinguish passages that the model relies on from those it ignores. Despite their effectiveness, they have been used exclusively as post-hoc explanation tools. Cross-encoder re-ranking 13 is the established approach for improving passage selection after initial retrieval, but existing re-rankers are trained on relevance judgments or distilled ranking signals rather than on measured generation influence. Yet attribution remains confined to explaining outputs after the fact, and re-ranking remains blind to how the generator actually consumes its input passages. We address this gap by training a lightweight cross-encoder on continuous C-LIME attribution scores to re-rank passages prior to generation. The approach is evaluated on the full AQuAECHR benchmark using five-fold cross-validation with two language models, Mistral-7B and Llama-3-8B, both used in the AQuAECHR baseline to enable direct comparison. The main contributions are as follows: 1. We repurpose perturbation-based attribution from a post-hoc explanation tool into a training signal for passage re-ranking, resulting in improvements in citation faithfulness as well as alignment with gold expert answers across both models. 2. We demonstrate that the improvement results from attribution training rather than the cross-encoder architecture itself. A pretrained cross-encoder used as a re-ranker without C-LIME training captures less than half of the citation gains, and its correlation with attribution scores increases substantially after training on C-LIME scores. 3. We find that two re-rankers trained independently on different language models converge beyond their raw attribution agreement. This indicates that the cross-encoder reduces model-specific noise and learns a shared passage-relevance signal that transfers across models, achieving consistent improvement over the baseline while retaining 43–51% of same-model citation gains. 2 Related Work Retrieval-augmented generation 7 is widely used for producing grounded, cited answers. However, citation quality remains a challenge, as advanced models lack complete citation support for about half of their statements. A growing body of evidence indicates that this issue results from a fundamental misalignment between retrieval relevance and generation utility. Liu et al. 9 observed that language models display a U-shaped performance curve over retrieved passages, often failing to use relevant information placed in the middle of the input context regardless of its relevance. Randl et al. 18 quantified this gap directly, finding that generators completely disregard the retriever’s top-ranked documents for 47–67% of queries and instead rely on lower-ranked passages 48–66% of the time. This misalignment also affects citation behavior. Wallat et al. 22 demonstrated that up to 57% of RAG citations are post-rationalized. That is, the model generates content using parametric memory and subsequently selects supporting passages, rather than citing the documents it actually used. Cross-encoder re-ranking 13 is the standard approach for improving passage selection following initial retrieval. The paradigm has since scaled from BERT-based cross-encoders to sequence-to-sequence models such as monoT5 14 and RankT5 25 , and more recently to LLM-scale pointwise rerankers like RankLLaMA 10 and listwise approaches distilled from proprietary models 16 . Recent literature frames reranking as the critical post-retrieval module for bridging the retriever–generator gap 3 . Beyond scaling, recent research has begun to adapt re-rankers to align with generator preferences rather than depending solely on surface relevance. Sun et al. 21 showed that LLM ranking judgments can be distilled into lightweight models through permutation distillation, and Jia et al. 5 extended this with RADIO, training re-rankers on LLM-generated text rationales about passage utility. However, these methods depend on the LLM’s explicit outputs, which are themselves subject to the faithfulness concerns identified by Wallat et al. Text rationales may indicate what the model claims to find useful rather than what it actually relies on during generation. Notably, none of these approaches examine whether the learned re-ranking signal generalizes across different generator models. Separately, Monteiro Paes et al. 11 introduced C-LIME and L-SHAP, perturbation-based attribution methods that