Calibrated Fusion for Heterogeneous Graph-Vector Retrieval in Multi-Hop QA

Andre Bacellar · 2026

Read Calibrated Fusion for Heterogeneous Graph-Vector Retrieval in Multi-Hop QA by Andre Bacellar (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2603-28886.txt.

Introduction

Bacellar studies how to fuse graph and vector scores for multi-hop question answering. Dense similarity scores and graph scores like Personalized PageRank have unlike spreads, so they are hard to compare. His method, PhaseGraph, maps both to a unit free scale with percentile rank normalization before fusion. On MuSiQue and 2WikiMultiHopQA, calibrated fusion improved held out last hop retrieval. Against the official HippoRAG 2 pipeline, results split by cutoff, ahead at one and behind at another. Ablations suggest percentile calibration is more robust than min-max scaling.

Catalog record

CREATORAndre Bacellar
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE32375
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2603.28886

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Calibrated Fusion for Heterogeneous Graph-Vector Retrieval in Multi-Hop QA Andre Bacellar Abstract Graph-augmented retrieval combines dense similarity with graph-based relevance signals such as Personalized PageRank (PPR), but these scores have different distributions and are not directly comparable. We study this as a score calibration problem for heterogeneous retrieval fusion in multi-hop question answering. Our method, PhaseGraph , maps vector and graph scores to a common unit-free scale using percentile-rank normalization (PIT) before fusion, enabling stable combination without discarding magnitude information. Across MuSiQue and 2WikiMultiHopQA, calibrated fusion improves held-out last-hop retrieval on HippoRAG2-style benchmarks: LastHop @5 increases from 75.1% to 76.5% on MuSiQue (8W/1L, p = 0.039 p{=}0.039 ) and from 51.7% to 53.6% on 2WikiMultiHopQA (11W/2L, p = 0.023 p{=}0.023 ), both on independent held-out test splits. A theory-driven ablation shows that percentile-based calibration is directionally more robust than min-max normalization on both tune and test splits (1W/6L, p = 0.125 p{=}0.125 ), while Boltzmann weighting performs comparably to linear fusion after calibration (0W/3L, p = 0.25 p{=}0.25 ). These results suggest that score commensuration is a robust design choice , and the exact post-calibration operator appears to matter less on these benchmarks. 1 Introduction Multi-hop question answering requires retrieving multiple evidence passages that form a reasoning chain, where later-hop passages are often weakly aligned with the original query. Dense retrieval is strong at lexical and semantic matching, but misses bridge passages whose relevance is mediated by graph structure rather than direct textual similarity. Graph-based retrieval offers a complementary signal through connectivity and diffusion, for example via Personalized PageRank (PPR) over an entity graph ( Yang et al. 2018 ; Trivedi et al. 2022 ) . A practical difficulty is that dense and graph retrieval scores are heterogeneous : cosine similarities cluster in a narrow Gaussian ( \mu\approx 0.09 , \sigma\approx 0.02 ), while PPR scores follow a power-law (most values \sim 0.001 , rare peaks \sim 0.3 ). Naïve score fusion is poorly behaved under this mismatch, while rank-only fusion (RRF) avoids calibration at the cost of discarding magnitude information. We study graph-vector retrieval fusion through the lens of score commensuration . We present PhaseGraph , a calibrated fusion method for heterogeneous graph-vector retrieval. The method first maps each retriever’s scores to a common unit-free scale using percentile-rank normalization (the probability integral transform, PIT), then combines the calibrated signals with a fusion rule. The central hypothesis is that calibration is the first-order step : once scores are made commensurable, the exact post-calibration operator appears to matter less than in uncalibrated fusion. We evaluate this on multi-hop benchmarks under both an earlier pipeline and a stronger HippoRAG2-style setup. Our contributions: • Calibrated heterogeneous-score fusion (Sections 5 , 6 ): We identify heterogeneous-score calibration as the central problem in graph-vector retrieval fusion, and propose PIT-based normalization. Held-out ablation is consistent with PIT being directionally more robust than min-max (1W/6L, p{=}0.125 ); the fusion operator (Boltzmann vs. linear) is empirically inconclusive (0W/3L, p{=}0.25 ). • Held-out confirmation on two benchmarks (Section 5 ): Under a strong HippoRAG2-style pipeline (Llama 3.3 70B FP8, NV-Embed-v2 4096d), PhaseGraph improves last-hop retrieval on held-out test splits of MuSiQue (8W/1L, p{=}0.039 ) and 2WikiMultiHopQA (11W/2L, p{=}0.023 ). True RRF is positive but does not confirm on either benchmark at these sample sizes. • Scale analysis and secondary evidence (Section 7 ): Pool explosion and entity fragmentation explain why uncalibrated fusion fails at corpus scale. Pool capping plus synonym linking restore zero-loss behavior (8W/0L, p{=}0.008 ). We also confirm empirically that cross-encoder reranking destroys multi-hop retrieval (Appendix B ), while Ising graph reranking adds marginal exploratory benefit on the curated subset only. 2 Related Work Score Fusion in IR. Reciprocal Rank Fusion ( Cormack et al. 2009 , RRF;) discards scores entirely, using only ranks. CombSUM and CombMNZ ( Fox and Shaw 1993 ) assume commensurable scores and apply additive or multiplicative combination. Montague and Aslam 2001 show that normalization matters more than the fusion algorithm, but leave the normalization choice unresolved. Our work tests this insight in the graph-vector setting, finding that percentile-rank normalization is a robust choice in this setting. BoltzRank ( Volkovs and Zemel 2009 ) applies Boltzmann distributions to permutations for learning-to-rank, but not to score fusion. Plackett-Luce models ( Hunter 2004 ) estimate item strengths from ranked lists via maximum likelihood, but assume homogeneous rankers. In our exploratory sweep, Plackett-Luce underperforms Boltzmann fusion when ranker score distributions differ fundamentally (Appendix A ). Graph-Augmented RAG. HippoRAG ( Gutiérrez et al. 2024 ) and HippoRAG 2 ( Gutiérrez et al. 2025 ) use Personalized PageRank for passage-level retrieval, achieving R@5 = 74.7% on MuSiQue. PropRAG ( Wang and Han 2025 ) extends this with beam search over the knowledge graph (R@5 = 77.3%, QA F1 = 78.3%). More recent energy-based approaches ( Yu et al. 2025 ) extend graph-RAG further, but require substantial additional LLM calls per query. Our approach differs in that we fuse existing vector and graph retrieval scores rather than designing new traversal algorithms. In principle, this makes the fusion step applicable to any graph-RAG system that produces two score lists, though we evaluate only our own pipeline. Multi-Hop Retrieval and Reranking. Standard cross-encoders evaluate passages independently and can demote bridge evidence in multi-hop QA. We confirm this empirically: cross-encoder reranking causes catastrophic last-hop failure on our benchmarks (Appendix B ). Our entity extraction follows a similarly minimal approach, avoiding full relation extraction to reduce construction cost. 3 Calibrated Fusion for Heterogeneous Retrieval 3.1 Problem Setting Given a query q , a vector retriever returns \mathcal{R}_{v}=\{(d_{i},s^{v}_{i})\}_{i=1}^{N_{v}} (typically N_{v}=10 ), and a graph retriever returns \mathcal{R}_{g}=\{(d_{j},s^{g}_{j})\}_{j=1}^{N_{g}} (typically N_{g}\gg N_{v} , as PPR returns all reachable nodes). The challenge: s^{v} and s^{g} are on incomparable scales and follow different distributions. 3.2 Percentile-Rank Normalization For each system k\in\{v,g\} , we compute the percentile rank of each document within its own score list: \hat{p}_{i}^{k}=\frac{|\{j:s_{j}^{k}\leq s_{i}^{k}\}|}{N_{k}} (1) This is simply the empirical CDF, mapping scores to [0,1] uniform (equivalently, the 1D optimal transport map to \mathcal{U}[0,1] ( Brenier 1991 ) ). This preserves within-system ordering while making cross-system values commensurable. 3.3 Boltzmann Energy and Temperature We convert percentile ranks to energies: E_{i}^{k}=-\ln(\hat{p}_{i}^{k}+\epsilon) (2) where \epsilon=10^{-6} prevents singularity at \hat{p}=0 . High-ranked documents have low energy (high percentile \rightarrow small E ). We set temperature as a fraction of mean energy: T_{k}=\frac{\bar{E}_{k}}{2}=\frac{1}{2N_{k}}\sum_{i=1}^{N_{k}}E_{i}^{k} (3) The factor of \frac{1}{2} is a heuristic that produces moderate sharpening; we did not tune this value. The effect is that Boltzmann probabilities are neither too peaked nor too flat, adapting to the spread of each system’s scores. 3.4 Weighted Boltzmann Fusion Boltzmann probabilities within each system: P_{i}^{k}=\frac{\exp(-E_{i}^{k}/T_{k})}{Z_{k}},\quad Z_{k}=\sum_{j}\exp(-E_{j}^{k}/T_{k}) (4) Final fusion with mixing parameter \alpha and consensus boost \beta : \text{score}(d_{i})=\alpha\cdot P_{i}^{v}+(1-\alpha)\cdot P_{i}^{g}+\beta\cdot\mathbb{1}[d_{i}\in\mathcal{R}_{v}\cap\mathcal{R}_{g}] (5) where \alpha controls vector-graph weighting and \beta boosts documents found by both systems (CombMNZ-inspired). Note that \beta operates on a different scale than the probability terms P_{i}^{k} ; in practice, \beta\in\{0.5,1.0,1.5\} acts as a discrete bonus for consensus documents. The appropriate \alpha depends on corpus size and graph density; empirical calibration is discussed in Section 7 . The top- K documents by fused score form the final retrieval set. 3.5 Comparison to RRF Reciprocal Rank Fusion computes \text{score}(d)=\sum_{k}\frac{1}{k_{0}+\text{rank}_{k}(d)} , using

Read the complete text