Double-Calibration: Towards Trustworthy LLMs via Calibrating Knowledge and Reasoning Confidence
Yuyin Lu, Ziran Liang, Yanghui Rao*, Wenqi Fan, Fu Lee Wang, Qing Li · 2026
Read Double-Calibration: Towards Trustworthy LLMs via Calibrating Knowledge and Reasoning Confidence by Yuyin Lu, Ziran Liang, Yanghui Rao*, Wenqi Fan, Fu Lee Wang, Qing Li (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2601-11956.txt.
Introduction
Yuyin Lu, Ziran Liang, Yanghui Rao, and their co-authors seek more trust in large language models. Such models can state false claims with full faith. Adding knowledge graphs can lift fact skill, but past tools do not gauge doubt in the proof they fetch or in the model's own steps.
Their frame, named DoublyCal in the work, rests on a double-calibration rule. It gauges doubt in both the fetched proof and the chain of thought. A model that knows its doubt can flag weak replies. That flag helps users judge when to trust an output.
Catalog record
| CREATOR | Yuyin Lu, Ziran Liang, Yanghui Rao*, Wenqi Fan, Fu Lee Wang, Qing Li |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 49 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2601-11956
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Double-Calibration: Towards Trustworthy LLMs via Calibrating Knowledge and Reasoning Confidence Yuyin Lu, Ziran Liang, Yanghui Rao*, Wenqi Fan, Fu Lee Wang, Qing Li Abstract Trustworthy reasoning in Large Language Models (LLMs) is challenged by their propensity for hallucination. While augmenting LLMs with Knowledge Graphs (KGs) improves factual accuracy, existing KG-augmented methods fail to quantify epistemic uncertainty in both the retrieved evidence and LLMs’ reasoning. To bridge this gap, we introduce DoublyCal, a framework built on a novel double - calibration principle. DoublyCal employs a lightweight proxy model to first generate KG evidence alongside a calibrated evidence confidence. This calibrated supporting evidence then guides a black-box LLM, yielding final predictions that are not only more accurate but also well-calibrated, with confidence scores traceable to the uncertainty of the supporting evidence. Experiments on knowledge-intensive benchmarks show that DoublyCal significantly improves both the accuracy and confidence calibration of black-box LLMs with low token cost. 1 Introduction The trustworthiness of Large Language Models (LLMs) is critically undermined by their tendency to hallucinate, a problem rooted in both intrinsic epistemic uncertainty (knowledge gaps) and extrinsic aleatoric uncertainty (data ambiguity) ( 10 ) . To mitigate this, Knowledge Graph-augmented Retrieval-Augmented Generation (KG-RAG) has emerged as a leading paradigm ( 40 ) . By augmenting LLM with structured evidence retrieved from external Knowledge Graphs (KGs), KG-RAG is helpful to reduce the model’s internal knowledge gaps and improve the factual accuracy of its responses ( 36 ) . However, this KG-augmentation mechanism introduces a new and critical dependency: the certainty of the retrieved evidence itself . Prevailing KG-RAG methods ( 23 ; 18 ) often rely on an idealistic assumption that the retrieved evidence is always both sufficient and certain to support correct reasoning for a given query. This assumption is routinely violated in practice due to ambiguous queries, the intrinsic incompleteness of KGs, and imperfections in the retrieval process. Consequently, when provided with partial evidence, LLMs may still produce confidently stated but incorrect predictions ( 14 ) . For example, as illustrated in Figure 1 , given the partial evidence “Belle is a sibling of Snoopy”, an LLM might incorrectly infer “Belle is Snoopy’s brother”. Thus, current KG-RAG lacks the ability to assess and control uncertainty at the very source of the reasoning chain. Concurrently, research on Uncertainty Quantification (UQ) for LLMs aims to calibrate prediction confidence but focuses predominantly on the final output ( 35 ) . For instance, verbalized UQ methods elicit confidence estimates from black-box LLMs, yet these estimates remain opaque and non-traceable ( 32 ; 37 ) . It is impossible to discern whether the expressed uncertainty stems from flawed evidence, deficiencies in the model’s own reasoning, or the intrinsic difficulty of the task. Therefore, existing UQ methods cannot synergize effectively with KG-RAG to provide a stepwise-calibrated view of the complete evidence-to-prediction chain. Figure 1: A motivating example of double-calibration against hallucination in KG-augmented LLMs. In summary, a principled solution for systematically managing the propagation of uncertainty in KG-augmented LLMs is lacking. To bridge this gap, we propose a novel double-calibration paradigm. Its core lies in moving beyond basic evidence retrieval to the construction of a calibrated reasoning chain , where confidence is explicitly estimated and made traceable from the retrieved KG evidence to the final LLM prediction. As visualized in Figure 1 , this enables the LLM to weigh alternative answers (e.g., correctly favoring “Spike” over “Belle”) based on the calibrated confidence of the supporting evidence. We instantiate this principle in DoublyCal , a framework that implements double-calibrated KG-RAG. DoublyCal grounds the LLM’s reasoning on verifiable KG evidence and performs dual calibration: it first calibrates the confidence of the retrieved evidence, then uses this calibrated evidence to guide and further calibrate the final LLM prediction. Specifically, we formalize KG evidence as constrained relational paths extracted from a KG. We then train a lightweight proxy model under Bayesian supervision to generate relevant KG evidence alongside a calibrated confidence score for each query. During inference, the primary LLM is prompted with both the KG evidence and its confidence estimate, leading to more accurate and better-calibrated predictions. Crucially, because the evidence confidence explicitly estimates the expected reasoning uncertainty of the LLM when utilizing the provided evidence, the final confidence becomes traceable to the verifiable KG evidence and its calibrated confidence, rather than remaining an opaque global estimate. Our main contributions are summarized as follows: • We establish the principle of double-calibration for trustworthy KG-augmented LLMs, which mandates explicit confidence calibration for both the KG evidence and the final LLM predictions. • We propose DoublyCal, a framework that implements this principle via a Bayesian-calibrated proxy model, providing the primary LLM reasoner with KG evidence accompanied by evidence confidence. • We empirically demonstrate that DoublyCal consistently and cost-effectively improves the accuracy and calibration of diverse black-box LLMs on knowledge-intensive benchmarks. 2 Related Work 2.1 Knowledge-Augmented Generation for Trustworthy LLMs Retrieval-Augmented Generation (RAG) reduces the inherent knowledge gaps of LLMs by providing external information, thereby improving the factual accuracy of their responses ( 40 ) . The choice of knowledge source defines a spectrum of RAG variants, ranging from (i) unstructured text in Vanilla RAG ( 7 ; 29 ) , to (ii) textual graphs that model latent connections in GraphRAG ( 8 ; 19 ) , and finally to (iii) formal Knowledge Graphs (KGs) with explicit relations in KG-RAG ( 23 ; 18 ; 26 ) . By providing precise and structured knowledge, KG-RAG offers a rigorous foundation for complex reasoning and has demonstrated superior performance on knowledge-intensive tasks ( 36 ) . However, prevailing KG-RAG methods typically overlook the inherent uncertainty in the retrieved evidence. To bridge this gap, we introduce a Bayesian-calibrated solution for sample-wise KG evidence confidence estimation, serving as the first stage of our double-calibration framework. 2.2 Uncertainty Quantification for LLMs Uncertainty Quantification (UQ) for LLMs aims to calibrate the confidence of model predictions to identify their epistemic boundaries, a vital step toward trustworthy AI 9 . While some studies incorporate uncertainty awareness during training ( 28 ) , most practical UQ methods often operate post-hoc and are categorized by model access ( 35 ) . For open - source LLMs, common techniques derive confidence from internal states, such as the feature - space distribution of hidden embeddings ( 3 ; 34 ) and the predictive entropy of the output distribution ( 24 ) . For black - box LLMs, where internal states are inaccessible, methods rely on API - based probing. A prevalent strategy generates multiple responses and evaluates their semantic consistency using similarity - based metrics ( 25 ) or entailment scores ( 16 ; 20 ) . A more efficient alternative is verbalized UQ, which directly prompts the LLM to verbalize its own confidence, eliciting introspective uncertainty estimates ( 32 ; 37 ; 31 ) . Due to its plug - and - play nature and low cost, verbalized UQ can be readily integrated with KG - RAG. This combination epitomizes the prevailing single-calibration paradigm and serves as a strong baseline in our work. A fundamental limitation across prior UQ paradigms is their exclusive focus on the final output, deriving confidence solely from the LLM’s internal states or self-assessment. This