DSVD: Dynamic Self-Verify Decoding for Faithful Generation in Large Language Models
YiQiu Guo, Yuchen Yang, Zhe Chen, Pingjie Wang, Yusheng Liao, Ya Zhang , Yanfeng Wang , Yu Wang ,\textbf{Yanfeng Wang},\textbf{Yu Wang } · 2025
Read DSVD: Dynamic Self-Verify Decoding for Faithful Generation in Large Language Models by YiQiu Guo, Yuchen Yang, Zhe Chen, Pingjie Wang, Yusheng Liao, Ya Zhang , Yanfeng Wang , Yu Wang ,\textbf{Yanfeng Wang},\textbf{Yu Wang } (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2503-03149.txt.
Introduction
YiQiu Guo and seven co-authors present DSVD, a 2025 decoding method that helps large language models catch their own errors while writing. Models often state false facts with confidence, and checking only after the full answer costs time. DSVD checks the text as it forms. One part grades the draft continuously, in parallel with writing. Another part rolls back to the error point and rewrites just that stretch when a problem appears.
Across five benchmarks, the method raised truthfulness in question answering and factual accuracy in scored summaries. It also stacked well with older faithful-decoding methods, lifting them further. The authors argue that live self-checking is a practical route to more trustworthy models, since it needs no separate verifier model and fits real deployment.
Catalog record
| CREATOR | YiQiu Guo, Yuchen Yang, Zhe Chen, Pingjie Wang, Yusheng Liao, Ya Zhang , Yanfeng Wang , Yu Wang ,\textbf{Yanfeng Wang},\textbf{Yu Wang } |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 54 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2503-03149
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
DSVD: Dynamic Self-Verify Decoding for Faithful Generation in Large Language Models YiQiu Guo, Yuchen Yang, Zhe Chen, Pingjie Wang, Yusheng Liao, Ya Zhang , Yanfeng Wang , Yu Wang ,\textbf{Yanfeng Wang},\textbf{Yu Wang } Abstract The reliability of large language models remains a critical challenge, particularly due to their susceptibility to hallucinations and factual inaccuracies during text generation. Existing solutions either underutilize models’ self-correction with preemptive strategies or use costly post-hoc verification. To further explore the potential of real-time self-verification and correction, we present Dynamic Self-Verify Decoding (DSVD), a novel decoding framework that enhances generation reliability through real-time hallucination detection and efficient error correction. DSVD integrates two key components: (1) parallel self-verification architecture for continuous quality assessment, (2) dynamic rollback mechanism for targeted error recovery. Extensive experiments across five benchmarks demonstrate DSVD’s effectiveness, achieving significant improvement in truthfulness (Quesetion-Answering) and factual accuracy (FActScore). Results show the DSVD can be further incorporated with existing faithful decoding methods to achieve stronger performance. Our work establishes that real-time self-verification during generation offers a viable path toward more trustworthy language models without sacrificing practical deployability. 1 Introduction Figure 1: Comparative analysis of different decoding strategies: (a) Direct decoding leaves existing errors unexploited. (b) Baseline backtrack decoding propagates geographic hallucination and incurs high computation costs. (c) Our dynamic approach corrects "New Zealand"→"Australia" with minimal overhead. Large Language Models (LLMs) have demonstrated remarkable capabilities across various natural language processing (NLP) tasks, including question-answering, summarization, and conversation generation OpenAI et al. (2024) ; DeepSeek-AI et al. (2025) ; Touvron et al. (2023) . Despite their impressive performance, these models frequently suffer from reliability issues manifested through hallucinations and factual inaccuracies Kadavath et al. (2022) ; Xiong et al. (2023) ; Gekhman et al. (2024) ; Li et al. (2024) . These deficiencies pose significant practical concerns as users may unwittingly trust erroneous information presented in the models’ confident and coherent outputs. Recent advancements in faithful generation have shifted focus towards inference-stage interventions Liang et al. (2024) ; Luo et al. (2024) ; Chen et al. (2024a) . Researchers recognize that even models containing factual knowledge during pre-training often fail to access this information during generation reliably. Decoding-time adjustment strategies present a promising direction, offering more cost-effective solutions compared to supervised fine-tuning (SFT), which requires substantial computation, or retrieval-augmented generation (RAG), which necessitates an external knowledge base. As illustrated in Figure 1 , existing faithful generation approaches can be categorized into two paradigms: Direct Decoding methods (e.g., ITI Li et al. (2023a) , DoLa Chuang et al. (2024) , TruthX Zhang et al. (2024b) ) steer model outputs toward truthful directions by manipulating internal representations, leveraging the model’s inherent truthful priors. While effective, these approaches fail to leverage the model’s ability for self-correction of errors and reflective reasoning, leaving the model powerless against error accumulation. Backtracking Decoding methods (e.g., Self-Refine Madaan et al. (2023a) , Reflexion Shinn et al. () ) employ post hoc verification of generated content, but existing implementations suffer from significant computational overhead and vulnerability to error accumulation, where initial errors propagate into subsequent generations through self-reinforcing mechanisms. Dataset Probing w/o Response Probing w/ Response SciQ 64.86 87.21 CoQA 62.98 76.88 TriviaQA 68.33 75.66 Table 1: Experiment results of validating our insights: delayed awareness of hallucinations, the metric is AUROC. More details and analysis are in Appendix D . To address these limitations, we propose D ynamic S elf- V erify D ecoding (DSVD), a novel decoding strategy that incorporates real-time self-verification with dynamic rollback mechanisms. Our approach builds on key insights: (1) Delayed Awareness of Hallucinations : models demonstrate superior ability in detecting existing errors compared to preemptively preventing them, as shown in 1 , and (2) Local Error Correction is more Efficient : localized rollback enables error correction at their source, offering higher efficiency than global rewriting. The method mirrors human behavioral patterns: speculating, verifying, and refining the consequences before reaching a conclusion. More specifically, the framework operates through two components: 1) Fine-grained hallucination detector trained on model-generated pseudo-labels; and 2) Parallel self-verification and dynamic rollback mechanism enabling real-time hallucination detection and error correction. Our experimental evaluation across multiple LLM architectures (LlaMA-2, LlaMA-3, Qwen-2.5) and benchmarks (TruthfulQA, StrQA, SciQ, EntityQuestions, FActScore) demonstrates consistent improvements in truthfulness and factual accuracy while maintaining computational efficiency. Notably, DSVD shows complementary benefits when combined with existing direct generation methods, suggesting orthogonal mechanisms of action. Our key contributions include: • We propose a straightforward yet intuitive semi-supervised hallucination labeling approach for fine-grained self-feedback. • We propose Dynamic Self-Verify Decoding: a novel decoding strategy that enables parallel self-verification and dynamic self-correction. • Comprehensive experiments across diverse LLMs and evaluation metrics reveal consistent performance improvements of DSVD. 2 Related Work 2.1 Faithful Decoding In recent years, a series of studies have focused on leveraging truthful distribution to intervene in the model’s next-token prediction. Some research has explored directing the model’s generation towards a "more truthful" direction through representation editing. ITI Li et al. (2023a) trains probing heads to identify a set of more truthful attention heads and enhances the weights of these heads during inference. TrFr Chen et al. (2024b) proposed the application of multi-dimensional orthogonal probes, which effectively extract features from both truthful and non-truthful texts to better identify effective attention heads. TruthX Zhang et al. (2024b) not only targets attention heads but also latent states in the forward feedback layer. By separately mapping these states using truthful and semantic encoders. Another line of research investigates contrastive decoding for faithful generation. The pioneering work by Li et al. (2023b) introduced Contrastive Decoding, which selects optimal tokens by contrasting probability distributions from expert and amateur models. Building on this foundation, DoLa Chuang et al. (2024) enhanced the framework by incorporating intermediate layer representations, thereby improving early-stage reasoning consistency and pre-answer alignment through its Decoding-by-Contrasting-Layers mechanism. SLED proposed by Zhang et al. (2024a) integrates latent knowledge into logits via single-step gradient-like operation instead of replacing original outputs in DoLa during inference. Our Innovation: The direct decoding methods mostly intervene before the model predicts the next token, thus the model’s self-awareness and self-feedback capabilities regarding hallucinations are unexploited, while DSVD intervene after the model encounter hallucination and thus fully utilize the self-reflection ability of large language models. 2.2 Self Feedback Figure 2: Illustration of the Dynamic Self-Verify Decoding Framework: Step 1 : Parallel hallucination detection through trained probing heads, operating concurrently with the LM Head’s next-token prediction; Step 2 : Dynamic rollback to pre-hallucination positions upon error detection; Step 3 : Sample candidate continuation with probing-head-derived penalty terms for re-ranking; Step 4 : Resumption of the decoding process with revised token sequences. Studies on self-feedback utilize the Large Language Model itself as a critic, enabling the model to generate feedback on its responses and further refine those responses based on the generated feedback. Self-Refine Madaan et al. (2023a) simply uses the LLM in SelfEvaluate(·) to generate textual feedback. Reflexion Shinn et al. () makes progress by regarding iterative refinement as Verbal Reinforcement Learning without weight updates. Self-Correct Welleck et al. (2022) uses the same framework but trains a Corrector model for better feedback. Yet, due to not being task-agnostic and the need for training, it reduces the framework’s flexibility. Our Innovation: Traditional self-feedback