Factual and Edit-Sensitive Graph-to-Sequence Generation via Graph-Aware Adaptive Noising
Aditya Hemant Shahane, Anuj Kumar Sirohi, Tanmoy Chakraborty, Prathosh A P, Sandeep Kumar · 2026
Read Factual and Edit-Sensitive Graph-to-Sequence Generation via Graph-Aware Adaptive Noising by Aditya Hemant Shahane, Anuj Kumar Sirohi, Tanmoy Chakraborty, Prathosh A P, Sandeep Kumar (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2604-24104.txt.
Introduction
Shahane and four colleagues present DLM4G, a diffusion framework for turning graphs into text with factual grounding and edit sensitivity. Fine-tuned autoregressive models often drift from graph facts and ignore small graph edits. DLM4G aligns graph entities and relations with sequence tokens and modulates noise by per-token denoising error, preserving structure and enabling local updates. It beats rival diffusion baselines and fine-tuned models up to 12 times larger, and it also transfers to molecule captioning.
Catalog record
| CREATOR | Aditya Hemant Shahane, Anuj Kumar Sirohi, Tanmoy Chakraborty, Prathosh A P, Sandeep Kumar |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 93668 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2604.24104
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Factual and Edit-Sensitive Graph-to-Sequence Generation via Graph-Aware Adaptive Noising Aditya Hemant Shahane, Anuj Kumar Sirohi, Tanmoy Chakraborty, Prathosh A P, Sandeep Kumar Abstract Fine-tuned autoregressive models for graph-to-sequence generation (G2S) often struggle with factual grounding and edit sensitivity. To tackle these issues, we propose a non-autoregressive diffusion framework that generates text by iterative refinement conditioned on an input graph, named as D iffusion L anguage M odel for G raphs ( DLM4G ). By aligning graph components (entities/relations) with their corresponding sequence tokens, DLM4G employs an adaptive noising strategy. The proposed strategy uses per-token denoising error as a signal to adaptively modulate noise on entity and relation tokens, improving preservation of graph structure and enabling localized updates under graph edits. Evaluated on three datasets, DLM4G consistently outperforms competitive G2S diffusion baselines trained on identical splits across both surface-form and embedding-based metrics. DLM4G further exceeds fine-tuned autoregressive baselines up to 12 × \times larger (e.g., T5-Large) and is competitive with zero-shot LLM transfer baselines up to 127 × \times larger. Relative to the strongest fine-tuned PLM baseline, DLM4G improves factual grounding ([email protected]) by +5.16% and edit sensitivity (ESR) by +7.9% ; compared to the best diffusion baseline, it yields gains of +3.75% in [email protected] and +23.6% in ESR. We additionally demonstrate applicability beyond textual graphs through experiments on molecule captioning, indicating the method’s generality for scientific G2S generation. 1 Introduction Graphs are widely used to represent relational data across domains such as social platforms, biological systems, and recommender systems ( 55 ; 56 ) . Natural-language verbalization of structured graph inputs is increasingly important, both for communicating graph content and as textual context for LLM-based reasoning ( 48 ) . This motivates graph-to-sequence (G2S) generation, where the goal is to produce a coherent output sequence conditioned on an input graph ( 13 ) . Real-world instances include (i) scientific captioning , where molecular or protein graphs are verbalized into concise natural-language descriptions ( 28 ) , and (ii) knowledge-graph based QA , where relevant KG subgraphs are verbalized to support multi-hop reasoning ( 59 ) . Early G2S approaches paired graph encoders with sequence decoders to inject structural bias explicitly ( 45 ; 46 ) . More recent work shows that autoregressive pre-trained language models (PLMs), when fine-tuned for G2S, can achieve strong performance on surface-overlap metrics such as BLEU and chrF++ even without graph-specific inductive biases ( 44 ) . However these metrics do not enforce coverage of the input graph: scores can remain high even when generated text contains factual omissions (missing entities/relations) or hallucinations (introducing unsupported facts). In addition, G2S systems are often deployed in interactive or evolving settings where the input graph is edited (e.g., adding/removing a neighbor, changing an entity, modifying a relation). In such cases, a desirable property is edit sensitivity : small, localized graph edits should induce predictable and localized changes in the generated text. One contributing factor for these failures is the left-to-right commitment of autoregressive decoding: once early tokens are generated, subsequent tokens are constrained by the prefix, making it difficult to revise earlier decisions ( 30 ; 16 ) . In contrast, non-autoregressive diffusion language models generate by iterative refinement , repeatedly denoising a corrupted sequence, which naturally supports global self-correction ( 15 ; 53 ) . While iterative refinement typically incurs higher inference cost than autoregressive decoding, it provides a built-in mechanism for revising earlier decisions and improving global consistency with the input graph. Yet standard diffusion schedules are typically data-agnostic , corrupting all tokens uniformly ( 21 ) . For G2S, uniform corruption can be misaligned with the input structure: tokens aligned to entities and relations may require different corruption/denoising than purely functional or syntactic tokens, and treating them identically can degrade factual grounding. To address these gaps, we propose D iffusion L anguage M odel for G raphs ( DLM4G ), a framework that leverages diffusion-based iterative refinement while preserving graph facts during corruption. Motivated by prior G2S findings that PLMs provide strong semantics but lack explicit structural guarantees ( 44 ) , we retain a PLM backbone and re-introduce graph structure through a soft inductive bias in the noising process, rather than imposing hard architectural constraints (e.g., graph encoders with permutation equivariance). The key idea is to align graph components (entities and relations) with their corresponding sequence tokens, and implement a graph-aware adaptive noising schedule that uses each aligned token’s denoising error as a difficulty signal: tokens that are harder to recover (typically entity/relation tokens) are corrupted less, while easier tokens are noised uniformly. This biased corruption preserves graph information during refinement and improves factual grounding, while also promoting localized updates when the input graph is edited. Finally, we complement modeling improvements with task-grounded evaluation by introducing metrics that directly measure (i) factual grounding and (ii) edit sensitivity, moving beyond surface overlap to quantify whether generated text faithfully reflects the input graph and responds predictably to local edits. Contributions. To sum up, our main contributions are: (1) a novel graph-aware adaptive noising schedule for diffusion-based G2S to improve factual grounding; (2) strong performance on three diverse benchmarks across surface and embedding-based metrics; (3) two task-grounded metrics for factual grounding and edit sensitivity; and (4) an extension to molecule captioning, demonstrating applicability to scientific G2S generation. 2 Background and Preliminaries 2.1 Related Work Graph-to-Sequence Learning. G2S has progressed from (i) template-based systems ( 26 ; 52 ) , to (ii) neural encoder–decoders with learned graph representations ( 45 ; 46 ) , and (iii) fine-tuned transformer-based models that achieve competitive performance on standard surface metrics (BLEU, chrF++, METEOR) even without explicit graph-specific inductive biases ( 44 ; 51 ) . This evolution motivates revisiting whether standard-metric performance also implies faithful realization of the input graph. PLMs for Graph Verbalization. A common strategy is to serialize the input graph (e.g., a KG) into a sequence of relational triples using special markers such as [HEAD] , [REL] , [TAIL] , and [SEP] (see Section 4.1 ). This enables direct use of standard encoder–decoder Transformers, but it gives up strict permutation invariance, a trade-off we revisit in Limitations 5 . Recent work also highlights challenges in (i) aligning graph elements to text and (ii) representing multi-level semantics across nodes, edges, and subgraphs ( 63 ) . While PLM-based approaches can be fluent, they remain constrained by left-to-right decoding, which can amplify early commitments and make local input edits propagate unpredictably ( 16 ) . Diffusion Models for Conditional Generation. Diffusion models provide a non-autoregressive alternative that generates by iterative refinement. In text, early approaches such as Diffusion-LM ( 30 ) and Analog Bits ( 7 ) explored controllable generation in continuous representations, while DIFFUSEQ ( 16 ) introduced sequence-to-sequence conditioning for diffusion language models. DLM4G builds on this line by conditioning explicitly on a serialized knowledge graph and introducing graph-aware adaptive noising to better preserve and recover graph-aligned components. Molecule Captioning. Graph-to-text generation is also central in scientific domains such as molecule and protein captioning ( 12 ; 33 ; 28 ; 14 ) . We include molecule captioning as an additional evaluation setting for graph-conditioned diffusion-based G2S. Table 6 compares prior paradigms with DLM4G . 2.2 Preliminaries: Denoising Diffusion Models