Joint Source-Channel-Generation Coding: From Distortion-oriented Reconstruction to Semantic-consistent Generation
Tong Wu, Zhiyong Chen, Guo Lu, Li Song, Feng Yang, Meixia Tao, Wenjun Zhang · 2026
Read Joint Source-Channel-Generation Coding: From Distortion-oriented Reconstruction to Semantic-consistent Generation by Tong Wu, Zhiyong Chen, Guo Lu, Li Song, Feng Yang, Meixia Tao, Wenjun Zhang (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2601-12808.txt.
Introduction
Tong Wu and six co authors rethink how communication systems rebuild images at the receiver. Conventional systems, and even AI driven joint source channel coding, follow Shannon rate distortion theory. Generic distortion metrics fail to capture human visual perception, though. Reconstructions guided by them often come out blurred or unrealistic to the eye.
The team move from distortion oriented reconstruction to semantic consistent generation. Their joint source channel generation coding aims to keep meaning, not just pixels. A generated image can look sharp while staying true to the sent content. That shift suits low bandwidth links where bits are precious. The work bridges information theory and generative modeling.
Catalog record
| CREATOR | Tong Wu, Zhiyong Chen, Guo Lu, Li Song, Feng Yang, Meixia Tao, Wenjun Zhang |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 26 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2601-12808
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Joint Source-Channel-Generation Coding: From Distortion-oriented Reconstruction to Semantic-consistent Generation Tong Wu, Zhiyong Chen, Guo Lu, Li Song, Feng Yang, Meixia Tao, Wenjun Zhang Abstract Conventional communication systems, including both separation-based coding and AI-driven joint source-channel coding (JSCC), are largely guided by Shannon’s rate-distortion theory. However, relying on generic distortion metrics fails to capture complex human visual perception, often resulting in blurred or unrealistic reconstructions. In this paper, we propose Joint Source-Channel-Generation Coding (JSCGC), a novel paradigm that shifts the focus from deterministic reconstruction to probabilistic generation. JSCGC leverages a generative model at the receiver as a generator rather than a conventional decoder to parameterize the data distribution, enabling direct maximization of mutual information under channel constraints while controlling stochastic sampling to produce outputs residing on the authentic data manifold with high fidelity. We further derive a theoretical lower bound on the maximum semantic inconsistency with given transmitted mutual information, elucidating the fundamental limits of communication in controlling the generative process. Extensive experiments on image transmission demonstrate that JSCGC substantially improves perceptual quality and semantic fidelity, significantly outperforming conventional distortion-oriented JSCC methods. I Introduction For decades, the architecture of many communication systems has been largely shaped by the reconstruction paradigm, rooted in Shannon’s rate-distortion (RD) theory [ 1 ] . This framework treats the recovery of information as a deterministic task: the goal is to find a decoder that produces a point estimate \hat{x} as close as possible to the original x within a given bit rate. Guided by this principle, classical standards and recent deep joint source-channel coding (JSCC) schemes [ 2 , 3 , 4 , 5 ] have focused on minimizing explicit distortion metrics, such as mean squared error (MSE). However, this pursuit of fidelity creates a fundamental conflict with visual perception [ 6 ] . The inherent limitation of the reconstruction-oriented approach is its tendency to produce average results. When communication resources are constrained, MSE-based optimization often leads to the smoothing trap, where high-frequency details are discarded in favor of a blurred, statistically safe reconstruction. While perceptual metrics like Learned Perceptual Image Patch Similarity (LPIPS) [ 7 ] attempt to bridge this gap, they often introduce unnatural artifacts or high-frequency noise because they operate within the constraints of explicit, often imperfect, distortion functions. Though recent rate-distortion-perception (RDP) based coding schemes [ 8 , 9 , 10 ] introduce perceptual constraints, e.g., adversarial losses, into the optimization objective, they still operate within a deterministic reconstruction. This paradigm-level constraint, rather than the specific choice of distortion function, limits their ability to model the structure of the natural data manifolds. Fig. 1: System model: (a) reconstruction paradigm; (b) generation paradigm. In this paper, we propose a new generative paradigm termed Joint Source-Channel-Generation Coding (JSCGC) . Our core philosophy is to advocate for generation instead of reconstruction . Rather than attempting to recover a single deterministic realization of the source data, JSCGC treats the receiver’s task as a conditional stochastic generation problem. In this new paradigm, the received signal is not used to reconstruct data directly, but serves as a semantic guide to sample from the posterior distribution p(x|\hat{y}) . By shifting from reconstruction to controlled stochastic generation, we enable the system to synthesize high-fidelity textures that are perceptually indistinguishable from authentic data even under severe channel noise, shifting the distortion to inconsistency. Inspired by these insights, we design a joint coding and generation framework derived directly from the mutual information (MI) function, subject to a perceptual constraint. By leveraging generative models, e.g., diffusion models (DMs) [ 11 , 12 , 13 , 14 ] , to parameterize the data distribution, we ensure that the generated content remains semantically consistent with the source without requiring a rigid loss function. Moreover, this paper provides a theoretical derivation of the lower bound on maximum semantic inconsistency, establishing a mathematical foundation for generative communications. To realize this paradigm, we implement an end-to-end system combining a Mamba-based encoder with a latent flow matching model [ 15 ] , integrated via a communication-aware adapter that controls the generative process with received signals while effectively preserving large-scale pre-trained knowledge. II System Model As shown in Fig. 1 , we consider a communication system in which the transmitter encodes the original data \mathbf{x}\in\mathbb{R}^{m} into \mathbf{y}\in\mathbb{C}^{l} via an encoder E_{\phi}(\cdot) parameterized by \phi , where l<m . The encoded signal \mathbf{y} is then transmitted through a communication channel, resulting in the received signal \hat{\mathbf{y}}\in\mathbb{C}^{l} . At the receiver, unlike traditional JSCC schemes that perform deterministic reconstruction, we formulate the decoding process as a conditional generative problem, in which the recovered data \hat{\mathbf{x}}\in\mathbb{R}^{m} is sampled from the conditional distribution q_{\theta}(\mathbf{x}|\hat{\mathbf{y}}) using a generative model G_{\theta}(\cdot) parameterized by \theta . Accordingly, the overall communication process can be modeled as the following Markov chain: \displaystyle\mathbf{X}\rightarrow\mathbf{Y}\rightarrow\hat{\mathbf{Y}}\rightarrow\hat{\mathbf{X}}. (1) Here, \mathbf{X},\ \mathbf{Y},\ \hat{\mathbf{Y}}, and \hat{\mathbf{X}} denote the random variables corresponding to the realizations \mathbf{x},\ \mathbf{y},\ \hat{\mathbf{y}}, and \hat{\mathbf{x}} , respectively. As established, the fundamental objective of communication is the reliable transfer of information. Meanwhile, the recovered signal \hat{\mathbf{X}} is intended for human perception, and the system must also ensure that \hat{\mathbf{X}} is perceptually faithful to the source. This necessitates a perceptual constraint, requiring the distribution of the generative signal to closely align with the natural image distribution p(\mathbf{X}) . Therefore, for the communication system defined in ( 1 ), we formulate the joint encoding and generation scheme as the following constrained optimization problem: \displaystyle\max_{\theta,\phi}\ I(\mathbf{X};\hat{\mathbf{Y}}), \displaystyle\text{s.t.}\quad d_{p}\big(p(\mathbf{X}),p(G_{\theta}(\hat{\mathbf{Y}}))\big)\leq\zeta. (2) where d_{p}(\cdot) denotes a divergence metric that quantifies the perceptual discrepancy between the distributions of \mathbf{X} and \hat{\mathbf{X}} , and \zeta is a prescribed tolerance. This design objective sets our work apart from traditional RD or RDP approaches. The primary distinction lies in our departure from explicit distortion functions; instead, we formulate the joint coding and generation scheme by maximizing mutual information subject exclusively to perceptual constraints. Remark 1 . Given that we design the coding and generation scheme in JSCGC from the perspective of MI, we primarily aim to explicitly obtain the encoding parameters \phi and generative parameters \theta through optimization from ( II ). In practice, the mutual information terms involved in ( II ) cannot be computed in closed form due to the unknown distributions of high-dimensional signals. To address this challenge, we propose a practical JSCGC scheme based on deep neural networks to approximate the objective in ( II ). III Proposed JSCGC Scheme In this section, we introduce the JSCGC scheme, optimized via joint source-channel-generation design. We provide a perceptually-constrained sampling algorithm for high-quality generation and derive the theoretical lower bound for maximum semantic inconsistency. III-A JSCGC Training Algorithm Since the source distribution p(\mathbf{X}) is fixed, maximizing I(\mathbf{X};\hat{\mathbf{Y}}) is equivalent to minimizing the conditional entropy H(\mathbf{X}|\hat{\mathbf{Y}})=\mathbb{E}_{(\mathbf{x},\hat{\mathbf{y}})\sim p(\mathbf{x},\hat{\mathbf{y}})}\left[-\log p(\mathbf{x}|\hat{\mathbf{y}})\right] . However, the true posterior distribution p(\mathbf{x}|\hat{\mathbf{y}}) requires knowledge of the ground-truth joint distribution of natural data and channel outputs, which is theoretically intractable. To address this issue, we have the following proposition. Proposition 1 . The conditional entropy H(\mathbf{X}|\hat{\mathbf{Y}}) can be minimized via the following gradient-based loss: \displaystyle\mathcal{L}(\theta,\phi)=\mathbb{E}_{t,\mathbf{x}_{0},\mathbf{x}_{T}}\left[\|(\mathbf{x}_{T}-\mathbf{x}_{0})-\mathbf{\nu}_{\theta}(\mathbf{x}_{t},t,\hat{\mathbf{y}})\|^{2}\right]. (3) where \mathbf{x}_{0}\sim p(\mathbf{x}) , \mathbf{x}_{T}\sim N(0,\mathbf{I}) , \mathbf{x}_{t}=(1-\frac{t}{T})\mathbf{x}_{0}+\frac{t}{T}\mathbf{x}_{T} for t\in\{1,2,...,T\} , and \mathbf{\nu}_{\theta}(\cdot) is the output of the neural network in the generator. Proof. We introduce a parameterized variational distribution q_{\theta}(\mathbf{x}|\hat{\mathbf{y}}) . Therefore, its