Text-Guided Diffusion Model-based Generative Communication for Wireless Image Transmission

Shengkang Chen, Tong Wu, Zhiyong Chen, Feng Yang, Meixia Tao, IEEE Fellow, Wenjun Zhang, IEEE Fellow · 2025

Read Text-Guided Diffusion Model-based Generative Communication for Wireless Image Transmission by Shengkang Chen, Tong Wu, Zhiyong Chen, Feng Yang, Meixia Tao, IEEE Fellow, Wenjun Zhang, IEEE Fellow (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2510-21299.txt.

Introduction

Shengkang Chen and five co-authors send images over wireless links at extremely low rates, where normal compression leaves pictures badly damaged. Their answer is to rebuild a picture that keeps the meaning of the scene, instead of chasing an exact copy of every pixel in the original.

The sender compresses the image with a learned joint source-channel code and also sends a short text prompt describing the scene. At the receiver, the damaged signal and the prompt are fused and passed to a Stable Diffusion model steered by ControlNet, which rebuilds a convincing image. Across varied channel conditions, tests beat both classic coding schemes and deep learning baselines on how good the resulting pictures look.

Catalog record

CREATORShengkang Chen, Tong Wu, Zhiyong Chen, Feng Yang, Meixia Tao, IEEE Fellow, Wenjun Zhang, IEEE Fellow
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE39 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2510-21299

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Text-Guided Diffusion Model-based Generative Communication for Wireless Image Transmission Thanks: The authors are with the Cooperative Medianet Innovation Center and the Department of Electronic Engineering, Shanghai Jiao Tong University, Shanghai 200240, China (e-mail: {chen20,wu_tong,zhiyongchen,yangfeng, mxtao, zhangwenjun}@sjtu.edu.cn). Shengkang Chen, Tong Wu, Zhiyong Chen, Feng Yang, Meixia Tao, IEEE Fellow, Wenjun Zhang, IEEE Fellow Abstract Reliable image transmission over wireless channels is particularly challenging at extremely low transmission rates, where conventional compression and channel coding schemes fail to preserve adequate visual quality. To address this issue, we propose a generative communication framework based on diffusion models, which integrates joint source–channel coding (JSCC) with semantic-guided reconstruction leveraging a pre-trained generative model. Unlike conventional architectures that aim to recover exact pixel values of the original image, the proposed method focuses on preserving and reconstructing semantically meaningful visual content under severely constrained rates, ensuring perceptual plausibility and faithfulness to the scene intent. Specifically, the transmitter encodes the source image via JSCC and jointly transmits it with a textual prompt over the wireless channel. At the receiver, the corrupted low-rate representation is fused with the prompt and reconstructed through a Stable Diffusion model with ControlNet, enabling high-quality visual recovery. Leveraging both generative priors and semantic guidance, the proposed framework produces perceptually convincing images even under extreme bandwidth limitations. Experimental results demonstrate that the proposed method consistently outperforms conventional coding-based schemes and deep learning baselines, achieving superior perceptual quality and robustness across various channel conditions. I Introduction Wireless communication demands have grown substantially in recent decades, driven by data-intensive applications such as remote sensing, autonomous driving, and multimedia services. Traditional communication technologies often struggle to meet these ever-increasing requirements, particularly in extreme scenarios such as underwater channels and satellite links, where bandwidth is severely constrained and channel impairments are significant. These limitations highlight the need for novel paradigms capable of maintaining semantic fidelity and perceptual quality under ultra-low bitrate constraints. Semantic communication has thus emerged as a promising direction, aiming to enhance efficiency by conveying the underlying meaning of information rather than transmitting raw data [ 1 , 2 ] . Among the various approaches to realizing semantic communication, deep learning-based joint source–channel coding (JSCC) has attracted considerable attention [ 3 , 4 ] . Unlike conventional image transmission pipelines that separate compression and error correction, JSCC directly maps source data into channel symbols through a compact latent representation. Early work such as DeepJSCC [ 5 ] demonstrated the potential of neural JSCC, outperforming traditional separate coding schemes that combine advanced image codecs like Better Portable Graphics (BPG) with low-density parity-check (LDPC) codes. Building on this foundation, subsequent studies have extended JSCC with attention mechanisms for channel adaptivity (ADJSCC [ 6 ] ), vision transformers and entropy models for semantic compression (NTSCC [ 7 ] ), hierarchical transformer backbones for improved robustness (SwinJSCC [ 8 ] ), and state space models for parameter-efficient architectures (MambaJSCC [ 9 ] ). Despite these advances, the performance of deep JSCC remains fundamentally constrained at extremely low bitrates. In such cases, heavy compression of the latent representation leads to blurry reconstructions that lack fine details and structural fidelity, causing the outputs to deviate from the natural image distribution. Conventional decoders trained with pixel-wise or perceptual losses often yield distorted and semantically ambiguous results, unable to capture the rich statistical patterns of real-world visual data. Generative models, on the other hand, excel at modeling complex, high-dimensional image distributions and provide strong priors that encode realistic textures, structures, and semantic coherence. Leveraging these generative priors offers a promising pathway to refine coarse JSCC reconstructions into visually plausible and perceptually faithful images, motivating the integration of generative models into JSCC frameworks. Recently, diffusion models have demonstrated remarkable generative capabilities in both image synthesis and restoration tasks [ 10 , 11 ] . Large-scale pre-trained models such as Stable Diffusion [ 12 ] achieve impressive results in unconditional and conditional generation. By leveraging powerful generative priors learned from massive image–text datasets, these models can produce semantically consistent and visually realistic outputs conditioned on textual prompts or auxiliary visual inputs. Such strengths make diffusion models highly appealing for wireless image transmission, where they hold the potential to enhance perceptual quality and preserve semantic fidelity beyond the capabilities of conventional JSCC decoders. Although diffusion models are typically trained under the assumption of Gaussian noise corruption, recent studies have shown their effectiveness in addressing more complex, non-Gaussian distortions. Soft Diffusion [ 13 ] extends score matching to a broader family of linear corruptions, including blurring and masking, thereby enabling robust restoration under structured degradations. Building on this idea, [ 14 ] demonstrated that diffusion models can be adapted to mitigate structured noise through joint posterior sampling. Even in the case of nonlinear and non-invertible distortions such as severe JPEG artifacts, CODiff [ 15 ] has been shown to successfully reconstruct original images by incorporating compression-aware priors into a one-step diffusion framework. These advances suggest that, although JSCC reconstructions deviate from the standard Gaussian corruption assumption, diffusion models can still be effectively exploited to refine and denoise such outputs, provided that the coarse semantic structure is preserved. Inspired by these successes, researchers have recently explored the application of diffusion models in wireless communication. For example, CDDM [ 16 ] employs a diffusion model to denoise the received signal at the decoder, while [ 17 ] applies diffusion-based post-processing modules to refine degraded images at the receiver. DiffCom [ 18 ] further incorporates diffusion-based denoising strategies for image transmission over wireless channels. However, since DiffCom does not exploit explicit semantic conditions, its reconstructions may appear perceptually plausible yet semantically inconsistent under ultra-low bitrate constraints. More recently, SGD-JSCC [ 19 ] introduced semantics-guided diffusion for JSCC, where edge maps are used as semantic guidance during diffusion sampling on JSCC-generated latent features. Although this approach improves performance in low-SNR channels, the reliance on edge maps as conditional inputs inevitably discards color and fine-grained texture information, and its evaluation is limited to slow fading channels. Beyond visual priors, advances in multimodal large language models (MLLMs) provide strong semantic understanding and text generation capabilities. Trained on massive multimodal datasets, these models can generate accurate and contextually rich textual descriptions of images, which in turn serve as powerful prompts for guiding generative models. In this paper, we leverage an MLLM to automatically produce descriptive prompts for transmitted images, thereby eliminating the need for manual annotations. This integration ensures that the diffusion model is guided by reliable semantic cues, enabling reconstructions that are not only perceptually realistic but also semantically faithful. In this paper, we applies diffusion sampling in the pixel space, using the JSCC-decoded blurry image together with textual prompts as conditional inputs to a pre-trained Stable Diffusion model equipped with ControlNet. This design enables the recovery of high-quality, semantically consistent images under ultra-low bitrate conditions without retraining the diffusion model. Although the JSCC-decoded image lacks fine details, it still preserves coarse spatial structures, color distributions, and global layout information that are critical for faithful reconstruction. By feeding this image into the diffusion model through the ControlNet conditioning mechanism, our framework effectively maintains the geometric and chromatic consistency of the original scene. This structural guidance complements the high-level semantic constraints provided by textual prompts, forming

Read the complete text