Feature Reconstruction Aided Federated Learning for Image Semantic Communication

Yoon Huh 1 2 , Bumjun Kim 1 2 , and Wan Choi 1 2 · 2025

Read Feature Reconstruction Aided Federated Learning for Image Semantic Communication by Yoon Huh 1 2 , Bumjun Kim 1 2 , and Wan Choi 1 2 (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2508-02048.txt.

Introduction

Yoon Huh, Bumjun Kim, and Wan Choi wrote this 2025 paper on keeping image-sending AI systems up to date. These systems send the meaning of an image rather than raw pixels, but their shared knowledge goes stale over time. Retraining them is hard when the training data sits on many separate devices.

The team propose FedSFR, a federated learning method in which devices train locally and share updates. Some devices send compact feature vectors instead of full updates, saving channel space. A rebuild step at the central server steadies training and lifts image quality. Tests beat rival methods, and the authors also prove how fast their method settles.

Catalog record

CREATORYoon Huh 1 2 , Bumjun Kim 1 2 , and Wan Choi 1 2
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE37 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2508-02048

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Feature Reconstruction Aided Federated Learning for Image Semantic Communication Thanks: This work was supported by Institute of Information & communications Technology Planning & Evaluation (IITP) grant funded by the Korea government(MSIT). (No.RS-2024-00398948, Next Generation Semantic Communication Network Research Center) Yoon Huh 1 2 , Bumjun Kim 1 2 , and Wan Choi 1 2 Abstract Research in semantic communication has garnered considerable attention, particularly in the area of image transmission, where joint source-channel coding (JSCC)-based neural network (NN) modules are frequently employed. However, these systems often experience performance degradation over time due to an outdated knowledge base, highlighting the need for periodic updates. To address this challenge in the context of training JSCC modules for image transmission, we propose a federated learning (FL) algorithm with semantic feature reconstruction (FR), named FedSFR. This algorithm more efficiently utilizes the available communication capacity by allowing some of the selected FL participants to transmit smaller feature vectors instead of local update information. Unlike conventional FL methods, our approach integrates FR at the parameter server (PS), stabilizing training and enhancing image transmission quality. Experimental results demonstrate that the proposed scheme significantly enhances both the stability and effectiveness of the FL process compared to other algorithms. Furthermore, we mathematically derive the convergence rate to validate the improved performance. {mnihy621, eithank96, wanchoi}@snu.ac.kr I Introduction To enable more spectral efficient communications in sixth-generation (6G) communication, which is evolving into task-oriented communication [ 1 ] , semantic communication [ 2 ] is gaining significant attention. This approach efficiently transmits only task-relevant data and effectively manages various types of source data. One notable application is the image reconstruction task, which is the focus of this paper and is referred to as image semantic communication (ISC). Semantic communication often relies on joint source-channel coding (JSCC) techniques [ 3 ] . Typically, JSCC is implemented using neural networks (NNs) and employs an autoencoder structure, consisting of an encoder and a decoder, in ISC. It optimizes source and channel coding together, reducing redundancy in the transmitted data and improving overall efficiency. However, over time, the underlying distribution of the knowledge base [ 2 ] , represented by the training dataset, gradually shifts 1 1 1 While continual learning techniques help mitigate distribution shifts like catastrophic forgetting, our focus in this paper is on developing an FL framework rather than addressing continual learning issues. . This shift leads to a steady decline in performance, especially in dynamic environments where data distributions change. As a result, continuous updates to the model are essential to maintain optimal performance. Federated learning (FL) has emerged as a vital solution to these challenges [ 4 , 5 ] , which facilitates the periodic updating of NN-based JSCC models by aggregating knowledge from multiple distributed sources while preserving data privacy. In this framework, ISC users act as FL clients, each equipped with both a JSCC encoder and decoder, which serve as the transmitter and receiver, respectively, during ISC and together form the local model during FL. The parameter server (PS) aggregates the global JSCC model as in conventional FL, however, ISC occurs exclusively among the clients, excluding the PS, after FL. Nevertheless, FL also faces significant challenges [ 6 ] , particularly in environments with limited communication resources. The process of aggregating updates from multiple clients can lead to substantial communication overhead since they transmit the whole local model to the PS. To alleviate these concerns, sparsification techniques [ 7 ] have been employed, where only the most significant values in local models are transmitted. Additionally, error feedback strategies [ 8 ] , i.e., storing the error from sparsification and incorporating them into future model updates, are used to compensate for the information loss due to sparsification to improve the overall efficiency of the FL process in capacity-constrained networks. Building on this, several studies have explored FL frameworks specifically designed for semantic communication. For example, the authors of [ 4 ] proposed an FL framework that optimizes JSCC model for image classification tasks. Some clients send their local models, while the others transmit output feature vectors from the encoder, along with the corresponding class information, to the PS. The PS updates the previous global model through knowledge distillation using the received feature vectors and ground-truth labels, and then aggregates the updated model with the received local models. However, since the class label in classification tasks corresponds to the original image for image reconstruction tasks, this approach incurs a privacy risk and is thus not applicable in ISC. In contrast, [ 5 ] extended the FL framework to handle image reconstruction. Instead of transmitting the local models, clients exchange output feature vectors from the encoder and intermediate feature vectors from the decoder with the PS. The model components corresponding to these feature vectors are updated via distillation at both the clients and the PS, while the remaining parts are optimized locally at the clients. Since the PS relies solely on feature vectors without direct local model exchanges via FL, forming a common global model becomes challenging. As a result, the encoder produces inconsistent feature vectors across the clients, inherently limiting performance without leveraging the benefits of local model averaging. To address these challenges, we propose an advanced FL algorithm with semantic feature reconstruction (FedSFR). Our key contributions are as follows: • We introduce FedSFR, which divides clients into two groups: one sends local model updates as in conventional FL, while the other transmits compressed feature vectors from the JSCC encoder to the PS. After the model aggregation, the PS further refines the model through additional feature reconstruction (FR) learning, where the received feature vectors are sequentially processed through the decoder and then the encoder—differing from client-side processing. This approach combines the benefit of autoencoder-based ISC, by reversing the model’s data processing order, with that of local model averaging in FL, leading to enhanced feature representation and improved model consistency. • FedSFR enables clients to adaptively select between transmitting conventional local updates or compact feature vectors based on their channel quality. This adaptive mechanism significantly reduces communication overhead while ensuring stable and efficient training. • We provide a comprehensive analysis of the proposed method, including its convergence bound, convergence rate, and the ability to mitigate sparsification errors through server-side updates. • Extensive experiments on both low-resolution and high-resolution datasets demonstrate that FedSFR substantially improves training stability and convergence speed compared to baseline methods, validating both its effectiveness and our theoretical analysis. II System Model This section presents the foundational framework for semantic communication and federated learning, which are central concepts explored in this paper. After optimizing the NN parameters for the JSCC encoder and decoder at the client through the wireless FL process, ISC is carried out between clients. In this setup, semantic communication users participates in FL as clients, working collaboratively with the PS. The commonly used notations are summarized in TABLE I . TABLE I: Notations about FL at global iteration t . Notation Description \boldsymbol{w}^{\scriptstyle(t)} Global model to be downloaded at the clients \boldsymbol{w}^{\scriptstyle(t+\frac{1}{2})} Global model to be updated at the PS \boldsymbol{w}_{k}^{\scriptstyle(t,e)} Local model of client k at local iteration e \boldsymbol{w}_{s}^{\scriptstyle(t+\frac{1}{2},e)} Server model at server iteration e \mathbf{g}_{k}^{\scriptstyle(t)} Local update information at client k \mathbf{m}_{k}^{\scriptstyle(t)} Error memory

Read the complete text