Semantic Communication with Hopfield Memories

Karim Nasreddine 1 , Christo Kurisummoottil Thomas 2 , and Walid Saad 3 · 2025

Read Semantic Communication with Hopfield Memories by Karim Nasreddine 1 , Christo Kurisummoottil Thomas 2 , and Walid Saad 3 (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2511-10302.txt.

Introduction

Karim Nasreddine, Christo Kurisummoottil Thomas, and Walid Saad give semantic communication systems a shared memory, so old concepts can be reused instead of retransmitted. Learned semantic codecs usually freeze their representations, wasting bandwidth when content drifts gradually. Their framework stores semantic concepts in modern Hopfield networks held by both transmitter and receiver, and retrieves matches through soft attention.

Soft retrieval adjusts stored prototype weights smoothly as data evolves, avoiding the constant memory updates that hard quantization schemes suffer. The team optimizes encoder, decoder, and retrieval jointly for a reasoning capacity metric combining reuse rate and compression, and proves a rate-distortion-reuse trade-off. Video simulations saved about fourteen percent of bits on average, and up to seventy percent under gradual change.

Catalog record

CREATORKarim Nasreddine 1 , Christo Kurisummoottil Thomas 2 , and Walid Saad 3
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE37 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2511-10302

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Semantic Communication with Hopfield Memories Karim Nasreddine 1 , Christo Kurisummoottil Thomas 2 , and Walid Saad 3 Abstract Traditional joint source-channel coding employs static learned semantic representations that cannot dynamically adapt to evolving source distributions. Shared semantic memories between transmitter and receiver can potentially enable bandwidth savings by reusing previously transmitted concepts as context to reconstruct data, but require effective mechanisms to determine when current content is similar enough to stored patterns. However, existing hard quantization approaches based on variational autoencoders are limited by frequent memory updates even under small changes in data dynamics, which leads to inefficient usage of bandwidth. To address this challenge, in this paper, a memory-augmented semantic communication framework is proposed where both transmitter and receiver maintain a shared memory of semantic concepts using modern Hopfield networks (MHNs). The proposed framework employs soft attention-based retrieval that smoothly adjusts stored semantic prototype weights as data evolves that enables stable matching decisions under gradual data dynamics. A joint optimization of encoder, decoder, and memory retrieval mechanism is performed with the objective of maximizing a reasoning capacity metric that quantifies semantic efficiency as the product of memory reuse rate and compression ratio. Theoretical analysis establishes the fundamental rate-distortion-reuse tradeoff and proves that soft retrieval reduces unnecessary transmissions compared to hard quantization under bounded semantic drift. Extensive simulations over diverse video scenarios demonstrate that the proposed MHN-based approach achieves substantial bit reductions around 14 % 14\% on average and up to 70 % 70\% in scenarios with gradual content changes compared to baseline. Moreover, the simulation results show that the proposed solution achieves a superior performance under transmission errors where baseline hard quantization methods fail abruptly. Email: [email protected], [email protected], [email protected] I Introduction Traditional source coding techniques [ 1 ] , even those based on artificial intelligence (AI) [ 2 ] typically transmit data representations tightly coupled to pixel-level fidelity for images, videos and word-level fidelity for text, thereby requiring substantial bandwidth even when the receiver is interested in only high-level semantic content. Semantic communication (SC) [ 3 , 4 , 5 , 6 ] offers a fundamentally different paradigm by faithfully representing the meaning of the data by encoding the semantics into low-dimensional latent representations that preserve task-relevant information. A learned decoder at the receiver then reconstructs the original content from these compact semantic features [ 7 , 8 ] . In SC, the transmitter and receiver can share a semantic knowledge base which allows them to reconstruct the content using its local knowledge. This approach leverages advanced AI capabilities at the receiver to minimize over-the-air payload and achieve bandwidth efficiency beyond traditional joint source-channel coding (JSCC). However, such contextual knowledge-based SC faces practical challenges such as (a) scalability , as the number of stored semantic concepts grows with content diversity, (b) need for efficient knowledge retrieval and update mechanisms that can adapt to dynamic changes in data while balancing bandwidth efficiency and reconstruction quality, and (c) guarantees of semantic consistency between transmitter and receiver knowledge bases under transmission and memory errors. I-A Related works A number of recents works [ 9 , 10 , 11 ] explored context-based and memory-augmented SC [ 9 , 10 , 11 ] . The work in [ 9 ] introduced the concept of semantic memory for communication systems, but focused primarily on transformer-based architectures without analyzing the fundamental tradeoffs between memory size, retrieval mechanisms, and communication cost. In [ 10 ] , the authors proposed emergent semantic languages through signaling games, establishing shared knowledge bases between communicating agents. The work in [ 12 ] proposed a knowledge base-assisted SC framework where graph neural networks (GNNs) are used to embed the stored relations among semantic concepts. Recent works like [ 13 ] and [ 14 ] leverage vector quantized variational autoencoders (VQ-VAE) [ 15 ] to learn discrete semantic codebooks for compact indexing and transmission, demonstrating significant compression gains over traditional deep JSCC. While these approaches demonstrate the potential of shared context in enhancing bandwidth efficiency, they face critical limitations. For instance, GNN inference for large-scale knowledge graphs incurs prohibitive latency, while the need for building comprehensive semantic knowledge graphs requires extensive domain-specific ontologies and manual curation, limiting generalizability to dynamic or novel semantic concepts. Moreover, existing SC methods such as those in [ 9 , 10 , 11 ] do not account for knowledge base mismatches between transmitter and receiver, necessitating efficient memory retrieval and storage mechanisms. I-B Contributions The main contribution of this paper is to address the aforementioned challenges by proposing a novel memory-augmented SC framework that leverages modern Hopfield networks (MHNs) [ 16 ] as shared associative memories. Associative memories use similarity-based retrieval rather than exact matching, providing robust recall under gradual data changes and transmission errors without domain-specific ontologies. Their soft attention mechanism scales efficiently as the number of semantic concepts increases. Although Hopfield networks have been successfully applied to perception and reasoning tasks, their potential for wireless communication systems, particularly for managing the semantic knowledge bases under bandwidth constraints and channel impairments remains unexplored. In summary, our key contributions are: • We develop a novel SC framework that integrates a shared associative memory using MHNs into the encoder–decoder system. • We introduce a new reasoning capacity metric (inspired from [ 3 ] ) that quantifies semantic communication efficiency as “useful queries per bit.” This metric connects memory reuse to an intelligence-per-bit measure and allows evaluation of how effectively the system converts transmitted bits into correct semantic retrievals. • We provide a novel theoretical analysis characterizing the rate–distortion–reuse tradeoff. We derive conditions under which the Hopfield associative recall mechanism outperforms VQ’s hard quantization, particularly near decision boundaries where small semantic drifts would cause VQ codebooks to proliferate. • Extensive simulations conducted over diverse data characteristics demonstrate that MHN-driven SC achieves approximately 14\% reduction in average bits per frame compared to VQ at matched reconstruction quality (PSNR within \pm 0.2 dB). Under dynamic scenarios, MHNs achieves 70\% bit savings ( 68.7 bits/frame vs. 228.5 bits/frame) by maintaining sustained reuse through gradual content changes, while baseline methods trigger frequent unnecessary refreshes. II System Model We consider a point-to-point SC system consisting of a transmitter (Tx) and a receiver (Rx) connected over a wireless channel. The Tx aims to communicate a video sequence \{\bm{x}_{t}\}_{t=1}^{T} to the Rx, where t indexes the frame number and each frame \bm{x}_{t}\in\mathbb{R}^{H\times W\times 3} represents an RGB image of height H and width W . The system objective is to minimize the average bits per frame while ensuring that the Rx’s reconstruction \hat{\bm{x}}_{t} satisfies a target distortion constraint \mathbb{E}[D(\bm{x}_{t},\hat{\bm{x}}_{t})]\leq D_{0} , where D(\cdot,\cdot) is a perceptual distortion metric (e.g., mean squared error or structural similarity). II-A Encoder-decoder architecture A learned encoder f_{\theta}:\mathbb{R}^{H\times W\times 3}\to\mathbb{S}^{d-1} maps each frame to a d -dimensional unit-norm latent vector \bm{z}_{t}=f_{\theta}(\bm{x}_{t}) , where \mathbb{S}^{d-1}=\{\bm{z}\in\mathbb{R}^{d}:\|\bm{z}\|_{2}=1\} is the (d-1) -dimensional unit sphere and \theta are the encoder parameters. A decoder g_{\phi}:\mathbb{S}^{d-1}\to\mathbb{R}^{H\times W\times 3} reconstructs frames as \hat{\bm{x}}_{t}=g_{\phi}(\hat{\bm{z}}_{t}) from the received latent representation \hat{\bm{z}}_{t} , where \phi are the decoder parameters. The spherical latent space enables similarity-based retrieval via cosine similarity \langle\bm{z}_{i},\bm{z}_{j}\rangle . The shared associative memory at time t is a set of M_{t} prototype vectors \mathcal{M}_{t}=\{\bm{\mu}_{1},\ldots,\bm{\mu}_{M_{t}}\},\bm{\mu}_{j}\in\mathbb{S}^{d-1}, where each prototype \bm{\mu}_{j} represents a learned

Read the complete text