Robust Transmission of Punctured Text with Large Language Model-based Recovery

Sojeong Park 1 , Hyeonho Noh 2 , and Hyun Jong Yang 2 · 2025

Read Robust Transmission of Punctured Text with Large Language Model-based Recovery by Sojeong Park 1 , Hyeonho Noh 2 , and Hyun Jong Yang 2 (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2503-14831.txt.

Introduction

Sending text through a lossy channel often means some letters never arrive. Sojeong Park, Hyeonho Noh, and Hyun Jong Yang send only a few letters on purpose in 2025, and let a large language model fill the gaps at the far end. Their wager is that meaning can ride on a small share of the whole.

A smart picker chooses which letters to send, based on how much each one helps the model rebuild the text. That pick beats a random pick in tests. The method holds its form across data sets and tasks, where older learned schemes tend to bend to their training set. It also beats the classic bit-by-bit route. The work joins a push for communication that sends sense, not just symbols.

Catalog record

CREATORSojeong Park 1 , Hyeonho Noh 2 , and Hyun Jong Yang 2
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE26 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2503-14831

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Robust Transmission of Punctured Text with Large Language Model-based Recovery Manuscript received December 1, 2012; revised August 26, 2015. Corresponding author: M. Shell (email: http://www.michaelshell.org/contact.html). Sojeong Park 1 , Hyeonho Noh 2 , and Hyun Jong Yang 2 Abstract With the recent advancements in deep learning, semantic communication which transmits only task-oriented features, has rapidly emerged. However, since feature extraction relies on learning-based models, its performance fundamentally depends on the training dataset or tasks. For practical scenarios, it is essential to design a model that demonstrates robust performance regardless of dataset or tasks. In this correspondence, we propose a novel text transmission model that selects and transmits only a few characters and recovers the missing characters at the receiver using a large language model (LLM). Additionally, we propose a novel importance character extractor (ICE), which selects transmitted characters to enhance LLM recovery performance. Simulations demonstrate that the proposed filter selection by ICE outperforms random filter selection, which selects transmitted characters randomly. Moreover, the proposed model exhibits robust performance across different datasets and tasks and outperforms traditional bit-based communication in low signal-to-noise ratio conditions. I Introduction Traditional communication utilizes source coding to convert entire information into bits through lossless compression. In contrast, semantic communication, despite being a form of lossy compression, transmits only task-oriented semantic features to increase data rates while effectively maintaining task performance [ 1 ] . In prior research, semantic features are obtained using learning-based models [ 2 , 3 , 4 , 5 , 6 ] . Due to the learning-based feature extraction, semantic communication faces limitations in data robustness, similar to typical machine learning frameworks. Specifically, when a pre-trained model encounters transmitted data that was not used during the training phase, the performance degrades significantly, even in relatively simple cases of text transmission. The upper part of Fig. 1 illustrates this problem: The conventional semantic communication model, DeepSC [ 2 ] , is trained on the Europarl dataset [ 7 ] . It utilizes transformer-based architectures to extract semantic features through word embeddings. Consequently, the distortion of features caused by channel noise leads to word-level changes for words not included in the training dataset. For instance, words ‘Steven’ and ‘caramel’, which are not derived from the training dataset, are inaccurately reconstructed by DeepSC. As a result, for a Q&A task with a question like ‘Who wants to have the cake?’, the model will produce a completely incorrect answer. Therefore, it is essential to create a robust transmission model for general tasks, regardless of the data. In this correspondence, we focus on a text transmission model that operates independently of the transmitted data. We propose a novel character-level text transmission model that transmits selected characters and reconstructs the full text using a large language model (LLM) at the receiver. If a missing character causes ambiguity between multiple possible words, the LLM may struggle to determine the correct one, potentially leading to reconstruction errors. To mitigate distortions, the proposed algorithm strategically selects characters that maximize LLM reconstruction accuracy. Furthermore, our model excludes the training process and instead leverages a pre-trained LLM that has been trained on a large volume of data to reduce dependency on specific training datasets. This extensive training enables the LLM to excel at understanding semantic information and context, even with typos or missing characters [ 8 ] . We employ GPT-3.5 Turbo [ 9 ] as a pre-trained LLM to infer the context from the received characters and reconstruct the omitted characters. The ultimate goal is to increase the transmission data rate while achieving higher task performance. Specifically, we focus on text transmission, addressing two major text-related tasks: the text reconstruction and Q&A tasks. To this end, our contribution is as follows: • Unlike existing transmission models that perform word-level feature extraction, the proposed model selectively transmits characters based on the proposed algorithm and reconstructs them at the receiver. To enhance recovery performance, we introduce an important character extractor (ICE) that determines which characters to be transmitted. • In conventional semantic communication, decoding relies on training process, resulting in strong dataset dependency and limiting its applicability to specific environments. In contrast, the proposed model utilizes a pre-trained LLM in the decoding process to reconstruct omitted characters, effectively reducing dataset dependency. Fig. 1 : Comparison of the conventional semantic communication model and the proposed model. Instead of extracting features based on word embeddings, such as transformer models, this correspondence is the first to propose a compression method through character omission combined with LLM-based recovery. As a result, the proposed model demonstrates significantly more robust performance than conventional semantic communication models, regardless of the dataset or task. Compared to a traditional communication model, it uses fewer data symbols under the same transmission rate, allowing for more symbols for channel coding. This enables powerful error correction, resulting in better performance in low signal-to-noise ratio (SNR). II System Model and Problem Formulation II-A System Model This correspondence considers an uplink single-input single-output system, with single antennas at the user equipment (UE) and base station (BS). As shown in Fig. 1 , ICE selects the transmitted characters, \mathbf{p} , from the original text, \mathbf{t} , and maps them into complex symbols, \mathbf{x} . After transmission, the received signal \mathbf{y} is decoded into the estimated text, \hat{\mathbf{t}} , using the missing character indicator and LLM. Specifically, the UE transmits text data, represented as \mathbf{t}=[w_{1},w_{2},\dots,w_{L}] , where w_{i} denotes the i -th word. Each word is composed of multiple characters, thus it can be expressed as w_{i}=[c^{i}_{1},c^{i}_{2},\dots,c^{i}_{l(w_{i})}] , where c^{i}_{j} represents the j -th character of w_{i} , and l(\cdot) indicates the length of the text. In ICE, transmitted characters are extracted using a filter selected from M predefined filters. The punctured text \mathbf{p} , which excludes some characters, can be written as \mathbf{p}=I_{\epsilon}(\mathbf{t}) (1) where I_{\epsilon}(\cdot) represents ICE with a compression ratio \epsilon=l(\mathbf{t})/l(\mathbf{p}) . We can express the punctured text as \mathbf{p}=[w^{\prime}_{1},w^{\prime}_{2},\dots,w^{\prime}_{L}] where w^{\prime}_{i} represents the i -th punctured word. If the k -th character of w_{i} is filtered out, then w^{\prime}_{i}=[c^{i}_{1},c^{i}_{2},\dots,c^{i}_{k-1},c^{i}_{k+1},\dots,c^{i}_{l(w_{i})}] , and for all i\in\{1,2,\dots,L\} , l(w^{\prime}_{i})\leq l(w_{i}) is satisfied. For example, when ‘filter 62’ is applied to ‘caramel’, the result is ‘cramel’. The selected filter index s is also transmitted, requiring only \left\lceil\log_{2}M\right\rceil bits. The details of ICE will be given in section III . Next, the complex symbols \mathbf{x}\in\mathbb{C}^{N\times 1} obtained through bit source and channel encoding can be represented as \mathbf{x}=B(\mathbf{p}) (2) where B(\cdot) is the source and channel encoder and N is the number of the transmitted symbols. when \mathbf{x} is transmitted, the received signal \mathbf{y}\in\mathbb{C}^{N\times 1} can be written as \mathbf{y}=h\mathbf{x}+\mathbf{n} (3) where h is a channel between the UE and BS, and \mathbf{n} is an additive white Gaussian noise (AWGN), which follows \mathcal{CN}(0,\sigma^{2}\mathbf{I}) . Using the estimated channel state information, the received signals can be recovered as \mathbf{\hat{x}} . At the receiver, the estimated symbols \mathbf{\hat{x}} are processed through bit source and channel decoding to obtain the recovered punctured text \hat{\mathbf{p}}=[\hat{w^{\prime}}_{1},\hat{w^{\prime}}_{2},\dots,\hat{w^{\prime}}_{L}] , represented as \hat{\mathbf{p}}=B^{-1}(\mathbf{\mathbf{\hat{x}}}) (4) where B^{-1}(\cdot) is the source and channel decoder. Then, the recovered i -th punctured word can be expressed as \hat{w^{\prime}}_{i}=[\hat{c}^{i}_{1},\hat{c}^{i}_{2},\dots,\hat{c}^{i}_{k-1},\hat{c}^{i}_{k+1},\dots,\hat{c}^{i}_{l(w_{i})}] where \hat{c}^{i}_{j} represents the recovered j -th character of w_{i} . Additionally, the selected filter index

Read the complete text