Adaptive Semantic Token Communication for Transformer-based Edge Inference

Alessio Devoto, Jary Pomponi, Mattia MerluzziPaolo Di Lorenzo, Simone Scardapane · 2025

Read Adaptive Semantic Token Communication for Transformer-based Edge Inference by Alessio Devoto, Jary Pomponi, Mattia MerluzziPaolo Di Lorenzo, Simone Scardapane (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2505-17604.txt.

Introduction

Alessio Devoto and four co-authors present a 2025 arXiv framework for squeezing AI perception through thin wireless links. Their scene is an edge device with little power that must send just enough upward for a server to detect objects, over channels that fade and surge. The design turns inputs into compact semantic tokens, high-level features rather than raw pixels, and sends only the informative ones.

A transformer refines the tokens, and a joint coding module compresses them further, flexibly trading token count and token size against conditions. A control algorithm grounded in Lyapunov optimization steers those choices in real time, balancing bandwidth against task accuracy. Across experiments the system beat existing baselines under shifting channel quality. The authors frame semantic token communication as a foundation for AI-native networks, where what travels is meaning, metered to the moment.

Catalog record

CREATORAlessio Devoto, Jary Pomponi, Mattia MerluzziPaolo Di Lorenzo, Simone Scardapane
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE63 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2505-17604

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Adaptive Semantic Token Communication for Transformer-based Edge Inference Alessio Devoto, Jary Pomponi, Mattia MerluzziPaolo Di Lorenzo, Simone Scardapane Abstract This paper presents an adaptive framework for edge inference based on a dynamically configurable transformer-powered deep joint source channel coding (DJSCC) architecture. Motivated by a practical scenario where a resource constrained edge device engages in goal oriented semantic communication, such as selectively transmitting essential features for object detection to an edge server, our approach enables efficient task aware data transmission under varying bandwidth and channel conditions. To achieve this, input data is tokenized into compact high level semantic representations, refined by a transformer, and transmitted over noisy wireless channels. As part of the DJSCC pipeline, we employ a semantic token selection mechanism that adaptively compresses informative features into a user specified number of tokens per sample. These tokens are then further compressed through the JSCC module, enabling a flexible token communication strategy that adjusts both the number of transmitted tokens and their embedding dimensions. We incorporate a resource allocation algorithm based on Lyapunov stochastic optimization to enhance robustness under dynamic network conditions, effectively balancing compression efficiency and task performance. Experimental results demonstrate that our system consistently outperforms existing baselines, highlighting its potential as a strong foundation for AI native semantic communication in edge intelligence applications. * * footnotetext: Equal contribution I Introduction Recently, there was a surge of interest in semantic and goal-oriented communications, establishing this paradigm as crucial for the development of 6G AI-native communication networks [ 1 , 2 ] . In contrast to traditional communication systems, which focus on efficiently transmitting the data in the form of raw bits, semantic communication emphasizes understanding the meaning and intent behind the data, allowing for more efficient information exchange tailored to the specific goals of the communication task. This prioritizes the meaning and purpose behind transmitted data instead of the transmission itself and reduces the need for exact bit-level reconstructions, thereby offering gains in efficiency and adaptability, particularly in dynamic or resource constrained environments [ 3 , 4 , 5 , 6 , 7 ] . A prominent use case for semantic communication is edge inference, where distributed edge devices collect high dimensional sensor data and must convey only the most task relevant information to a nearby server for real time decision making [ 7 ] . In such scenarios, goal-oriented semantic communication enables these devices to transmit compressed, high level representations that are sufficient for the target inference task, such as object detection or anomaly recognition, while significantly reducing communication overhead. This is especially important in wireless edge networks, where constraints on bandwidth, latency, and power demand intelligent mechanisms to select and transmit only semantically meaningful features aligned with the task objective. In this context, transformer-based neural networks, with their inherent attention mechanisms and contextual processing capabilities, serve as ideal candidates for extracting and compressing semantic information from data into a lower-dimensional representation space [ 8 ] . This representation consists of multiple vectors called tokens , each capturing different semantic aspects of the input. By treating tokens as discrete transmission units, the system can selectively transmit only the most semantically relevant tokens while discarding less informative ones; this property has enabled the development of semantic token-based communication systems that adapt to dynamic channel conditions and application requirements, see, e.g. [ 9 , 4 , 10 ] . I-A Related works Edge inference comes with the drawback of an ad-hoc deployment of computing resources, which might be more costly in terms of economic and environmental aspects, if compared to a shared infrastructure. However, it allows end devices and service consumers to keep data local, thus promoting privacy and secrecy, and retrieving results within strict latency constraints, without the need of reaching distant central clouds. To achieve a positive balance between benefits and drawbacks, the joint optimization of communication and computing segments is a fundamental brick [ 7 ] . In general, edge inference can take place: i) locally at an end device, ii) fully offloaded to an edge server (or, Mobile Edge Host-MEH), or iii) partially performed at the two end sides [ 11 ] . The first solution (full local inference) typically requires model optimization techniques such as pruning or quantization [ 12 ] , to be able to efficiently run a full (deep) model locally in a resource limited device. This comes with no communication overhead and full secrecy as no data is exposed, however at a cost of local computation and possibly reduced inference accuracy. The second solution (full offloading) [ 13 , 14 ] relieves the device from any processing, but it can lead to high communication overhead and data exposure. In the third case (partial offloading), which is the most flexible one, as also highlighted in [ 15 , 5 ] , part of the computation is performed locally to extract relevant (possibly lower dimensional) features. This is helpful if the edge server is temporary unavailable, or to reduce the communication overhead needed to offload the inference task. Several methodologies have been proposed to enable local feature extraction with the goal of minimizing communication overhead while maintaining acceptable task performance, typically measured in terms of accuracy. For a comprehensive overview, the reader is referred to the recent survey in [ 16 ] . One straightforward approach is to split a deep neural network (DNN) between a local device and an edge server, a technique commonly referred to as DNN splitting [ 11 ] . This splitting can be adapted dynamically based on channel conditions and the availability of radio and computational resources [ 15 , 17 ] . From the perspective of task performance (e.g., achieving a target classification accuracy), prior work has shown that features extracted at different layers—corresponding to different splitting points—exhibit varying levels of robustness to channel noise and transmission errors [ 5 ] . This observation highlights the importance of selecting optimal split points depending on system constraints. When scaling to powerful architectures such as Vision Transformers, the opportunity for DNN splitting is further enriched by mechanisms like token pruning, which can significantly reduce communication costs by discarding less informative tokens. Moreover, when the model is trained end-to-end with the wireless channel modeled as a differentiable layer, DNN splitting effectively performs joint source and channel coding. This results in a learned Deep Joint Source-Channel Coding (DJSCC) scheme that simultaneously achieves compression and error resilience, optimized for the specific communication environment. Recent advancements in wireless semantic communication have highlighted DJSCC as a promising approach, see, e.g., [ 18 , 19 ] . DJSCC systems integrate source and channel coding into a unified process over wireless channels, employing deep neural networks to tailor transmission strategies to specific application requirements. By modeling the communication channel as a differentiable layer within the network, these systems enable end-to-end learning that jointly adapts both source and channel coding, leading to improved performance across key metrics. DJSCC methods have shown clear advantages over traditional coding schemes, particularly in semantic communication settings, by exhibiting graceful performance degradation below information-theoretic signal-to-noise ratio (SNR) thresholds and avoiding the sharp declines characteristic of conventional approaches [ 20 , 21 ] . This makes DJSCC a compelling choice for practical scenarios characterized by

Read the complete text