Goal-Oriented Framework for Optical Flow-based Multi-User Multi-Task Video Transmission

Yujie Xu, Shutong Chen, Nan Li, Yansha Deng, Jinhong Yuan, and Robert Schober · 2026

Read Goal-Oriented Framework for Optical Flow-based Multi-User Multi-Task Video Transmission by Yujie Xu, Shutong Chen, Nan Li, Yansha Deng, Jinhong Yuan, and Robert Schober (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2603-19995.txt.

Introduction

Xu and co-authors design a goal-oriented system for sending video to many users for many tasks. Sending full video wastes scarce wireless resources. Their OF-GSC frame selects key semantic content based on optical flow and image patches. A transformer decoder at the receiver rebuilds the video and also classifies it. A learning-based allocator shares bandwidth among users to cut total sending time. For reconstruction, video quality rose 13.47 percent over a DeepJSCC baseline. Classification matched a strong model while using only a quarter of the data.

Catalog record

CREATORYujie Xu, Shutong Chen, Nan Li, Yansha Deng, Jinhong Yuan, and Robert Schober
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE68284
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2603.19995

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Goal-Oriented Framework for Optical Flow-based Multi-User Multi-Task Video Transmission An early version of this work was presented in part at the IEEE International Conference on Communications (ICC), June 2025 [ 36 ] . Yujie Xu, Shutong Chen, Nan Li and Yansha Deng are with the Department of Engineering, King’s College London, London WC2R 2LS, U.K. (e-mail: { yujie.xu, shutong.chen, nan.3.li, yansha.deng}@kcl.ac.uk); Jinhong Yuan is with the School of Electrical Engineering and Telecommunications at University of New South Wales (e-mail: [email protected]); Robert Schober is with the Institute for Digital Communications, FriedrichAlexander-Universität Erlangen-Nürnberg, 91058 Erlangen, Germany (e-mail:[email protected])(Corresponding author: Yansha Deng). Yujie Xu, Shutong Chen, Nan Li, Yansha Deng, Jinhong Yuan, and Robert Schober Abstract Efficient multi-user multi-task video transmission is an important research topic within the realm of current wireless communication systems. To reduce the transmission burden and save communication resources, we propose a goal-oriented semantic communication framework for optical flow-based multi-user multi-task video transmission (OF-GSC). At the transmitter, we design a semantic encoder that consists of a motion extractor and a patch-level optical flow-based semantic representation extractor to effectively identify and select important semantic representations. At the receiver, we design a transformer-based semantic decoder for high-quality video reconstruction and video classification tasks. To minimize the communication time, we develop a deep deterministic policy gradient (DDPG)-based bandwidth allocation algorithm for multi-user transmission. For video reconstruction tasks, our OF-GSC framework achieves a significant improvement in the received video quality, as evidenced by a 13.47% increase in the structural similarity index measure (SSIM) score in comparison to DeepJSCC. For video classification tasks, OF-GSC achieves a Top-1 accuracy slightly surpassing the performance of VideoMAE with only 25% required data under the same mask ratio of 0.3. For bandwidth allocation optimization, our DDPG-based algorithm reduces the maximum transmission time by 25.97% compared with the baseline equal-bandwidth allocation scheme. I Introduction The escalating demand for high-quality videos, fueled by the proliferation of smart devices and visual applications such as live video streaming, video conferencing, and video surveillance, has made efficient multi-user multi-task video transmission an important task in current wireless networks [ 6 , 27 , 7 , 1 ] . However, the demand for efficient and reliable multi-task video transmission has imposed considerable pressure on existing communication systems due to the tremendous data volume and high fidelity requirements of high resolution videos [ 31 , 33 ] . Traditional video transmission methods primarily focus on reducing data volume through various compression techniques. In the early 1990s, the adoption of JPEG for video frames gave rise to Motion JPEG (M-JPEG). To alleviate the blocking artifacts and compression inefficiency of M-JPEG, JPEG2000 was introduced in the early 2000s and subsequently applied to intra-frame video coding, leading to M-JPEG2000. More advanced codecs, such as H.264 [ 12 ] in the early 2000s and H.265 [ 11 ] in the mid 2010s, were subsequently developed to further improve compression efficiency by exploiting temporal redundancy through inter-frame prediction and motion compensation. However, these approaches remain fundamentally constrained by bit-level optimization based on Shannon’s communication framework, which limits their ability to efficiently support high-quality video transmission in stringent communication environments [ 8 , 39 ] . To address this challenge, goal-oriented semantic communication (GSC) has emerged as a promising method, which may lead to a revolution in wireless network design [ 34 , 22 , 38 , 15 , 4 ] . By extracting and transmitting only goal-relevant semantic representations, GSC aligns communication with task requirements while improving scalability and sustainability [ 34 , 22 , 38 , 15 ] . Most existing GSC studies focus on single-domain, single-modality tasks, and thus have not yet addressed the multi-user, multi-task video transmission requirements in future 6G scenarios [ 4 ] . Early GSC works for video predominantly modeled videos as sequences of independent images [ 13 ] . The DeepJSCC algorithm [ 2 ] leverages Deep Neural Networks (DNNs) to directly map frames to channel symbols and demonstrate robustness in high-resolution image transmission over wireless communication channels. Nevertheless, DeepJSCC suffers significant performance degradation in low signal-to-noise ratio (SNR) scenarios due to its limited capability to dynamically adjust the transmission based on feature importance [ 20 ] . To address this limitation, attention mechanisms were integrated into DeepJSCC to enable dynamic emphasis on salient image regions by prioritizing transmission of perceptually important areas and suppressing less relevant content, thereby improving reconstructed image quality [ 35 ] . Recently, the authors in [ 16 ] introduced a pixel-level 3D attention map to capture spatial semantic importance across video frames. However, these approaches still rely on frame-by-frame representations, generating independent latent representations for each frame without explicitly exploiting inter-frame temporal correlations, resulting in limited transmission efficiency for video data. To better exploit temporal correlations for efficient video transmission, VISTA [ 18 ] was designed to only transmit background segments, behavior-related segments, and semantic location graphs that describe the relationships among detected dynamic objects, thereby incorporating temporal correlations directly. To exploit temporal correlations more precisely, a generative AI–based GSC framework [ 17 ] was proposed that selects motion-aware keyframes, transmits compact semantic representation, and reconstructs the full video via semantic reconstruction and frame interpolation. However, motion modeling in this framework is primarily conducted at the pixel level, which limits its ability to capture structured and coherent temporal dynamics. To further exploit temporal correlations and move beyond pixel-level representations, optical flow has emerged as a natural and effective approach for explicitly capturing inter-frame motion [ 21 , 37 ] . In [ 21 ] , an end-to-end deep video compression framework (DVC) applies a DNN to encode optical flow between consecutive frames into a compact latent representation at the encoder and reconstructs these two consecutive frames using generated representation from encoder together with the first frame, enabling more compact representation of motion information to reduce overall bit rate while preserving reconstruction fidelity. However, this design does not account for the distortion caused by wireless channels during end-to-end training, leading to limited robustness when deployed in wireless communication system spaces. In [ 37 ] , deep learning enabled semantic communication systems for video transmission (DL-SC) adopted a similar optical flow-based reconstruction pipeline as DVC while introducing feature fusion and denoising modules to improve transmission quality under low SNR conditions by treating latent vectors as semantic representations. Nevertheless, existing optical flow-based approaches only modeled inter-frame redundancy and did not jointly consider intra-frame redundancy. Fig. 1 : Optical flow-based goal-oriented semantic communication framework. In practical wireless video transmission scenarios, it is insufficient to consider the video reconstruction task alone, as the transmitted video is also expected to support other tasks, such as video classification, action recognition, and object detection. Current research on multi-task video in computer vision area primarily explores unified frameworks for jointly addressing multiple visual objectives [ 5 , 9 , 19 ] . The scalable video coding for humans and machines [ 5 ] enables a single video stream to support both reconstruction and object detection by jointly considering task requirements and visual fidelity constraints in the coding process. However, this design imposes rigid structural constraints on representation allocation, which limits flexibility and coding efficiency. To improve flexibility in representation, learned scalable video coding framework [ 9 ]

Read the complete text