Large Language Model Enabled Multi-Task Physical Layer Network
Tianyue Zheng, Graduate Student Member, IEEE, Linglong Dai, Fellow, IEEE · 2024
Read Large Language Model Enabled Multi-Task Physical Layer Network by Tianyue Zheng, Graduate Student Member, IEEE, Linglong Dai, Fellow, IEEE (2024) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2412-20772.txt.
Description
Abstract The advance of Artificial Intelligence (AI) is continuously reshaping the future 6G wireless communications. Particularly, the development of Large Language Models (LLMs) offers a promising approach to effectively improve the performance and generalization of AI in different physical-layer (PHY) tasks. However, most existing works finetune dedicated LLM networks for a single wireless communication task separately.
Catalog record
| CREATOR | Tianyue Zheng, Graduate Student Member, IEEE, Linglong Dai, Fellow, IEEE |
|---|---|
| YEAR | 2024 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 60 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2412-20772
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Large Language Model Enabled Multi-Task Physical Layer Network Tianyue Zheng, Graduate Student Member, IEEE, Linglong Dai, Fellow, IEEE Abstract The advance of Artificial Intelligence (AI) is continuously reshaping the future 6G wireless communications. Particularly, the development of Large Language Models (LLMs) offers a promising approach to effectively improve the performance and generalization of AI in different physical-layer (PHY) tasks. However, most existing works finetune dedicated LLM networks for a single wireless communication task separately. Thus performing diverse PHY tasks requires extremely high training resources, memory usage, and deployment costs. To solve the problem, we propose a LLM-enabled multi-task PHY network to unify multiple tasks with a single LLM, by exploiting the excellent semantic understanding and generation capabilities of LLMs. Specifically, we first propose a multi-task LLM framework, which finetunes LLM to perform multi-user precoding, signal detection and channel prediction simultaneously. Besides, multi-task instruction module, input encoders, as well as output decoders, are elaborately designed to distinguish different tasks. The proposed design allows different wireless data types to be well aligned with the LLM input format. Moreover, low-rank adaptation (LoRA) is utilized for LLM fine-tuning. To reduce the memory requirement during LLM fine-tuning, a LoRA fine-tuning-aware quantization method is introduced. Extensive numerical simulations are also displayed to verify the effectiveness of the proposed method. I Introduction The deep integration of Artificial Intelligence (AI) with wireless communications is one of the key features of Sixth-Generation (6G) communications [ 1 ] . With the development of 6G communications, the application of AI becomes essential to manage the explosively-growing data service. Attributed to the strong feature extraction capability, AI, especially deep learning (DL), has demonstrated great potential in a wide range of physical-layer (PHY) communication tasks [ 2 ] , including CSI feedback [ 3 ] , channel estimation [ 4 ] , channel prediction [ 5 ] , signal detection [ 6 ] , etc. The AI Radio Access Network (AI-RAN) is anticipated to offer reduced latency, improved bandwidth, spectrum efficiency, and coverage. Moreover, significant efforts have been made by 3rd Generation Partnership Project (3GPP) to standardize DL in wireless networks recently [ 7 ] . Despite the significant progress, existing DL empowered methods still face some fundamental issues that limit their applications in practical communication networks. First, with the dramatically increasing channel dimension and rapidly changing wireless environment in 6G applications, it is difficult for existing DL methods to comprehensively recognize patterns from complicated data distribution, due to their small model size and simple network structure. This insufficiency restricts their ability to provide reliable solutions in dynamic real-world scenarios. Secondly, existing DL methods exhibit a poor generalization capability to different wireless environments. For instance, a channel prediction model trained in an indoor scenario usually requires retraining when the wireless environment is changed to urban. Fortunately, recent advancements in large language models (LLMs) [ 8 , 9 , 10 ] have provided a radical solution to the challenges of existing DL methods. Bearing a huge amount of parameters, LLMs possess the ability to capture universal knowledge, which has demonstrated impressive language understanding and generation capabilities to various tasks in different domains. Very recently, several initial studies [ 11 , 12 , 13 , 14 ] have been conducted since 2024 to leverage LLMs for boosting the performance of PHY communication systems. To be specific, in [ 11 ] LLM is adopted to perform power allocation using a few-shot learning approach. In this method, the channel gains and the corresponding transmit power strategies are provided as the input of LLM, also called “prompt” in the society of AI. The paper [ 11 ] demonstrates that LLM can automatically understand the principle of water-filling based optimal power allocation without any retraining. The authors in [ 12 ] effectively utilize LLMs to enhance AI-based CSI feedback performance in various scenarios. They incorporate the channel distribution as a prompt within the decoder to further enhance channel reconstruction quality. Besides, in order to obtain more accurate channel prediction and improve the generalization capability, the authors in [ 13 ] propose a LLM-driven channel prediction approach. [ 14 ] also unleashes the strength of LLMs for time series forecasting to improve the robustness of beam prediction. Moreover, a few review papers [ 15 , 16 , 17 , 18 , 19 ] have explored the potential transformative impact of LLMs and provide envisions for LLM-aided wireless communications. However, most existing works finetune dedicated LLM networks for a single wireless communication task such as the previously mentioned channel prediction and CSI feedback, etc. In reality, wireless communication systems involve a multitude of tasks, each with distinct requirements that necessitate the adoption of LLMs. Given the huge model size of LLM, designing and training dedicated models for each task separately will lead to extremely high computational complexity, memory usage, and deployment costs. To solve the problem, we propose a LLM-enabled multi-task PHY network to unify multiple tasks with a single LLM, by leveraging the excellent language understanding and generation capabilities of LLMs. The main contributions of this paper are summarized as follows 1 1 1 Simulation codes will be provided to reproduce the results in this paper: http://oa.ee.tsinghua.edu.cn/dailinglong/publications/publications.html . . • We propose a multi-task LLM framework for PHY communications. The framework enables us to input different task requirements to LLMs with natural language, and explore the global feature extraction ability of LLMs to execute multiple tasks within one network. Particularly, this work focuses on three of the most typical tasks in PHY communications, multi-user precoding, signal detection, and channel prediction. The proposed framework can also accommodate any other PHY tasks. • For the design of the proposed framework, dedicated modules are proposed to adapt the LLMs for multiple tasks in the wireless domain. It mainly consists of two components: multi-task instructions, and task-specific encoders and decoders. Specifically, first, in order to distinguish and cope with different tasks and different data formats of the tasks, we elaborately design multi-task instructions as prompts of LLM; secondly, to bridge the gap between the feature of wireless data and that of the LLM, task-specific encoders and decoders are proposed to adapt the text-based pre-trained LLM. • For the fine-tuning of the proposed framework, we introduce low-rank adaptation (LoRA) [ 20 ] , which freezes the pre-trained model weights and injects trainable low-rank matrices. Furthermore, to mitigate the computational and memory demands of the proposed model, we employ a LoRA fine-tuning-aware quantization method [ 21 ] that simultaneously quantizes an LLM and finds a proper low-rank initialization for LoRA fine-tuning. • Extensive simulation experiments have been conducted to verify the effectiveness of the proposed method, which achieves comparable performance with the dedicated designed network for each task. In addition, the LoRA fine-tuning-aware quantization method achieves 75% storage reduction during fine-tuning with almost no performance degradation. Moreover the proposed multi-task instruction effectively accelerates the convergence and improve the performance. The rest of the paper is organized as follows. Section II introduces the system model. Then the three selected tasks, namely multi-user precoding, signal detection, and channel prediction, are formulated respectively. In Section III, the design of the proposed multi-task PHY LLM is illustrated in detail. Besides, Section IV elaborates the fine-tuning strategy of the proposed multi-task