Prompt-Tuned LLM-Augmented DRL for Dynamic O-RAN Network Slicing

Fatemeh Lotfi, Hossein Rajoli, Fatemeh Afghah · 2025

Read Prompt-Tuned LLM-Augmented DRL for Dynamic O-RAN Network Slicing by Fatemeh Lotfi, Hossein Rajoli, Fatemeh Afghah (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2506-00574.txt.

Introduction

Fatemeh Lotfi, Hossein Rajoli, and Fatemeh Afghah blended language models with reinforcement learning for wireless slicing in 2025. Open radio access networks split one physical network into virtual slices for different services, and conditions shift constantly. Learning agents struggle because feedback arrives scattered and changes meaning as the network evolves.

A language model trained on network knowledge first turns raw signals like noise, power, and speed into grouped, meaningful state features. Short learnable prompts, tuned for the task, replace costly full retraining of the model. With several agents learning together, rewards rose faster and adaptation improved in their tests. Smarter state reading could make dynamic slicing cheaper to control.

Catalog record

CREATORFatemeh Lotfi, Hossein Rajoli, Fatemeh Afghah
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE33 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2506-00574

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Prompt-Tuned LLM-Augmented DRL for Dynamic O-RAN Network Slicing Thanks: This material is based upon work supported by the National Science Foundation under Grant Numbers CNS-2202972, CNS- 2318726, and CNS-2232048. Fatemeh Lotfi, Hossein Rajoli, Fatemeh Afghah Abstract Modern wireless networks must adapt to dynamic conditions while efficiently managing diverse service demands. Traditional deep reinforcement learning (DRL) struggles in these environments, as scattered and evolving feedback makes optimal decision-making challenging. Large Language Models (LLMs) offer a solution by structuring unorganized network feedback into meaningful latent representations, helping RL agents recognize patterns more effectively. For example, in O-RAN slicing, concepts like SNR, power levels and throughput are semantically related, and LLMs can naturally cluster them, providing a more interpretable state representation. To leverage this capability, we introduce a contextualization-based adaptation method that integrates learnable prompts into an LLM-augmented DRL framework. Instead of relying on full model fine-tuning, we refine state representations through task-specific prompts that dynamically adjust to network conditions. Utilizing ORANSight, an LLM trained on O-RAN knowledge, we develop Prompt-Augmented Multi agent RL (PA-MRL) framework. Learnable prompts optimize both semantic clustering and RL objectives, allowing RL agents to achieve higher rewards in fewer iterations and adapt more efficiently. By incorporating prompt-augmented learning, our approach enables faster, more scalable, and adaptive resource allocation in O-RAN slicing. Experimental results show that it accelerates convergence and outperforms other baselines. I Introduction Advanced wireless networks must be highly dynamic and capable of managing heterogeneous service demands efficiently. Network slicing technique is a key feature of these networks that enable them to support heterogeneous services like eMBB (enhanced Mobile Broadband), URLLC (Ultra-Reliable Low-Latency Communications), and mMTC (massive Machine-Type Communications). Recognizing its importance, the open radio access network (O-RAN) ALLIANCE has identified network slicing as a critical technology for building flexible and intelligent radio access networks [ 1 ] . The modular nature of the O-RAN architecture through its separation of components like the RAN Intelligent Controller (RIC), Centralized Units (CUs), and Distributed Units (DUs), lays the foundation for scalable and distributed network control [ 2 , 3 ] . This disaggregation makes it well-suited for deploying multi-agent reinforcement learning (MARL), where multiple deep reinforcement learning (DRL) agents operate across the CUs and DUs [ 4 , 5 ] . At the DU level, agents manage localized tasks such as resource block (RB) allocation and user scheduling, while inter-agent coordination enables system-wide optimization. Despite these advances, traditional DRL approaches often struggle to adapt quickly in dynamic and rapidly changing network conditions. They typically require numerous training iterations to establish reliable state-action mappings. Moreover, the scattered and noisy nature of real-time feedback in wireless environments complicates the agent’s ability to extract structured representations and make optimal decisions. Recent studies have attempted to augment deep learning approaches by incorporating Large Language Models (LLMs) into wireless systems, enhancing deep learning-based decision-making [ 6 , 7 , 8 , 9 , 10 , 11 , 12 ] . Transfer learning, particularly with LLMs, plays a crucial role by enabling generalization across tasks through pretraining and fine-tuning. However, conventional fine-tuning demands substantial labeled data and computational resources and challenges that are particularly limiting in dynamic and resource constrained environments like O-RAN slicing. To mitigate this, more efficient adaptation strategies such as LoRA [ 13 ] and adapter-based methods [ 14 ] have been proposed. These methods introduce lightweight trainable components into frozen models, allowing task adaptation with significantly fewer parameters. To further enhance efficiency and adaptability in O-RAN, this work adopts a lightweight and scalable prompt-learning LLM adaptation strategy that uses learnable prompts, small trainable embeddings that guide the LLM without modifying its core parameters. In contrast to traditional fine-tuning or adapter-based approaches, this approach offers a lightweight and scalable solution that is well-suited for real-time, decentralized wireless environments scenarios like O-RAN, where agents must adapt on the fly with limited feedback. We employ ORANSight, a specialized LLM pre-trained on comprehensive O-RAN knowledge [ 15 , 16 ] , to contextualize observations into meaningful latent representations and transform scattered network feedback into semantically meaningful latent representations. These representations are used by decentralized DRL agents to make more adaptive and generalizable slicing and scheduling decisions. This work moves beyond traditional fine-tuning by proposing a novel contextualization-based framework that enhances decision-making in O-RAN slicing by coupling LLM-driven representations with MARL. Unlike traditional fine-tuning approaches that modify the entire language model, we introduce learnable prompts as compact and trainable tokens that guide the pre-trained LLM in generating semantically meaningful representations of the environment. These prompts are updated with the DRL agent’s policy, ensuring that state representations align with the agent’s learning objectives. This design allows the model to flexibly interpret complex network dynamics, such as varying QoS levels or RF conditions, without incurring the high costs of full model retraining. Additionally, by maintaining the pretrained LLM’s core parameters, our method preserves its generalization capabilities while adapting efficiently to task-specific goals. Integrated into a MARL framework, this setup enables decentralized agents to develop faster, more robust policies for joint slicing and resource scheduling in O-RAN. As a result, our approach delivers improved adaptability, faster convergence, and lower computational overhead, making it well-suited for real-time and scalable wireless network management. The main contributions of this paper are summarized as follows: • We propose a novel Prompt-Augmented Multi-Agent Reinforcement Learning (PA-MRL) framework tailored for O-RAN slicing, which integrates both learnable and informal prompts to enhance state representation and decision-making. • We combine LLM-generated textual embeddings with raw network metrics using pre-trained adapter networks, forming a unified and semantically meaningful representation that improves agent awareness and convergence. • By embedding prompt learning into the RL loop, our method enables faster learning and better generalization in highly dynamic, decentralized O-RAN environments, outperforming traditional fine-tuning approaches in efficiency and adaptability. • We implement our approach using Soft Actor-Critic (SAC) in a MARL setup distributed across DUs, enabling fine-grained, scalable resource allocation while preserving stability in continuous action spaces. To the best of our knowledge, this is the first work to adaptively integrate context-aware LLM-based state representation into a MARL framework for joint O-RAN slicing and scheduling . Fig. 1 : Prompt Enhancement LLM Augmented MARL System Framework. II Related Works LLMs have recently emerged as powerful tools in wireless communication research. Their capacity to handle and reason over diverse data types, such as textual inputs and RF signal measurements, has led to their growing application in wireless communication areas, including intelligent network control, anomaly detection, and analysis of 3GPP standards [ 9 , 10 , 15 ] . II-A Transfer Learning Foundations Transfer learning enables deep learning models to apply pre-existing knowledge to new tasks, typically through pretraining on large-scale corpora followed by task-specific fine-tuning. Although effective, fine-tuning is resource-intensive and relies on labeled data, which limits its practicality in dynamic, low-resource settings like O-RAN slicing. To address this, recent studies propose parameter-efficient alternatives such as LoRA (Low-Rank Adaptation) [ 13 ] and adapter modules [ 14 ] , which insert lightweight, trainable components into frozen models. Another alternative is prompt learning [ 17 , 18 ] , where learnable prompts steer model outputs without modifying core weights, making it ideal for few-shot or zero-shot settings. II-B LLM-Augmented RL in Wireless Network

Read the complete text