An Autonomous Network Orchestration Framework Integrating Large Language Models with Continual Reinforcement Learning

Masoud Shokrnezhad 1 , and Tarik Taleb 2 · 2025

Read An Autonomous Network Orchestration Framework Integrating Large Language Models with Continual Reinforcement Learning by Masoud Shokrnezhad 1 , and Tarik Taleb 2 (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2502-16198.txt.

Introduction

Masoud Shokrnezhad and Tarik Taleb propose ARC, a framework that runs future 6G networks with language models steering and learning agents executing, in a paper from 2025. Sixth-generation networks will weave space, air, and ground links with meaning-based communication, a tangle too complex for fixed rules. Language models can plan, but they hallucinate and lag behind network change.

ARC splits the job in two tiers. A retrieval-fed language model watches services and resources and drafts high-level plans, using chain-of-thought prompts sharpened by contrastive learning. Reinforcement learning agents then make the low-level calls, learning continually from a managed replay buffer. Simulations back the design, and the authors map open problems for making such autonomy trustworthy.

Catalog record

CREATORMasoud Shokrnezhad 1 , and Tarik Taleb 2
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE36 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2502-16198

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

An Autonomous Network Orchestration Framework Integrating Large Language Models with Continual Reinforcement Learning Masoud Shokrnezhad 1 , and Tarik Taleb 2 Abstract 6G networks aim to achieve global coverage, massive connectivity, and ultra-stringent requirements. Space-Air-Ground Integrated Networks (SAGINs) and Semantic Communication (SemCom) are essential for realizing these goals, yet they introduce considerable complexity in resource orchestration. Drawing inspiration from research in robotics, a viable solution to manage this complexity is the application of Large Language Models (LLMs). Although the use of LLMs in network orchestration has recently gained attention, existing solutions have not sufficiently addressed LLM hallucinations or their adaptation to network dynamics. To address this gap, this paper proposes a framework called Autonomous Reinforcement Coordination (ARC) for a SemCom-enabled SAGIN. This framework employs an LLM-based Retrieval-Augmented Generator (RAG) monitors services, users, and resources and processes the collected data, while a Hierarchical Action Planner (HAP) orchestrates resources. ARC decomposes orchestration into two tiers, utilizing LLMs for high-level planning and Reinforcement Learning (RL) agents for low-level decision-making, in alignment with the Mixture of Experts (MoE) concept. The LLMs utilize Chain-of-Thought (CoT) reasoning for few-shot learning, empowered by contrastive learning, while the RL agents employ replay buffer management for continual learning, thereby achieving efficiency, accuracy, and adaptability. Simulations are provided to demonstrate the effectiveness of ARC, along with a comprehensive discussion on potential future research directions to enhance and upgrade ARC. I Introduction The requirements for sixth-generation (6G) networks include global coverage, which necessitates integrating diverse communication modalities through Space-Air-Ground Integrated Networks (SAGINs). This integration ensures seamless connectivity and consistent service quality in urban, rural, and remote areas [ 1 ] . Additionally, 6G must enable massive connectivity, ultra-high data rates, and ultra-low latency. Semantic Communication (SemCom) addresses these needs by focusing on message meaning and context rather than bit-oriented transmission, optimizing bandwidth usage and enhancing resource efficiency [ 2 ] . The problem of orchestrating 6G networks, involving the allocation of resources to users while considering quality of service requirements and system-level objectives, presents inherent complexity. The integration of domains such as space within SAGIN further exacerbates this complexity, as the dynamics of resources remain largely unknown. Furthermore, the implementation of SemCom renders the evaluation of service quality using conventional metrics unfeasible, thereby complicating the problem further [ 2 ] . Generally, one potential solution to address decision-making challenges in complex problems where the solution space is excessively large, non-numeric, or characterized by unknown dimensions—rendering algorithmic exploration impractical—is the utilization of Large Language Models (LLMs). This concept has been widely utilized in robotics research, illustrating that LLMs can greatly improve decision-making processes when specific objectives need to be met by identifying an optimal sequence from an infinite array of actions. [ 3 ] . However, the use of LLMs in network orchestration remains at an early stage. For instance, Qiu et al. [ 4 ] focused on optimizing the number and placement of wireless access points with LLMs. Sun et al. [ 5 ] adopted LLMs for trajectory planning and wireless resource allocation in scenarios serving Unmanned Aerial Vehicles (UAVs). Liu et al. [ 6 ] equipped Reconfigurable Intelligent Surfaces (RISs) with LLMs to deduce optimized strategies for wireless resource allocation and signal decoding order. Mekrache et al. [ 7 ] and Rong et al. [ 8 ] expanded this idea to multi-domain resource allocation, addressing network and cloud as well as terrestrial and non-terrestrial integrated infrastructures, respectively. Despite the high quality claimed by these studies, all aspects of resource allocation are exclusively determined by LLMs in their methods, increasing the risk of hallucinations, particularly in expansive solution spaces. Additionally, the adaptation of these LLM-based mechanisms to probable fluctuations, especially unknown ones, has not been investigated. Furthermore, SemCom is entirely overlooked. The only solution superficially proposed is fine-tuning; which is impractical in live setups due to its resource- and time-intensive nature. Thus, the development of a sophisticated LLM-based solution for 6G network orchestration is essential [ 9 , 10 ] . To address the gaps in existing research, we propose a framework named Autonomous Reinforcement Coordination (ARC) for a system of SemCom-enabled SAGIN. To decompose the inherent complexity of the problem, the resource allocation process is structured within a two-tier component called the Hierarchical Action Planner (HAP). In the outer tier, users and their corresponding actions (required resource allocation decisions) are sequenced, while in the inner tier, each action is executed in isolation, following the Mixture of Experts (MoE) concept. The integration of HAP with the system is facilitated by a Retrieval-Augmented Generator (RAG), which gathers data from system elements to prepare inputs for HAP. We utilize LLMs in both RAG, to comprehend strategical textual commands and to evaluate the quality of SemCom connections, and HAP, to derive efficient sequences in the outer tier. For the inner tier of HAP, we utilize pre-trained Reinforcement Learning (RL) agents to minimize the computation and storage overheads associated with training, with each agent assigned to a specific task. While LLMs are reserved for high-level tasks, we implement Chain-of-Thought (CoT) reasoning through few-shot learning to reduce the likelihood of hallucinations to enhance accuracy. To adapt ARC to system dynamics, RAG calculates rewards for recent actions and incorporates them into the inputs of HAP. These rewards are utilized through contrastive learning to select the highest-reward exemplars in few-shot learning, producing updated sequences, and through continual learning to keep the RL agents updated via replay buffer management. To the best of our knowledge, this is the first work to employ LLMs and related techniques, such as few-shot learning, to enhance continual RL-based resource allocation for improved efficiency, accuracy, and adaptability. The paper is structured as follows. Section II provides essential background by describing the system model, formulating the resource allocation problem, and outlining the associated challenges. Section III details the solutions, including the main components and workflows of ARC. An ablation study is presented in Section IV , followed by potential directions for future work in Section V . Concluding remarks are drawn in Section VI . II Preliminaries II-A System Model In this paper, we consider a system following the SAGIN model [ 1 ] , which includes a bi-dimensional terrestrial network layer augmented by a third dimension provided by two non-terrestrial network layers—airborne and spaceborne—at different altitudes and orbits (hereafter referred to as the infrastructure ). The ground layer, operating at or near the surface of the Earth, consists of cellular and satellite ground stations, interconnected through network nodes via fiber optic and cable links. The air layer, operating in the stratosphere and lower atmosphere, comprises high-altitude platform stations and unmanned aerial vehicles, supporting both cellular and satellite communication interfaces. The space layer, operating in orbit around the Earth, consists of satellite constellations in various orbits (geostationary, low Earth, medium Earth, and highly elliptical). Furthermore, all nodes across the three layers are equipped with computing resources, collectively forming a distributed cloud that provides extensive computational capabilities throughout the network [ 11 ] . The infrastructure can be leveraged as a multi-layered, integrated communication and computation platform to enable innovative services for 6G and beyond. We consider a set of predefined services (hereafter referred to as the services ), each represented by a directed graph of

Read the complete text