RadialRouter: Structured Representation for Efficient and Robust Large Language Models Routing
Ruihan Jin, Pengpeng Shao, Zhengqi Wen, Jinyang Wu, Shuai Zhang, Jianhua Tao, [email protected] · 2025
Read RadialRouter: Structured Representation for Efficient and Robust Large Language Models Routing by Ruihan Jin, Pengpeng Shao, Zhengqi Wen, Jinyang Wu, Shuai Zhang, Jianhua Tao, [email protected] (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2506-03880.txt.
Introduction
Ruihan Jin and co-authors build a smarter switchboard for large language models in 2025. With many models to choose from, a router must send each user question to the model that fits it best, balancing quality and cost. Older routers, the team argues, do not dig deep enough into how a question relates to what each model does well.
Their system, RadialRouter, uses a small transformer with a radial layout to map that link, then picks a model from the final states. On a standard routing benchmark it beat rival methods in both balanced and cost-first modes. It also adapts when the pool of models changes.
Catalog record
| CREATOR | Ruihan Jin, Pengpeng Shao, Zhengqi Wen, Jinyang Wu, Shuai Zhang, Jianhua Tao, [email protected] |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 43 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2506-03880
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
RadialRouter: Structured Representation for Efficient and Robust Large Language Models Routing Ruihan Jin, Pengpeng Shao, Zhengqi Wen, Jinyang Wu, Shuai Zhang, Jianhua Tao, [email protected] Abstract The rapid advancements in large language models (LLMs) have led to the emergence of routing techniques, which aim to efficiently select the optimal LLM from diverse candidates to tackle specific tasks, optimizing performance while reducing costs. Current LLM routing methods are limited in effectiveness due to insufficient exploration of the intrinsic connection between user queries and the characteristics of LLMs. To address this issue, in this paper, we present RadialRouter , a novel framework for LLM routing which employs a lightweight Transformer-based backbone with a radial structure named RadialFormer to articulate the query-LLMs relationship. The optimal LLM selection is performed based on the final states of RadialFormer. The pipeline is further refined by an objective function that combines Kullback-Leibler divergence with the query-query contrastive loss to enhance robustness. Experimental results on RouterBench show that RadialRouter significantly outperforms existing routing methods by 9.2% and 5.8% in the Balance and Cost First scenarios, respectively. Additionally, its adaptability toward different performance-cost trade-offs and the dynamic LLM pool demonstrates practical application potential. Figure 1: Paradigm comparison between different LLM routing methods. Existing methods lack the modeling of the interrelation between the query and LLMs, while the proposed RadialRouter unifies the routing process in a structured representation. 1 Introduction Recent advances in natural language processing, significantly driven by the development of large language models (LLMs), has opened new frontiers across numerous applications. LLMs demonstrate outstanding performance on various tasks, including mathematical problem-solving Romera-Paredes et al. 2024 , commonsense reasoning Zhao et al. 2023 , and code generation Wu et al. 2023 . The growing reliance on LLMs gives rise to the concept of LLM ensemble Chen et al. 2025 , which integrates multiple LLMs to establish a system capable of multitasking, thereby generating more accurate and robust responses to user inputs. Through this collaborative approach, practitioners can leverage the unique strengths of different LLMs, potentially improving the overall performance and reliability in addressing diverse requirements. However, as the size and complexity of LLMs increase, the challenges of deploying LLM ensemble —such as computational cost, latency, and scalability —also intensify. To address these challenges, LLM routing is explored to dynamically assign a specific LLM in the LLM ensemble to user queries. As shown in Fig. 1 , early attempts Ding et al. 2024 ; Ong et al. 2024 employ a binary score router to decide whether to select a smaller LLM or a larger one for a given query. Chen et al. 2023 utilizes a router to guide cascaded LLMs for generating responses. Further research Chen et al. 2024 introduces similarity matching to align the query with LLMs and select the most appropriate LLM. However, these methods are limited in the following aspects: 1) They narrow the routing sorely to identify the optimal LLM, failing to capture the intrinsic connection between the query and LLMs, which undermines their effectiveness in achieving ideal routing results. 2) The exclusive dependence on the features extracted by the BERT-based text encoder hinders their ability to leverage the contextual information and prevents a nuanced understanding of the underlying relationships and constraining the efficiency of routing. 3) Existing methods that limit routing to a fixed number of LLMs struggle to adapt to a dynamically evolving pool of LLMs. 4) Certain approaches neglect the actual requirements of the task, rendering them ineffective in scenarios that necessitate a simultaneous consideration of both performance and cost. In this paper, we propose a novel LLM routing approach named RadialRouter , which leverages a Transformer-based architecture to enhance performance for efficient and robust LLM routing. To represent the interrelationship between the query and LLMs, we propose RadialFormer as the backbone of RadialRouter and incorporate the structure consisting of a relay node and n satellite nodes. During update, these nodes are processed through the multi-head attention mechanism. Compared with the standard Transformer, RadialFormer reduces the computational complexity from O(n^{2}d) to O(nd) . The optimal LLM selection is performed based on the final states of satellite nodes. To guide the selection and enhance comprehensive representation, we employ a Kullback-Leibler divergence loss for supervision. Additionally, we cluster the queries into groups and introduce a query-query contrastive loss, which fosters the generation of similar embeddings for semantically related queries, facilitating robust LLM routing. We conduct experiments on challenging RouterBench Hu et al. 2024 encompassing 4 task domains (commonsense reasoning, knowledge-based language understanding, math, and coding) to evaluate the proposed RadialRouter in three scenarios. Extensive experiments demonstrate that RadialRouter efficiently harnesses the interrelationship of routing and outperforms existing routing methods by a large margin. Furthermore, the adaptability of RadialRouter toward different performance-cost trade-offs and the dynamic LLM pool is verified through extended experiments. Our contributions are summarized as follows: • We propose RadialRouter, a novel framework that leverages a Transformer-based architecture to dynamically route user queries to suitable LLMs. • We introduce a lightweight architecture RadialFormer as the backbone of RadialRouter to capture the interrelationship between the query and LLMs in routing. To improve the robustness of routing, we incorporate contrastive loss in the optimization of RadialRouter. • Experimental results show that RadialRouter outperforms baseline routing methods and achieves efficient and robust routing in three scenarios with different performance-cost trade-offs. (a) The framework of our proposed RadialRouter . RadialRouter captures the interrelation between the query and LLMs through a lightweight Transformer-based architecture. It selects the optimal LLM based on the final states of satellite nodes, while optimizing the pipeline through an objective function that combines Kullback-Leibler divergence and query-query contrastive loss. (b) Connections of one layer in RadialFormer , where each satellite node is exclusively connected to the relay node to form a radial configuration. Figure 2: Overview of RadialRouter methodology. 2 Related Work LLM Ensemble The remarkable performance exhibited by a single LLM, coupled with the increasing demand for enhanced cross-domain capabilities, has catalyzed the emergence of the concept of LLM ensemble. Majority voting Wang et al. 2022 ; Li et al. 2024 is a simple yet effective method to achieve the LLM ensemble. Jiang et al. 2023 proposes a supervised ensembling method to produce enhanced results by synthesizing the outputs of all LLMs. Du et al. 2023 develops a debate framework for LLM collaboration. LLM cascading Chen et al. 2023 ; Yue et al. 2023 ; Gupta et al. 2024 ; Nie et al. 2024 adopts a serialized model architecture and halts when the output quality is satisfied. Wang et al. 2023 tackles the fusion-of-experts problem by combining outputs from models with diverse knowledge domains. These approaches frequently exhibit considerable computational complexity, leading to substantial time latency and cost in practical applications. In contrast, our RadialRouter is highly efficient, as it requires only a single invocation of the routed LLM. LLM Routing Similar to Mixture-of-Expert (MoE) approaches Jacobs et al. 1991 ; Collobert et al. 2003 ; Jiang et al. 2024 , LLM routing is designed to identify the most suitable LLM for a query, allowing the lightweight activation of LLM ensemble. Shnitzer et al. 2023 addresses the LLM selection problem with a series of binary classification tasks. ZOOTER Lu et al. 2023 develops a