ECVL-ROUTER: Scenario-Aware Routing for Vision-Language Models
Xin Tang, Youfang Han, Fangfei Gou, Wei Zhao, Xin Meng, Yang Yu, [0.8em] Jinguo Zhang, Yuanchun Shi, Yuntao Wang, Tengxiang Zhang, [1.2em] Tsinghua University Goertek Inc · 2025
Read ECVL-ROUTER: Scenario-Aware Routing for Vision-Language Models by Xin Tang, Youfang Han, Fangfei Gou, Wei Zhao, Xin Meng, Yang Yu, [0.8em] Jinguo Zhang, Yuanchun Shi, Yuntao Wang, Tengxiang Zhang, [1.2em] Tsinghua University Goertek Inc (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2510-27256.txt.
Introduction
Xin Tang leads nine co-authors in ECVL-ROUTER, a scenario-aware router that sends each vision-language query to the right-sized model. User needs vary between fast response, high quality, and low energy use. Cloud giants answer everything at high latency and energy cost, while small edge models handle simple tasks cheaply, so choosing per query saves resources.
The framework introduces a routing strategy and evaluation metrics that pick a model per query to maximize overall utility under the stated requirement. A multimodal response-quality dataset tailored for router training supports the design. Extensive experiments route over eighty percent of queries to the small model while dropping problem-solving success by under ten percent. The authors call it the first scenario-aware routing framework for vision-language models.
Catalog record
| CREATOR | Xin Tang, Youfang Han, Fangfei Gou, Wei Zhao, Xin Meng, Yang Yu, [0.8em] Jinguo Zhang, Yuanchun Shi, Yuntao Wang, Tengxiang Zhang, [1.2em] Tsinghua University Goertek Inc |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 61 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2510-27256
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
ECVL-ROUTER: Scenario-Aware Routing for Vision-Language Models Xin Tang, Youfang Han, Fangfei Gou, Wei Zhao, Xin Meng, Yang Yu, [0.8em] Jinguo Zhang, Yuanchun Shi, Yuntao Wang, Tengxiang Zhang, [1.2em] Tsinghua University Goertek Inc Abstract Vision-Language Models (VLMs) excel in diverse multimodal tasks. However, user requirements vary across scenarios, which can be categorized into fast response, high-quality output, and low energy consumption. Relying solely on large models deployed in the cloud for all queries often leads to high latency and energy cost, while small models deployed on edge devices are capable of handling simpler tasks with low latency and energy cost. To fully leverage the strengths of both large and small models, we propose ECVL-ROUTER, the first scenario-aware routing framework for VLMs. Our approach introduces a new routing strategy and evaluation metrics that dynamically select the appropriate model for each query based on user requirements, maximizing overall utility. We also construct a multimodal response-quality dataset tailored for router training and validate the approach through extensive experiments. Results show that our approach successfully routes over 80% of queries to the small model while incurring less than 10% drop in problem solving probability. 1 Introduction Vision-Language Models (VLMs), which integrate visual and textual understanding, have become crucial components in a wide range of AI applications, from robotics control to user interface navigation ( Zhang et al. 2024 ; Shinde et al. 2025 ; Li et al. 2024 ) . In practice, predominant deployment strategy relies heavily on powerful, cloud-hosted Large VLMs (LVLMs) to serve all user queries ( Jang & Morabito 2025 ; Navardi et al. 2025 ; Zheng et al. 2025 ) , which excel at complex reasoning but incur latency and energy costs ( Fernandez et al. 2025 ; Jegham et al. 2025 ; Charyyev et al. 2020 ) while underutilizing capable small VLMs (SVLMs) on edge devices ( Sharshar et al. 2025 ; Belcak et al. 2025 ) . Moreover, a one-size-fits-all deployment strategy is suboptimal, as users increasingly expect systems that not only deliver high-quality responses but also adapt to diverse real-world scenarios with varying demands for latency, cost, and privacy. To effectively integrate the strengths of both LVLMs and SVLMs, edge–cloud collaborative routing ( Yuan et al. 2025 ; Hao et al. 2024 ) is a natural fits. At its core is a lightweight model router ( Ding et al. 2024 ; Ong et al. 2024 ) that inspects each query and selects an appropriate VLM. However, a general router is insufficient, routing must be scenario-aware : behaviors vary across diverse application contexts and can be configured by users or automatically inferred by scenario detection algorithms ( Fifty et al. 2023 ; Someki et al. 2025 ) . Existing routers are often text-centric and optimize a fixed trade-off between cost and quality, failing to adapt to multimodal, scenario-aware user needs. For example, real-time games interaction prioritizes low latency, medical diagnostics emphasizes answer quality, and mobile assistants require low energy use and strong privacy ( Asgari et al. 2025 ) . Therefore, an ideal model router should align with diverse user requirements and make the most appropriate routing decision in different scenarios. In this work, we introduce E dge- C loud V ision- L anguage Router ( ECVL-ROUTER ), a novel, scenario-aware routing framework for VLMs. Motivated by heterogeneous real-world scenarios, we distill three primary user requirements: (1) fast response, (2) high-quality output, and (3) low energy consumption & data privacy. Accordingly, our routing objective is to maximize the use of small edge models while meeting user satisfaction. To achieve this, we introduce the Minimal Expectation Score (MES) to quantify the user’s acceptable quality threshold in different scenarios. Guided by MES, the router prefers an SVLM whenever its predicted output meets the MES threshold and escalates to a larger cloud model otherwise. To implement this strategy, we design a lightweight routing classifier and construct a response quality dataset using an MES-based annotation policy to support its training. For evaluation, we propose the Routing Comprehensive Score (RCS), integrating three key aspects: Average Problem-Solving Probability (APSP), Cost Advantage (CA), and Average Inference Latency (AIL), which correspond directly to three core user requirements. Extensive experiments demonstrate that our routing strategy and framework delivers favorable trade-offs across quality, latency, and cost while adhering to scenario-specific user requirements. The main contributions of this paper can be summarized as follows: 1. We propose a novel, scenario-aware routing strategy for VLMs and a new set of model routing evaluation metrics (Section 3 ). This strategy moves beyond the traditional cost-quality trade-off by centering on the user’s dynamic requirements to maximize the system’s overall utility. 2. We design and implement ECVL-ROUTER, the first cloud-edge collaborative, scenario-aware routing framework for VLMs, which effectively combines the high performance of large cloud models with the low-cost advantages of small edge models (Section 4.2 ). 3. We construct the first response quality dataset for training and evaluating VLM routers (Section 4.1 ). Through extensive experiments on this dataset, we have thoroughly validated the effectiveness of our ECVL-ROUTER framework (Section 5 ). We also open source our dataset and framework for users to train their routers based on their specific requirements. 1 1 1 The code is available at https://anonymous.4open.science/r/ECVL-Router-977D 2 Related Work 2.1 SVLM vs. LVLM Vision-Language Models now underpin applications from accessibility assistance and UI navigation to robotics and scientific content creation ( Zhang et al. 2024 ; Shinde et al. 2025 ) . User requirements in these domains differ widely. Some applications, like real-time games or augmented reality, demand fast response where low latency is critical ( Vasu et al. 2025 ; Liu et al. 2025 ) . Others, such as medical diagnostics, insist on high-quality answers where accuracy is paramount ( Singhal et al. 2025 ; Singhal et al. 2023 ) . For applications on mobile or IoT devices, low energy consumption and privacy constraints are decisive factors ( Chu et al. 2023 ; Wang et al. 2025 ) . Cloud-hosted LVLMs (e.g., GPT-4o and Gemini 2.5 Pro) offer strong multimodal reasoning but impose round-trip latency and significant compute costs, making them ill-suited for latency-sensitive or privacy-constrained settings ( Hurst et al. 2024 ; Team et al. 2023 ; Charyyev et al. 2020 ; Fernandez et al. 2025 ) . In parallel, SVLMs have become increasingly capable on consumer hardware. For example, Google’s Gemma 3 270M can run on a mobile phone with very low power consumption; internal tests showed that 25 conversations on a Pixel 9 Pro consumed only 0.75% of the battery ( Lacombe et al. 2025 ) . Meanwhile, Microsoft’s Phi-4-multimodal, with only 5.6 billion parameters, runs on personal computers to efficiently handle tasks like OCR and chart understanding ( Abouelenin et al. 2025 ) . Despite these advances, production stacks still over-rely on cloud LVLMs. This gap motivates scenario-aware edge–cloud designs that preferentially serve queries on-device and escalate only when quality requirements exceed local capacity. 2.2 Model Routing From objective functions and decision paradigms, prior work falls into three types: (1) Outcome-optimal, non-predictive cascades. These methods prioritize the quality of the final answer by allowing multiple model calls until a target quality threshold is met. A common strategy is cascading : invoke models from low