INAR-VL: Input-Aware Routing for Edge–Cloud Vision–Language Inference CCS: Computer systems organization Embedded and cyber-physical systems CCS: Computer systems organization Embedded systems CCS: Computing methodologies Machine learning approaches

Ahmed Šabanović, Paul Joe Maliakel, Ivona Brandić · 2026

Read INAR-VL: Input-Aware Routing for Edge–Cloud Vision–Language Inference CCS: Computer systems organization Embedded and cyber-physical systems CCS: Computer systems organization Embedded systems CCS: Computing methodologies Machine learning approaches by Ahmed Šabanović, Paul Joe Maliakel, Ivona Brandić (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-18853.txt.

Introduction

Ahmed Šabanović and two colleagues route vision-language inference between edge and cloud by input difficulty. Cloud models answer well but add delay and energy cost, while edge models run fast with less capacity. Static placement wastes both, since image quality and reasoning difficulty vary. Their INAR-VL system reads light image and text complexity signals, keeps simple queries local, and offloads hard ones. Tests show about thirty-six percent of requests stay on the edge, with latency down about twenty-four percent and energy down about twenty-six percent.

Catalog record

CREATORAhmed Šabanović, Paul Joe Maliakel, Ivona Brandić
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE28276
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2605.18853

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

INAR-VL: Input-Aware Routing for Edge–Cloud Vision–Language Inference CCS: Computer systems organization Embedded and cyber-physical systems CCS: Computer systems organization Embedded systems CCS: Computing methodologies Machine learning approaches Ahmed Šabanović, Paul Joe Maliakel, Ivona Brandić Abstract . Edge deployment of Vision–Language Models (VLMs) faces a tradeoff between latency and accuracy: cloud execution provides high-quality predictions but incurs communication delay and energy cost, while edge-only execution is faster but less accurate due to limited model capacity. This tradeoff is further complicated by heterogeneity in image quality and reasoning complexity, making static placement suboptimal. We present INAR-VL, a lightweight edge–cloud routing system for multimodal inference in a two-tier deployment. INAR-VL maintains complementary VLMs across edge and cloud and uses lightweight image and text complexity signals to guide routing and model selection, executing simple queries locally while offloading complex ones when beneficial. Evaluation on visual question answering shows that INAR-VL executes 36% of requests on the edge, reduces latency by 24%, lowers energy by 26%, and preserves 97% of cloud-level accuracy. 1. INTRODUCTION VLMs are increasingly deployed in edge environments such as mobile devices, robots, and embedded cameras, where inference must operate under strict constraints in latency, energy, memory, and bandwidth ( 18 ; 1 ; 13 ) . Consider a field technician using smart glasses to query equipment labels and maintenance instructions while moving through a factory floor with patchy Wi-Fi. In such scenarios, sending every request to the cloud introduces unpredictable communication delays, while relying solely on a small on-device model risks incorrect answers on visually degraded inputs or complex reasoning queries. Serving VLMs efficiently in such environments remains challenging ( 12 ) . Unlike text-only workloads, multimodal inputs exhibit high variability in image quality (e.g., blur, lighting, compression artifacts) and text complexity, which strongly affects both model accuracy and inference cost ( 15 ) . Existing approaches largely ignore this heterogeneity. Systems such as MoA-Off and recent routing approaches inspired by RouteLLM-style difficulty estimation rely on static deployment choices or primarily text-based signals, which are insufficient for multimodal workloads ( 19 ; 17 ; 6 ) . While effective for text-only LLMs ( 6 ) , they fail to capture visual degradation effects, where, for example, increasing resolution helps fine-grained images but not blurred ones ( 15 ) . Our measurements highlight this trade-off: edge-only inference is faster and more energy-efficient but less accurate than cloud-only execution. To make this trade-off explicit, our baselines show that edge-only execution achieves 66.5% accuracy at 824 ms and 7.5 J per sample, while cloud-only reaches 74.4% accuracy at 2408 ms and 26.0 J. This gap motivates input-aware routing instead of static placement. However, designing such a router for edge environments introduces several challenges. First, different VLM architectures exhibit complementary strengths: some models are better at perception-heavy tasks such as OCR or fine-grained visual recognition, while others perform better on language-intensive reasoning queries ( 2 ) . Second, routing decisions must be made before expensive inference, requiring lightweight request characterization rather than additional model evaluations. Third, routing must respect practical deployment constraints including limited edge GPU memory, device energy budgets, network variability, and strict overhead limits for pre-inference analysis. To address these challenges, we present INAR-VL, a practical routing system for adaptive edge–cloud VLM inference. Unlike prior approaches that rely on a single model or static offloading decisions, INAR-VL maintains a heterogeneous pool of complementary vision–language models distributed across edge and cloud resources. A key novelty of our approach is the use of lightweight multimodal signals—capturing both image quality and text complexity—extracted prior to inference to guide routing decisions without invoking expensive VLM computation. These signals drive a routing policy that jointly optimizes three dimensions: model architecture, input resolution, and execution placement. This joint optimization across model, resolution, and placement has not been explicitly addressed in prior VLM routing systems. Contributions. We design INAR-VL, a routing system that uses lightweight multimodal signals for pre-inference request characterization, enabling model and placement decisions without additional VLM cost. The policy is deployment-aware, jointly optimizing model choice, resolution, and edge/ cloud placement under practical constraints such as memory, energy, and network variability. We further provide an end-to-end evaluation on an 8 GB edge GPU and cloud setup across three VQA benchmarks, demonstrating improved latency–accuracy trade-offs over static baselines. 2. BACKGROUND & MOTIVATION Designing an effective routing policy requires understanding both the hardware constraints of edge deployments and the variability in model capabilities across different VLM architectures. In this section, we briefly review these two aspects and highlight why static edge or cloud placement performs poorly for heterogeneous multimodal workloads. 2.1. Edge Resource Constraints Modern VLMs combine language backbones, visual encoders, and multimodal projection layers, resulting in substantial memory and compute requirements. For example, a 7B model executed in FP16 typically requires roughly 14–16 GB of GPU memory for weights alone, excluding activations and decoding caches ( 18 ; 1 ) . These requirements exceed the capacity of many edge platforms operating under strict power and thermal limits. As a result, executing large VLMs entirely on edge hardware is often impractical, while sending every request to the cloud introduces additional network latency and bandwidth overhead. Hybrid edge–cloud deployments therefore represent a practical compromise. 2.2. Heterogeneity Across Vision–Language Models VLM architectures show complementary strengths. Qwen-VL achieves 79.5% VQAv2, 59.3% GQA, 63.8% TextVQA , excelling at OCR and visually grounded language tasks ( 3 ) , whereas LLaVA-1.5 scores 78.5% VQAv2, 62.0% GQA, 58.2% TextVQA , favoring reasoning-oriented VQA ( 4 ) . Other studies confirm that LLaVA-OneVision-1.5 is stronger on reasoning benchmarks, while Qwen-VL outperforms on OCR tasks ( 2 ) . No single model dominates across all inputs, motivating adaptive per-request selection. 3. INAR-VL INAR-VL is an input-aware routing system for edge–cloud VLM inference. In the deployment studied here, the router chooses among four complementary models: two compact edge VLMs that fit within an 8 GB device budget and two larger cloud VLMs that provide stronger visual and reasoning capability. Given an (\text{image},\text{question}) request, INAR-VL constructs a per-request Pareto frontier over a configuration space spanning model architecture, numeric precision, input resolution, and compute placement. The router enumerates valid configurations, scores each on predicted answer quality and estimated cost, and then selects among Pareto-optimal candidates using a calibrated edge-preference rule. 3.1. System Overview Figure 1. INAR-VL architecture. A multimodal request is routed via a Pareto-based optimizer that selects the model and execution location (edge or cloud) to balance accuracy, latency, and energy. System architecture of INAR-VL. An image-question request enters the edge node, where lightweight image and text feature extraction runs in parallel and produces a fused feature vector. A Pareto-based router evaluates edge and cloud candidates by predicted quality and estimated cost, selects one execution path, dispatches to the chosen model, and returns the answer. Table 1. Model pool used by INAR-VL. Model Params Deploy Prec. Vision Encoder Qwen3-VL-2B 2B Edge INT8 Qwen + DeepStack SmolVLM-2B 2B Edge INT8 SigLIP LLaVA-OV-8B 8B Cloud FP16 RICE-ViT Qwen3-VL-8B 8B Cloud FP16 Qwen + DeepStack Figure 1 shows the INAR-VL architecture used in our evaluation, and Table 1 summarizes the model pool used by the router. We follow the pipeline from left to right. (1) Input ingestion: the request is provided as an

Read the complete text