On-Device Robotic Planning: Eliminating Inference Redundancy for Efficient Decision-Making
Joonhee Lee, Hyunseung Shin, Hyunmi Kim, Pei Zhang, Jeonggil Ko · 2026
Read On-Device Robotic Planning: Eliminating Inference Redundancy for Efficient Decision-Making by Joonhee Lee, Hyunseung Shin, Hyunmi Kim, Pei Zhang, Jeonggil Ko (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-31460.txt.
Introduction
Lee and colleagues present REIS, a robot control framework that trims needless thought while keeping meaning-aware flexibility. Robot policies built on large language and vision models plan well, but answer slowly, which blocks live use. The team notes much repeat work in robot thought: back-to-back views often yield the same moves and subgoals. REIS mixes light scene checks, KV-guided action routing and careful thought to speed control. Tests on ALFRED and live robot jobs show REIS slashes thought overhead while holding strong task scores.
Catalog record
| CREATOR | Joonhee Lee, Hyunseung Shin, Hyunmi Kim, Pei Zhang, Jeonggil Ko |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 42186 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2605.31460
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
On-Device Robotic Planning: Eliminating Inference Redundancy for Efficient Decision-Making Joonhee Lee, Hyunseung Shin, Hyunmi Kim, Pei Zhang, Jeonggil Ko Abstract Reasoning-based robotic policies using large language and vision-language models achieve strong semantic planning capabilities but mostly suffer from a high inference latency that limits practical real-time deployment. In this work, we observe that robotic reasoning workloads contain substantial temporal redundancy, where consecutive observations frequently produce identical actions and subgoals. Based on this insight, we present REIS , a human cognition inspired robotic decision-making framework that minimizes unnecessary reasoning while preserving semantic adaptability. REIS combines lightweight scene gating, KV-steered affordance routing, and deliberative reasoning to accelerate robotic control under embodied constraints. Experiments on ALFRED, and real-world robotic tasks demonstrate that REIS significantly suppresses reasoning overhead while maintaining competitive task performance. Keywords: On-Device, Embodied AI, Decision-Making, Planning, Navigation, Manipulation 1 Introduction Recent large-scale foundation models have enabled a new “reason-and-act” paradigm for robotic decision-making, where intermediate reasoning improves task planning and semantic understanding. Systems such as CoT-VLA [ 55 ] and DiffusionVLA [ 47 ] demonstrate that explicit reasoning can improve generalization and manipulation performance by leveraging richer contextual and task-level information. These approaches suggest that reasoning is becoming a core component of modern robotic policies rather than a purely auxiliary capability. However, reasoning introduces substantial computational overhead that conflicts with the real-time constraints of robotic control. Robotic systems typically require inference at 10–100 ms timescales [ 23 , 12 , 38 , 36 ] , yet reasoning-heavy policies often incur significantly slower execution due to autoregressive generation or iterative denoising [ 16 , 11 , 46 , 56 , 15 ] . In dynamic environments, this latency becomes particularly problematic as observations rapidly become stale, causing reasoning outputs and planned trajectories to lose validity before execution. As a result, richer reasoning can paradoxically reduce reactivity and robustness during real-world interaction. Prior work addresses this challenge through hierarchical or dual-process architectures that separate slow semantic reasoning from fast low-level action generation. Systems such as HiRT [ 54 ] , RoboDual [ 8 ] , and DP-VLA [ 18 ] pair a large reasoning model with lightweight action specialists to improve control frequency. While effective, these approaches fundamentally treat reasoning latency as unavoidable, isolating it rather than fixing it. Consequently, they introduce a fragile trade-off between semantic freshness and control responsiveness: infrequent reasoning updates lead to stale contextual understanding when the environment shifts, while frequent updates negate the latency benefits of the fast controller entirely [ 29 , 20 , 43 , 22 ] . In this work, we argue that the primary bottleneck is not action generation, but rather in the redundant reasoning itself. Robotic reasoning pipelines repeatedly perform similar inference across temporally adjacent observations despite minimal semantic change in the environment [ 44 , 49 , 45 , 30 ] , causing increased latency. This observation motivates a new design direction. Instead of decoupling reasoning from acting, robotic systems should directly minimize unnecessary reasoning and accelerate the remaining inference process. By treating temporal continuity as an asset rather than a computational burden, we can maintain an active reasoning loop without sacrificing real-time reactivity. To this end, we present REIS (Redundancy Elimination for Inference-efficient Systems), a lightweight framework for efficient LM-based robotic decision-making systems. REIS reduces redundant frame-level reasoning while preserving semantic consistency across sequential observations. Specifically, the framework combines lightweight visual feature selection with a priming-effect-inspired inference mechanism that reuses prior reasoning context to accelerate future decisions. By evaluating with benchmarks and real-world robotic experiment data, our experiments show that REIS significantly reduces reasoning latency and redundant action generation while maintaining competitive task performance. The contributions of this work are summarized as follows: (1) We design a lightweight vision-encoder head selection method that prunes redundant visual tokens for efficient robotic perception. (2) We develop a priming-effect-inspired inference mechanism that leverages historical context to bypass redundant reasoning steps and accelerate LM-based robotic control. (3) We validate our framework across diverse foundation model architectures and robotic decision-making pipelines, demonstrating substantial latency reductions with minimal impact on task success in both simulation and real-world environments. 2 Background and Relative Works Embodied AI Robot Deployment Challenges While generalist robot policies achieve strong long-horizon performance, their computational requirements hinder edge deployment. For example, OpenVLA requires 15GB VRAM and merely generates six action tokens per second on an RTX 4090 [ 24 , 32 ] , creating a major mismatch with resource-constrained robotic platforms such as Jetson Orin NX [ 33 ] . This gap motivates the need for lightweight and efficient robotic decision-making pipelines suitable for real-world deployment. LM as Decision Maker Vision-Language Models (VLMs) enable robots to leverage semantic world knowledge for contextual reasoning and action selection [ 39 , 52 , 51 ] , unlike conventional MLP-based policies that depend primarily on spatial subgoals [ 6 , 14 , 19 ] . Hybrid frameworks such as LLM Planner combine LM-based high-level planning with low-level execution policies [ 42 , 1 , 41 , 13 ] . However, repeated sequential LM inference introduces substantial latency and computational overhead, limiting real-time robotic deployment [ 16 , 29 , 23 , 43 ] . Navigation and Manipulation Tasks Navigation and arm manipulation represent two fundamental forms of robotic physical interaction. Navigation requires scene-level reasoning for long-range movement [ 39 , 17 , 57 ] , while manipulation focuses on fine-grained object interaction [ 26 , 34 ] (e.g., ALFRED [ 40 ] ). By evaluating both domains, REIS targets a broad spectrum of robotic decision-making, from high-level environmental understanding to precise physical control [ 25 , 7 , 5 ] . 3 Motivation Addressing VLM Reasoning Redundancy Table 1: Action and subgoal redundancy rates across ALFRED and real-world environments under different levels of prior environmental knowledge. Dataset Env. Knowledge Metric Type Sample Size Redundancy Rate ALFRED Full Prior Subgoal 320,696 pairs 87.10% Action (6,574 episodes) 46.89% Real World Incomplete Action 4,262 pairs 83.04% Robotic reasoning workloads exhibit substantial temporal redundancy [ 16 , 45 , 49 , 44 ] . We analyze action and subgoal persistence across consecutive observations in ALFRED and self-collected real-world navigation trajectories. As Table 1 shows, ALFRED exhibits an 87.1% subgoal redundancy rate across 320K observation pairs and a 46.9% action redundancy rate across 6,574 episodes. In real-world environments without prior environmental knowledge, action redundancy remains high at 83% over 4,262 observation pairs. This indicates that VLMs repeatedly generate identical reasoning outputs under minimal semantic context changes, with meaningful reasoning updates occurring primarily near structural transitions such as corners or intersections. This suggests that continuous per-frame reasoning is unnecessarily and that lightweight controllers can handle redundant states while reserving VLM inference for semantically complex situations. Priming Effect on LM Inference Table 2: Ablation of KV-based priming on ALFRED action selection. B-Dir, B-Reas, KV-Dir, and KV-Reas denote direct baseline, reasoning baseline, KV-steered direct, and KV-steered reasoning, respectively. Metric B-D B-R KV-D KV-R Acc. \uparrow 21.1% 34.5% 67.6% 85.2% Corr. Ent. \downarrow 0.251 0.602 0.070 0.334 Incorr. Ent. \downarrow 0.351 0.374 0.171 0.871 Motivated by this redundancy, we investigate whether previous reasoning states can be reused as latent decision priors through KV-cache steering [ 4 ] . Rather than repeatedly regenerating similar reasoning outputs, prior