Do What You Say: Steering Vision-Language-Action Models via Runtime Reasoning-Action Alignment Verification
Yilin Wu, Anqi Li, Tucker Hermans, Fabio Ramos, Andrea Bajcsy, Claudia Pérez-D’Arpino · 2025
Read Do What You Say: Steering Vision-Language-Action Models via Runtime Reasoning-Action Alignment Verification by Yilin Wu, Anqi Li, Tucker Hermans, Fabio Ramos, Andrea Bajcsy, Claudia Pérez-D’Arpino (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2510-16281.txt.
Introduction
Yilin Wu and five co-authors fix a gap between what a robot says it will do and what it then does. Reasoning robot models write a step-by-step plan in words before acting. Yet the actions can drift from the plan, most in new settings. The authors name this drift a lack of embodied faithfulness.
Their fix runs at action time with no retraining. The same model offers several action sequences, and a simulator previews where each one leads. A vision-language model then picks the outcome that best matches the plan. Only plan-true sequences get carried out. On behavior tests, the method lifts success by up to fifteen percent over prior work. It also shines at mixing skills in new ways. The team also releases a plan-marked test set for others to probe this faithfulness gap.
Catalog record
| CREATOR | Yilin Wu, Anqi Li, Tucker Hermans, Fabio Ramos, Andrea Bajcsy, Claudia Pérez-D’Arpino |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 57 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2510-16281
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Do What You Say: Steering Vision-Language-Action Models via Runtime Reasoning-Action Alignment Verification Yilin Wu, Anqi Li, Tucker Hermans, Fabio Ramos, Andrea Bajcsy, Claudia Pérez-D’Arpino Abstract Reasoning Vision Language Action (VLA) models improve robotic instruction-following by generating step-by-step textual plans before low-level actions, an approach inspired by Chain-of-Thought (CoT) reasoning in language models. Yet even with a correct textual plan, the generated actions can still miss the intended outcomes in the plan, especially in out-of-distribution (OOD) scenarios. We formalize this phenomenon as a lack of embodied CoT faithfulness , and introduce a training-free, runtime policy steering method for reasoning-action alignment. Given a reasoning VLA’s intermediate textual plan, our framework samples multiple candidate action sequences from the same model, predicts their outcomes via simulation, and uses a pre-trained Vision-Language Model (VLM) to select the sequence whose outcome best aligns with the VLA’s own textual plan. Only executing action sequences that align with the textual reasoning turns our base VLA’s natural action diversity from a source of error into a strength, boosting robustness to semantic and visual OOD perturbations and enabling novel behavior composition without costly re-training. We also contribute a reasoning-annotated extension of LIBERO-100, environment variations tailored for OOD evaluation, and demonstrate up to 15% performance gain over prior work on behavior composition tasks and scales with compute and data diversity. Project Website at: https://yilin-wu98.github.io/steering-reasoning-vla/ I Introduction Vision Language Action (VLA) models [ 1 ] [ 2 ] , which combine pre-trained vision-language backbones with action-generation heads fine-tuned on robot data [ 3 ] , promise significant advances in generalizable robotic control. However, in practice, these models are not yet true generalists: their performance often degrades in novel scenes and over long horizons, and their ability to follow complex language instructions can be brittle [ 4 ] [ 5 ] [ 6 ] . To tackle these challenges and better exploit the vision-language backbone of the VLA, a growing body of work has introduced “reasoning VLAs” [ 7 ] [ 8 ] . This approach is inspired by the success of Chain-of-Thought (CoT) in large language models (LLMs): by producing intermediate step-by-step textual reasoning before generating a textual response, LLMs can significantly increase the complexity and reliability of language generation [ 9 ] . Similarly, reasoning VLA’s implement a form of “embodied CoT” [ 7 ] : instead of generating actions directly, the model first generates intermediate textual reasoning, often grounded by visual observations, before producing reasoning-conditioned actions. For example, given the task description of “Put both the alphabet soup and the butter in the basket” in Fig. , the model generates the textual reasoning “First I pick up the soup, then place it in the basket…, Now I need to pick up the soup” and then generates a low-level action sequence (e.g., a sequence of relative end-effector poses). However, even with the perfect textual reasoning, the VLA-sampled action sequence may lead to different outcomes than the generated plan (e.g., grasping the cream cheese instead of the soup, as shown in Fig. ). This misalignment—where the outcomes of the generated actions fail to match the textual plan—is particularly prevalent in out-of-distribution (OOD) scenarios (novel instructions, objects, backgrounds, etc.), where enforcing alignment shows up to a 20% performance gain in Sec. V-C . This also mirrors observations from prior work on training reasoning VLAs [ 10 ] , where the reasoning capabilities of the models are stronger than language-conditioned action generation capabilities. Analogous to Chain-of-Thought (CoT) faithfulness in LLMs, which questions whether a model’s generated reasoning accurately reflects the process used to derive an output [ 11 ] , we formalize this reasoning-action misalignment as a robotics version of lack of embodied CoT faithfulness : the expectation that outcomes described by a VLA’s textual reasoning will be reliably realized by its subsequent low-level actions. To address this gap, we introduce a runtime reasoning-action alignment method. This approach enforces embodied CoT faithfulness by actively steering a reasoning VLA’s actions to match its own textual reasoning during execution. Specifically, right before the reasoning is updated, we autoregressively generate multiple action sequences from the base model, conditioned on the current textual plan and the observation from the preceding step (center, Fig. ). We predict the outcomes of action samples via parallel simulation, and leverage an off-the-shelf Vision Language Model (VLM) verifier to score the alignment between the outcome and the preceding text plan. Only the action sequence that induces the correct outcome is executed on the robot. Our contributions are summarized as follows: • Runtime Policy Steering for Reasoning-Action Alignment: we propose a new runtime framework for ensuring that a VLA’s generated actions induce the outcomes described by its own textual reasoning. Even in in-distribution (ID) scenarios, we find that reasoning-action alignment improves task success by 8% and preserves long-horizon semantic coherence without needing any additional fine-tuning data. More benefits are shown in OOD scenarios below. • Extensive Generalization Experiments and Scaling Analysis: Through a controlled series of experiments on OOD shifts and behavior composition tasks in our extended version of LIBERO benchmark [ 12 ] , we find that our method outperforms relevant baselines by up to 15%. This performance gap widens as the fine-tuning dataset grows, since a stronger policy generates a better set of candidate actions for our verifier to select from. • A Reasoning-Annotated VLA Dataset and an Extended Benchmark for Generalization Tests: We contribute an open-sourced reasoning-annotated LIBERO-100 dataset for reasoning VLA training. We also contribute an extension of the LIBERO benchmark with new task descriptions, visual variations, and new behavior compositions. We use these for our evaluations but also plan to release it for the community to continue to study generalization capabilities of reasoning VLAs. II Related Work Vision Language Action Models & Reasoning. Large robotics datasets [ 3 ] have enabled the creation of generalist Vision Language Action (VLA) models [ 1 ] [ 2 ] . Despite their success, these VLA models often struggle with long-horizon tasks and out-of-distribution (OOD) generalization due to compounding errors and a limited ability to follow complex or abstract language instructions. To improve robustness, many approaches augment VLAs with Chain-of-Thought reasoning, either through hierarchical planner-controller dual systems [ 5 ] or an unified model that interleave intermediate reasoning and action generation [ 7 ] [ 8 ] [ 13 ] [ 14 ] . We build on the unified approach but identify a critical failure mode: a reasoning-action mismatch, where motor actions do not faithfully execute the model’s self-generated textual plan. Runtime Optimization and Steering of Large Models. Runtime optimization improves model’s performance without costly retraining. In robotics, runtime steering has improved the alignment between human intent and robot actions in diffusion policies [ 15 ] , and in VLA policies by sampling multiple action chunks and verifying them against learned critic. These critics are typically either offline Q-functions [ 16 ] (which can struggle in novel scenarios), or specially fine-tuned VLMs from synthetic preference labels which may not correlate with true task success [ 6 ] . Critically, these methods verify actions in isolation and do not address the reasoning-action faithfulness gap—the challenge of ensuring that a sequence of low-level actions are semantically coherent with a model’s self-generated textual plan. Inspired by