VLA-Trace: Diagnosing Vision-Language-Action Models through Representation and Behavior Tracing
Haoyuan Shi, Xiancong Ren, Yingji Zhang, Qinfan Zhang, Jiayu Hu, Haozhe Shan, Han Dong, Jinpeng Lu, Yinda Chen, Yi Zhang, Yong Dai, Xiaozhu Ju · 2026
Read VLA-Trace: Diagnosing Vision-Language-Action Models through Representation and Behavior Tracing by Haoyuan Shi, Xiancong Ren, Yingji Zhang, Qinfan Zhang, Jiayu Hu, Haozhe Shan, Han Dong, Jinpeng Lu, Yinda Chen, Yi Zhang, Yong Dai, Xiaozhu Ju (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-30117.txt.
Introduction
Haoyuan Shi and eleven colleagues present VLA-Trace, a diagnostic framework for vision-language-action robot control models. It traces one evidence chain from shifting representations to causal control links and outward behaviour. Kernel alignment tracks drift, attention knockout maps pathways, and rollout probes test grounding, shortcuts and semantic following. Tests on two models reveal distinct adaptation patterns, different routing and layer links. Both models ground trajectories well in vision but follow fine-grained meaning poorly. The findings point toward adaptation that preserves representations, causal circuits and compositional semantic control.
Catalog record
| CREATOR | Haoyuan Shi, Xiancong Ren, Yingji Zhang, Qinfan Zhang, Jiayu Hu, Haozhe Shan, Han Dong, Jinpeng Lu, Yinda Chen, Yi Zhang, Yong Dai, Xiaozhu Ju |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 97420 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2605.30117
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
VLA-Trace: Diagnosing Vision-Language-Action Models through Representation and Behavior Tracing Haoyuan Shi, Xiancong Ren, Yingji Zhang, Qinfan Zhang, Jiayu Hu, Haozhe Shan, Han Dong, Jinpeng Lu, Yinda Chen, Yi Zhang, Yong Dai, Xiaozhu Ju Abstract Understanding how Vision-Language-Action (VLA) models transform multimodal knowledge into embodied control remains an open challenge. We present VLA-Trace, a progressive diagnostic framework that analyzes VLA models through a unified evidence chain from representation dynamics to causal control attribution and behavioral manifestation. It specifically combines cross-modal and checkpoint-drift centered kernel alignment (CKA) to trace representation evolution, attention knockout interventions to identify modality-specific control pathways, and rollout-level behavioral probes to examine grounding, shortcut dependence, and semantic following. Experiments on π 0.5 \pi_{0.5} and OpenVLA reveal three key findings. First, the two models exhibit distinct modality-specific adaptation dynamics during VLA finetuning. Second, they rely on different multimodal routing strategies and layer-wise dependencies during action decoding. Third, although VLA policies excel at visually grounded trajectory generation, they remain limited in fine-grained semantic following. These findings highlight future directions for representation-preserving adaptation, causal VLA circuits, and compositional semantic control. 1 Introduction Vision-Language-Action (VLA) models have become a promising paradigm for advancing embodied intelligence in real-world environments. Built upon large-scale pretrained vision-language models (VLMs) 1 ; 33 ; 34 , recent systems, including X-VLA 37 , OpenVLA-style models 18 ; 17 , and \pi -style models 3 ; 27 ; 14 ; 13 , achieve strong robotic control performance by unifying visual-language perception and action generation within a single policy framework. However, despite these successes, the internal mechanisms underlying action generation remain poorly understood. This issue is critical because, without understanding how robotic policy learning reshapes internal representations, it is difficult to diagnose model failures and, consequently, to design more effective VLA architectures. In particular, it remains unclear whether multimodal knowledge is preserved, how visual and linguistic signals are aligned during policy learning, and which modalities govern action decoding. Figure 1: Overview of VLA-Trace. The framework progressively diagnoses VLA models by tracing representation dynamics, identifying causal pathways for action decoding, and probing behavioral reliance on shortcut dependence. To answer these questions, we propose VLA-Trace , a systematic three-stage analysis framework that progressively investigates VLA models from latent representation evolution to causal control attribution and finally to behavioral dependence. As illustrated in Fig. 1 , instead of treating the policy as an end-to-end system, we construct a unified evidence chain for interpreting VLA adaptation. First, we employ cross-modal and checkpoint-drift centered kernel alignment (CKA) to identify whether vision-language prior is preserved or reshaped during fine-tuning. Second, we perform targeted attention knockouts to examine whether vision-language information is functionally utilized. Third, we combine attention localization and input editing to assess whether the resulting routing patterns reflect robust vision-language grounding or shallow visual shortcuts. Through this framework, we uncover the following key observations: \pi_{0.5} and OpenVLA exhibit distinct modality-specific adaptation dynamics during VLA finetuning. Specifically, our CKA analysis reveals that \pi_{0.5} exhibits highly unstable layer-wise cross-modal fusion dynamics, primarily reorganizing textual representations into task-conditioned control features. In contrast, OpenVLA exhibits smoother but often weaker intermediate-layer image–text alignment, while preserving its text-pooled representations and reorganizing primarily vision-pooled and joint-pooled subspaces. \pi_{0.5} and OpenVLA exhibit distinct multimodal routing strategies and layer-wise dependency during action decoding. Attention knockout experiment across visual and textual pathways illustrates that \pi_{0.5} is dominated by a concentrated visual-to-action routing pathway during decoding, which limits the functional role of language. Conversely, OpenVLA distributes control-relevant information across both visual and textual access. VLA policies excel at visually grounded trajectory but remain limited in fine-grained semantic following. Attention localization and input editing reveal a gap between visual grounding and semantic following. Although policies localize to manipulation-relevant regions, they often ignore fine-grained semantic modifications, suggesting strong visually grounded trajectory imitation and weak compositional language control. These observations highlight future directions for effective representation preservation and adaptation, causal VLA circuits, and compositional semantic control in VLA systems. Overall, this work proposes a systematic framework for probing how vision-language prior knowledge influences action control in VLA models. By identifying these bottlenecks in multimodal adaptation, causal routing, and semantic grounding, our progressive diagnosis provides guidance for developing more interpretable and robust next-generation VLA models. 2 Related Work In this section, we review related work around two topics: VLA models and mechanistic and representation analysis of VLA models . These areas motivate the design of VLA-Trace as a progressive analysis framework. 2.1 Vision-Language-Action Models VLA models adapt the vision-language priors of VLMs to embodied control. Unlike world foundation models 26 ; 22 ; 36 , which model environment dynamics through high-dimensional video generation, VLAs offer a more direct and lightweight paradigm by mapping multimodal representations to robot actions 28 ; 4 ; 38 ; 21 . Recent advances, including Octo 32 , OpenVLA 18 , and \pi -style policies 3 ; 14 ; 13 ; 12 , have shown strong generalization across tasks and embodiments, while systems such as Gemini Robotics 31 ; 30 and GR00T N1 2 further extend VLA capabilities to complex manipulation and control. Despite these advances, it remains unclear how VLM representations are reshaped during VLA adaptation and whether pretrained knowledge is preserved and functionally utilized during action generation. Prior work shows that direct VLM-to-VLA finetuning can degrade visual and language representations 8 ; 9 , motivating a closer examination of how representation changes influence embodied behavior. 2.2 Mechanistic Analysis of VLA Models Recent studies investigate the underlying mechanisms of VLA models, which can be broadly categorized into three directions: Representation-level analysis investigates the information encoded in hidden states. Probing analyses show that hidden states can encode symbolic properties, relations, action states, and latent transition information, suggesting that semantically meaningful state information is present in VLA representations 5 ; 9 ; 7 . Causal mechanism analysis moves toward mechanistic interpretability by steering activations, identifying task-relevant attention heads, and extracting sparse features that influence policy behavior 25 ; 11 ; 29 . Input intervention analysis examines grounding and robustness at input levels, revealing that VLA policies remain sensitive to distractors, background changes, object grounding errors, and failures to follow language instructions 16 ; 20 ; 10 ; 35 ; 6 . Nevertheless, existing analyses remain fragmented. Representation-level studies reveal what information is encoded, but not whether it is causally used during action decoding. Input intervention analyses expose behavioral failures, yet often leave the internal pathway from representation shifts to failure modes unclear. To bridge this gap, we propose VLA-Trace , a unified diagnostic framework that integrates representation geometry analysis, causal analysis, and input intervention into a stage-wise progressive pipeline to understand embodied control behavior. 3 Representation Analysis Framework Stage Analysis Probe Benchmarks Models S1 Cross-Modal CKA vision–language alignment LI/CO P/O Checkpoint-Drift CKA stage-wise drift LI P/O S2 Attention Knockout modality reliance LI/CV/RT P/O/F Layer-Wise Knockout layerwise control LI/CV/RT P/O/F S3 Attention IoU region overlap L10 P/O Visual Patch Mask object/spatial/background LI/CV/RT/SP P/O/F/X Input Editing image/prompt sensitivity L10 P/O Table 1: Overview of the VLA-Trace framework. Probes progress from representation geometry to causal attribution and behavioral manifestation; italic text summarizes each probe’s target. Blue benchmark tags: LI =LIBERO, CO =COCO, CV =CALVIN, RT =RoboTwin2.0, SP =Simpler, L10 =LIBERO-10. Green model tags: P = \pi_{0.5} , O =OpenVLA, F =OFT,