Mechanistic Interpretability for Steering Vision-Language-Action Models

Bear Häon, Kaylene Stocking, Ian Chuang, Claire Tomlin · 2025

Read Mechanistic Interpretability for Steering Vision-Language-Action Models by Bear Häon, Kaylene Stocking, Ian Chuang, Claire Tomlin (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2509-00328.txt.

Description

Bear Häon, Kaylene Stocking, Ian Chuang, and Claire Tomlin introduce a framework in 2025 for interpreting and steering vision-language-action models from inside. Such models promise generalist embodied agents that adapt fast to new tasks and settings. Classic robotics steers by explicit models of motion and control, while learned policies offer little insight into their choices.

Borrowing from mechanistic interpretability work on language models, the team projects internal activations onto the token embedding basis. Sparse semantic directions emerge, such as speed, that can be intervened on at inference time. Direct steering of behavior follows from those directions. The method aims to bring robustness and explainability to learned robot policies.

Catalog record

CREATORBear Häon, Kaylene Stocking, Ian Chuang, Claire Tomlin
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE45 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2509-00328

Browse the full library · About the Tower of Babel