Revisiting Embodied Chain-of-Thought for Generalizable Robot Manipulation

Nan Sun Yuan Zhang Yongkun Yang Wentao Zhao Peiyan Li Jun Guo Wenxuan Song Pengxiang Ding Runze Suo Yifei Su Xin Xiao Xinghang Li Huaping Liu Tsinghua University Xiaomi Robotics Peking University CASIA HKUST(GZ) Zhejiang University Fudan University Wuhan University Shanghai Innovation Institute Project Leader Corresponding Author · 2026

Read Revisiting Embodied Chain-of-Thought for Generalizable Robot Manipulation by Nan Sun Yuan Zhang Yongkun Yang Wentao Zhao Peiyan Li Jun Guo Wenxuan Song Pengxiang Ding Runze Suo Yifei Su Xin Xiao Xinghang Li Huaping Liu Tsinghua University Xiaomi Robotics Peking University CASIA HKUST(GZ) Zhejiang University Fudan University Wuhan University Shanghai Innovation Institute Project Leader Corresponding Author (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2606-03784.txt.

Introduction

Sun and co-authors revisit embodied chain-of-thought for robot control at record scale. Reasoning traces should bridge language and physical action. Their best form and integration stayed unclear. The team curates the largest embodied CoT corpus yet. It holds nearly a million trajectories and over 2,500 hours of data.

Effective traces ground high-level meaning in concrete action guidance. End-effector moves and image-space paths work; abstract reasoning alone adds little. Explicit traces also fail as an autoregressive prefix, since errors compound at inference. Their model, ERVLA, trains with a CoT-dropout plan. Rich traces teach the model, yet it predicts actions directly at run time. Success reaches about 87 percent on LIBERO-Plus and 53 percent on VLABench. Real-robot tests beat strong rivals, above all on ambiguous and long tasks. Code, data, and checkpoints are public.

Catalog record

CREATORNan Sun Yuan Zhang Yongkun Yang Wentao Zhao Peiyan Li Jun Guo Wenxuan Song Pengxiang Ding Runze Suo Yifei Su Xin Xiao Xinghang Li Huaping Liu Tsinghua University Xiaomi Robotics Peking University CASIA HKUST(GZ) Zhejiang University Fudan University Wuhan University Shanghai Innovation Institute Project Leader Corresponding Author
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE96 KB
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2606.03784

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Revisiting Embodied Chain-of-Thought for Generalizable Robot Manipulation Nan Sun Yuan Zhang Yongkun Yang Wentao Zhao Peiyan Li Jun Guo Wenxuan Song Pengxiang Ding Runze Suo Yifei Su Xin Xiao Xinghang Li Huaping Liu Tsinghua University Xiaomi Robotics Peking University CASIA HKUST(GZ) Zhejiang University Fudan University Wuhan University Shanghai Innovation Institute Project Leader Corresponding Author Abstract Embodied chain-of-thought (CoT) aims to bridge linguistic reasoning with robotic control, yet its effective form and integration remain underexplored. In this paper, we revisit embodied CoT for robotic control at an unprecedented scale. We curate the largest embodied CoT corpus to date, comprising 978,743 trajectories, 226.3M samples, and 2592.5 hours of data. Through extensive experiments, we show that effective CoT must ground high-level semantic understaning in concrete linguistic action guidance – such as end-effector movement and image-space trajectories – whereas high-level reasoning alone yields marginal gains. More importantly, we identify that explicit CoT does not scale reliably as an autoregressive action prefix, suffering from compounding errors during inference. To address these challenges, we propose ERVLA, a vision-language-action (VLA) model that effectively leverages linguistic reasoning in generalizable robot manipulation. ERVLA is trained using a CoT-dropout strategy, allowing the model to leverage rich reasoning traces during training while predicting actions directly without CoT during inference to bypass autoregressive instability. This approach enables reliable scaling with increasing pre-training data. ERVLA achieves state-of-the-art results on LIBERO-Plus with an 86.9% success rate and reaches 53.2% on VLABench, showcasing superior performance in out-of-distribution settings. Furthermore, ERVLA outperforms competitive state-of-the-art baselines in real-robot experiments, especially in handling semantic ambiguity and long-horizon tasks. Code, data, and model checkpoints will be released. https://taoshuaiz.github.io/ERVLA/ Figure 1: Overview of ERVLA. We build the largest and most comprehensive embodied CoT corpus to date, which serves as the reasoning pre-training foundation for ERVLA. ERVLA uses CoT supervision, auxiliary action queries, knowledge-truncated KV conditioning, and reasoning dropout to internalize reasoning into action-aware VLM representations and predict continuous actions without mandatory test-time CoT. It achieves state-of-the-art performance on two simulation benchmarks, benefits from more embodied CoT data, and shows stronger real-world robustness under semantic ambiguity and long-horizon tasks. 1 Introduction Vision-language-action (VLA) models have emerged as a promising paradigm for generalizable robot control by transferring large-scale visual and semantic priors from pre-trained vision-language models (VLMs) into embodied decision making [ 5 , 20 , 25 , 65 ] . By grounding language instructions in visual observations and mapping them to robot actions, these models provide a unified interface for open-vocabulary and long-horizon manipulation. However, enhanced perception and broader semantic coverage do not inherently guarantee better action generation [ 57 ] . In practice, VLA models often struggles in complex, out-of-distributed scenarios which demands deliberation prior to execution. This bottleneck has motivated the development of reason-before-act policies, which leverages intermediate reasoning to guide action generation. Previous embodied chain-of-thought (ECoT) methods [ 55 ] showcase that supervising explicit reasoning traces via next-token prediction can improve robustness and generalization. Since then, the design space has rapidly expanded, encompassing diverse reasoning modalities, architectural designs for reasoning-action coupling, and specialized training recipes [ 10 ] . Despite the progress, a systematic understanding of reasoning in VLA remains elusive. The essence of ECoT is not to make robots “say more,” but to translate the semantic abstractions learned by VLMs from internet-scale data into intermediate representations that are genuinely useful for action generation. Existing methods instantiate reasoning through various lenses, including scene understanding, subtask decomposition, spatial grounding, end-effector trajectory prediction, and future-frame prediction [ 24 , 55 , 44 , 59 , 20 ] . However, these choices are tightly coupled with specific architectures and training objectives, making it difficult to identify what actually improves control. Consequently, the field lacks a principled understanding of which forms of embodied reasoning truly facilitate effective action generation. A second unresolved question is how should reasoning interact with the policy. Early ECoT-style formulations [ 55 , 44 ] model reasoning as a prefix of action generation: the model first predicts embodied reasoning traces and then generates discretized action tokens through next-token prediction. This design is simple but slow and brittle, since long reasoning traces increase latency and action generation depends heavily on preceding reasoning, leading to compounding errors during inference. Recent works decouple reasoning from action by distilling reasoning into latent plans [ 34 , 18 ] , conditioning diffusion-based action heads on selected reasoning states [ 19 , 45 ] , or learning from reasoning data during training while predicting actions without CoT at inference time [ 10 ] . This raises another central question: should reasoning be taken as a visible trace, a latent variable, or a training signal that reshapes the policy representation? A third but equally important question is whether embodied CoT can effectively scale. Recent VLA pre-training efforts have begun to include richer semantic or reasoning-related supervision [ 20 , 45 ] . However, the scaling behavior of CoT data remains unclear. Public reasoning-enhanced robotic datasets are scarce, given that embodied CoT is expensive to annotate and often requires dense labeling of multiple perception tasks within a large amount of images. Consequently, a large-scale embodied CoT dataset and a systematic scaling study is essential to determine whether scaling reasoning supervision yields stronger action generation. In this paper, we revisit embodied CoT for robot control on an unprecedented scale, with a focus on linguistic CoT (see Fig. 1 ). We first establishes a comprehensive CoT format and an auto-labeling pipeline, extending prior single-view reasoning to multi-view settings. Based on this pipeline, we curate the largest embodied CoT robot dataset to date, consisting of 978,743 trajectories, 226.3M samples, and 2592.5 hours of data. We perform extensive experiments on this dataset in a unified setting to compare the effectiveness of various CoT signals. We identifies a critical yet overlooked failure mode that we term CoT contamination . Noisy labels are inevitable in large-scale auto-labeled CoT annotations. These dense but inaccurate supervision, especially highly-jitterred signals such as unstable bounding boxes and drifting end-effector coordinates across consecutive similar frames, can hinder adaptation to embodied control by imposing inconsistent supervision on semantically similar observations. We show that this issue can be mitigated through improved annotation pipelines, sparse supervision for unstable grounding tasks, and properly designed reasoning dropout. Importantly, reasoning dropout is not merely a regularizer, it acts as training scaffolding that reduces noisy CoT contamination and reshapes internal VLA representations. This leads to a central conclusion: embodied CoT should be treated not merely as a test-time verbalization channel, but as a training signal that reshapes the representation space from perception to action. Building on these insights, we present ERVLA , an E mbodied R easoning V ision- L anguage- A ction model trained on a large-scale robot dataset with embodied CoT (see Fig. 2 ). ERVLA leverages a mixture-of-transformers architecture [ 5 , 20 , 45 ] , where a vision-language model [ 2 ] processes visual observations and text instructions, and a diffusion transformer (DiT) [ 38 ] generates continuous robot actions via flow matching [ 30 ] . The DiT is conditioned on the robot state and the key-value (KV) cache from the VLM, allowing action generation to directly access semantic representations produced by the reasoning model. Drawing inspiration from

Read the complete text