Act to See, See to Act: Diffusion-Driven Perception-Action Interplay for Adaptive Policies

Jing Wang, Weiting Peng, Jing Tang, Zeyu Gong, Xihua Wang, Bo Tao, Li Cheng · 2025

Read Act to See, See to Act: Diffusion-Driven Perception-Action Interplay for Adaptive Policies by Jing Wang, Weiting Peng, Jing Tang, Zeyu Gong, Xihua Wang, Bo Tao, Li Cheng (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2509-25822.txt.

Introduction

Jing Wang and six co-authors present a robot learning method in two thousand twenty-five that ties perception and action together. Imitation learning usually treats seeing and acting as separate steps. Humans, by contrast, use each to shape the other when they adapt their behavior.

Their method, DP-AG, encodes observations into a probabilistic latent state and evolves it with an action-guided equation. A signal from the diffusion policy's noise predictor acts as a structured force on that state. A cycle-consistent contrastive loss keeps perception and action updates in step. The authors prove a variational bound and a continuity result for the scheme. Tests on simulation benchmarks and a real robot arm beat leading methods.

Catalog record

CREATORJing Wang, Weiting Peng, Jing Tang, Zeyu Gong, Xihua Wang, Bo Tao, Li Cheng
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE128 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2509-25822

Browse the full library · About the Tower of Babel