EgoAgent: A Joint Predictive Agent Model in Egocentric Worlds

Lu Chen * Yizhou Wang * Shixiang Tang Qianhong Ma Tong He, Wanli Ouyang Xiaowei Zhou Hujun Bao Sida PengState Key Lab of CAD&CG, Zhejiang University The Chinese University of Hong KongShanghai Jiao Tong University Shanghai Artificial Intelligence Laboratory, ∗ Equal contribution Corresponding author · 2025

Read EgoAgent: A Joint Predictive Agent Model in Egocentric Worlds by Lu Chen * Yizhou Wang * Shixiang Tang Qianhong Ma Tong He, Wanli Ouyang Xiaowei Zhou Hujun Bao Sida PengState Key Lab of CAD&CG, Zhejiang University The Chinese University of Hong KongShanghai Jiao Tong University Shanghai Artificial Intelligence Laboratory, ∗ Equal contribution Corresponding author (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2502-05857.txt.

Description

Lu Chen and co-authors build EgoAgent, one model that sees, predicts, and acts from a first-person view, in a paper from two thousand twenty-five. Past work trains split models for each skill, so the skills cannot teach each other. The new model weaves states and actions into one sequence inside a single transformer.

Split branches watch and forecast with a time-skewed design that aids joint learning. Tests covered image sorting, future view forecasting, and three-dimensional body motion forecasting. The joint model beat split rivals across those tasks. Code and trained models are set for public release. Readers in robot vision get a human-like loop of see, guess, and act in one net.

Catalog record

CREATORLu Chen * Yizhou Wang * Shixiang Tang Qianhong Ma Tong He, Wanli Ouyang Xiaowei Zhou Hujun Bao Sida PengState Key Lab of CAD&CG, Zhejiang University The Chinese University of Hong KongShanghai Jiao Tong University Shanghai Artificial Intelligence Laboratory, ∗ Equal contribution Corresponding author
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE49 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2502-05857

Browse the full library · About the Tower of Babel