Humanoid World Models : Open World Foundation Models for Humanoid Robotics
Qasim Ali, Aditya Sridhar, Shahbuland Matiana, Alex Wong, Mohammad Al-Sharman · 2025
Read Humanoid World Models : Open World Foundation Models for Humanoid Robotics by Qasim Ali, Aditya Sridhar, Shahbuland Matiana, Alex Wong, Mohammad Al-Sharman (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2506-01182.txt.
Introduction
Muhammad Qasim Ali and co-authors wrote a paper from 2025 on world models for human-shaped robots. A world model guesses what will happen next if the robot takes a given act. That guess lets a robot plan far ahead and gives it fake practice data. Humanoid robots suit the places built for people, but open, messy settings still strain them.
Their models watch through the robot's own eyes and forecast the next stretch of video, steered by control signals. They trained two kinds of video model on one hundred hours of humanoid demonstrations. A weight-sharing trick cut the models' size by a third to a half with little loss in quality. The models are light and open, built to train and run on one or two graphics cards. The team aims them at small labs that lack big computing power.
Catalog record
| CREATOR | Qasim Ali, Aditya Sridhar, Shahbuland Matiana, Alex Wong, Mohammad Al-Sharman |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 35 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2506-01182
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Humanoid World Models : Open World Foundation Models for Humanoid Robotics Qasim Ali, Aditya Sridhar, Shahbuland Matiana, Alex Wong, Mohammad Al-Sharman Abstract Humanoid robots have the potential to perform complex tasks in human-centered environments but require robust predictive models to reason about the outcomes of their actions. We introduce Humanoid World Models (HWM), a family of lightweight, open-source video-based models that forecast future egocentric observations conditioned on actions. We train two types of generative models—Masked Transformers and Flow-Matching—on 100 hours of humanoid demonstrations. Additionally, we explore architectural variants with different attention mechanisms and parameter-sharing strategies. Our parameter-sharing techniques reduce model size by 33–53% with minimal impact on performance or visual fidelity. HWM is designed to be trained and deployed in practical academic and small-lab settings, such as 1–2 GPUs. † † affiliationnotice: Equal contribution 1 Introduction Autonomous humanoid robots have the potential to transform both industry and daily life by automating tasks that are dull, dangerous, or physically demanding ( Goswami & Vadakkepat 2019 ) . Their human-like morphology allows them to operate in spaces built for people, interact naturally with humans, and easily learn from teleoperated demonstrations. Realizing this promise, however, requires more than hardware: these agents must be equipped with sophisticated models of reasoning to navigate the complexity and unpredictability of real-world environments. While large multimodal models ( Bommasani et al. 2022 ; Zhang et al. 2024 ; Kim et al. 2024 ; Black et al. 2024 ) show some promise in generalization and reasoning, they often lack the accuracy, reliability, and robustness needed for embodied AI in open-world settings ( Tong et al. 2024 ; Pfeifer & Iida 2004 ) . As a result, they struggle to meet the demands of embodied agents that must act safely and effectively in dynamic, unstructured environments ( Pfeifer & Iida 2004 ; Li et al. 2024 ; Duan et al. 2024 ) . Figure 1: Overview of Humanoid World Model. One avenue for improving humanoid intelligence and control is through World Models —predictive models trained to forecast future outcomes based on past observations and actions ( Ha & Schmidhuber 2018 ) . In our setting, these function as action-conditioned video generators: they predict future visual states as sequences of frames, enabling agents to simulate the consequences of candidate actions ( Du et al. 2023 ) . By simulating the outcomes of actions before trying them in the real world, world models support long-horizon planning, counterfactual reasoning, and data-efficient policy learning through synthetic rollouts ( Yang et al. 2024 ; Yang et al. 2023 ) . Despite recent progress in video generation, most models are built for entertainment applications ( Liu et al. 2024 ; Polyak et al. 2024 ) , emphasizing visual appeal over physical, ego-centric plausibility. Many world models remain closed-source, require large-scale compute for training or inference, or are not designed for humanoid robots ( Yang et al. 2023 ; NVIDIA et al. 2025 ; Zhu et al. 2024 ) . As a result, there is a clear gap: few open-source models are both physically grounded for humanoid robots and lightweight enough to train or run inference on modest academic hardware (e.g., 2–3 GPUs). To address this, we ask: Can we build a physically plausible, humanoid-specific world model that runs on just two GPUs? We introduce Humanoid World Models (HWM), an open-source lightweight world model for humanoid robotics on 100 hours of humanoid video demonstrations. We investigate two distinct video generation paradigms: Masked Transformers and Flow-Matching models. Drawing from recent advances in image and video generation, we explore four architectural variations within each framework. These variations are defined along two axes: (1) joint vs. cross-attention mechanisms, and (2) shared vs. separate parameters across token streams. This design space is motivated by successful architecture strategies from the much more explored text-to-image models. Our experiments show that, given our dataset and compute constraints, Masked Transformers consistently outperformed Flow-Matching models—even when the latter used more parameters and were trained for longer. Across both families of models, we observed that the different architectural variants performed comparably in most scenarios. However, key trends emerged: within the Masked Transformer framework, the joint attention variant achieved the best overall performance. In contrast, for Flow-Matching models, split attention proved most effective. Importantly, we found that parameter-sharing strategies yielded near-identical performance to their non-shared counterparts while reducing parameter counts by 33–53%, significantly lowering computational requirements. These results suggest that architectural and efficiency trade-offs—particularly attention design and parameter sharing—can be leveraged to build lightweight, performant world models without compromising quality. Related Works 1.1 Humanoid Robots Humanoid robots (Humanoids) are actuated, bipedal, and bimanual robots designed to anthropomorphically resemble the human body structure ( Goswami & Vadakkepat 2019 ) . Their morphology is well-suited for operating in human-centered environments, enabling effective interaction and physical compatibility with everyday settings. This embodiment facilitates deployment across a wide range of real-world domains, including households ( Imtiaz & Khan 2024 ) , manufacturing facilities ( Hirose & Ogawa 2007 ) , elderly care homes ( Imtiaz & Khan 2024 ) , and clinical or medical environments ( Goswami & Vadakkepat 2019 ) . Beyond physical compatibility with environments, the human-like form of humanoids allows for more natural interaction with people and the ability to imitate human behaviors ( Vianello et al. 2021 ) . Furthermore, collecting demonstration data is relatively straightforward for these platforms, as they can directly mimic human motions and tasks ( Zhao et al. 2023 ) . Our world models are trained specifically for humanoids, but can easily be extended to other embodiments. 1.2 World Models A core challenge in robotics is enabling agents to perceive their environment, reason over possible actions, and plan goal-directed behavior—especially in novel or unstructured settings. Foundation models ( Bommasani et al. 2022 ) attempt to generalize across tasks through large-scale, multimodal training. Recent approaches include Vision-Language-Action (VLA) models that predict actions from video and language inputs ( Black et al. 2024 ; Kim et al. 2024 ) , and large language or vision-language models used as high-level planners ( Liang et al. 2023 ; Li et al. 2024 ; Wang et al. 2024 ) . However, these methods often struggle with spatial reasoning ( Tong et al. 2024 ) , continuous sensorimotor processing, and require complex prompting strategies ( Li et al. 2024 ) . In contrast, video generation models offer a more grounded alternative for embodied agents. Video captures fine-grained physical and temporal structure that language alone cannot express ( Yang et al. 2024 ) . When trained to predict future video frames given past observations and actions, these models serve as World Models —internal simulators that forecast the outcomes of potential action sequences ( Ha & Schmidhuber 2018 ; Du et al. 2023 ) . This enables agents to plan through imagination, reason counterfactually, and generate synthetic experience for policy training. In this work, we develop a humanoid-specific world model implemented as a video generator: it predicts future egocentric video frames from past video and action sequences. This generative modeling approach allows us to simulate plausible futures, supporting both planning and data-efficient learning in complex environments. 1.3 Video Generation Models Early video generation relied on GANs ( Goodfellow et al. 2014 )