Simulation Distillation: Pretraining World Models in Simulation for Rapid Real-World Adaptation
Jacob Levy, Tyler Westenbroek, Kevin Huang, Fernando Palafox, Patrick Yin, Shayegan Omidshafiei, Dong-Ki Kim, Abhishek Gupta, David Fridovich-Keil, https://sim-dist.github.io · 2026
Read Simulation Distillation: Pretraining World Models in Simulation for Rapid Real-World Adaptation by Jacob Levy, Tyler Westenbroek, Kevin Huang, Fernando Palafox, Patrick Yin, Shayegan Omidshafiei, Dong-Ki Kim, Abhishek Gupta, David Fridovich-Keil, https://sim-dist.github.io (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2603-15759.txt.
Introduction
Levy and co-authors pretrain robot world models inside physics simulators. Real-world training data for action-aware world models is scarce and mixed in quality. Their Simulation Distillation method distills simulator structure into the model first. The world model then plans from raw real sensor input. At adaptation time, encoder, reward model, and value function transfer as-is. Only the latent dynamics model updates, from real prediction errors. Tests cover contact-rich manipulation and quadruped locomotion. Gains came fast, while rival methods stalled or degraded.
Catalog record
| CREATOR | Jacob Levy, Tyler Westenbroek, Kevin Huang, Fernando Palafox, Patrick Yin, Shayegan Omidshafiei, Dong-Ki Kim, Abhishek Gupta, David Fridovich-Keil, https://sim-dist.github.io |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 61375 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2603.15759
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Simulation Distillation: Pretraining World Models in Simulation for Rapid Real-World Adaptation Jacob Levy, Tyler Westenbroek, Kevin Huang, Fernando Palafox, Patrick Yin, Shayegan Omidshafiei, Dong-Ki Kim, Abhishek Gupta, David Fridovich-Keil, https://sim-dist.github.io Abstract Simulation-to-real transfer remains a central challenge in robotics, as mismatches between simulated and real-world dynamics often lead to failures. While reinforcement learning offers a principled mechanism for adaptation, existing sim-to-real finetuning methods struggle with exploration and long-horizon credit assignment in the low-data regimes typical of real-world robotics. We introduce Simulation Distillation ( SimDist ), a sim-to-real framework that distills structural priors from a simulator into a latent world model and enables rapid real-world adaptation via online planning and supervised dynamics finetuning. By transferring reward and value models directly from simulation, SimDist provides dense planning signals from raw perception without requiring value learning during deployment. As a result, real-world adaptation reduces to short-horizon system identification, avoiding long-horizon credit assignment and enabling fast, stable improvement. Across precise manipulation and quadruped locomotion tasks, SimDist substantially outperforms prior methods in data efficiency, stability, and final performance. Project website and code: https://sim-dist.github.io Fig. 2 : SimDist overview. 1) An expert policy, policy checkpoints, and a value function are trained in simulation using privileged state. 2) Large-scale training data are generated by combining expert and sub-optimal policies with contiguous action perturbations, yielding diverse trajectories with dense reward and value supervision. 3) A planning-oriented latent world model is pretrained on this data, learning representations, dynamics, rewards, and values from raw observations. 4a) At deployment, the learned representation and dense reward and value models are transferred to the real robot to enable planning with the latent dynamics. 4b) Real-world data is then used to finetune only the dynamics via supervised system identification, with representations, rewards, and values frozen. Deployment and finetuning are iterated, enabling rapid and stable real-world adaptation. I Introduction Robotic systems must operate in complex, novel environments where success requires rapid deployment-time adaptation. Learning-based approaches [ 19 , 32 , 25 ] have shown great promise in this setting, but training robots directly in the real world remains challenging due to safety, cost, and time constraints. As a result, simulators have emerged as a powerful tool for scaling robot learning, providing access to large volumes of interaction data and broad coverage over initial conditions and environment parameters [ 42 , 58 , 39 ] . While this strategy has enabled many recent success stories [ 30 , 27 , 44 ] , sim-to-real failures remain common due to the inevitable dynamics “gap” between the two domains. Bootstrapping behavior in simulation and finetuning in the real world with limited on-domain data to overcome dynamics mismatch is an attractive paradigm for robotics [ 46 , 57 , 50 , 49 ] . However, using existing end-to-end policy optimization approaches for this purpose often suffers performance collapse when finetuning in new domains [ 62 , 57 ] due to catastrophic forgetting of the priors learned during pretraining. This reflects the difficulty of exploration and credit assignment in low-data regimes, particularly when learning from raw perception. Recent off-policy RL methods attempt to improve sample efficiency by aggressively bootstrapping from limited datasets [ 2 , 7 , 17 , 36 ] , but face brittle trade-offs between data efficiency and training stability. We argue that this behavior stems from a fundamental limitation of end-to-end model-free policy finetuning: by tightly entangling task representations, rewards, and dynamics, these methods force algorithms to relearn the full task structure when transferred to new domains. In this work we ask: how can we retain physical priors and known task structure throughout finetuning? We argue that world models, rather than end-to-end policies, are the appropriate abstraction for transferring experience from simulation to the real world. Unlike policies, world models decompose decision making into state representations, reward and value functions, and environment-specific dynamics that are learned jointly but maintained as distinct components. This decomposition enables an approximate separation between global and local task structure. State encoders, rewards, and values capture global structure—such as which objects matter, their semantic roles, and distances to goal configurations—while the dynamics model captures local, action-dependent behavior specific to the environment. Our key insight is that this global structure is largely invariant across simulation and reality for dynamic, high-precision problems, whereas local dynamics are not. We exploit this asymmetry to focus real-world adaptation on dynamics while preserving global task structure learned in simulation. Concretely, we introduce Simulation Distillation ( SimDist ), a framework that bootstraps a world model in simulation and adapts in the real world through online planning and supervised finetuning of the latent dynamics model. At deployment, we freeze the state encoder, reward model, and value function learned in simulation, and adapt only the dynamics. The planner uses the frozen reward and value models to estimate long-horizon returns, enabling immediate behavioral improvement as dynamics predictions become more accurate, while avoiding difficult bootstrapping problems. By planning over the world model, the robot can reason counterfactually about trajectories it has not directly experienced, mitigating exploration challenges. Altogether, this sidesteps the instability of existing off-policy methods by reducing real-world adaptation to a simple, short-horizon supervised learning problem in latent space. Enabling this reliable and efficient adaptation requires systematically distilling the structure of the simulator into the world model during pretraining. In particular, the dynamics model must generalize from only a small number of real-world samples and remain accurate along the many sub-optimal trajectories sampled by the planner. To achieve this, we build on standard teacher–student distillation frameworks, leveraging privileged, state-based expert policies in simulation to generate high-quality behavior at scale. Importantly, rather than just collecting expert data, we deliberately perturb actions during data generation. This introduces failures and recovery behaviors, yielding broad state–action coverage that keeps representations, rewards, values, and dynamics in-distribution during real-world planning and finetuning. Relying on privileged experts for pretraining enables massive parallelism and scalable training, in contrast to computationally intensive online model-based RL methods [ 14 , 15 ] whose performance often saturates on challenging robotics problems. We summarize our contributions as follows: 1) We introduce SimDist , a world-model–based paradigm for sim-to-real transfer that reduces real-world adaptation to a simple supervised dynamics learning problem, avoiding long-horizon credit assignment and enabling reliable, off-policy, and sample-efficient adaptation. SimDist integrates seamlessly with existing student–teacher distillation. 2) We instantiate SimDist on two precise, contact-rich manipulation tasks and two quadruped locomotion tasks in the real world, achieving effective autonomous improvement with just 15–30 minutes of real-world data. 3) We show that informative reward and value models trained in simulation can transfer zero-shot to the real world from raw perception, highlighting a promising direction for leveraging privileged simulation supervision even beyond the settings studied in this work. II Related Work Real-World Reinforcement Learning. A growing body of work studies reinforcement learning on real-world robotic systems [ 36 , 33 , 57 , 47 , 50 , 51 , 32 , 24 , 13 , 29 ] . However, both model-free and model-based approaches remain challenging to apply reliably in low-data regimes and typically require sophisticated regularization. Model-free methods aggressively reuse off-policy data and rely on frequent critic updates [ 2 , 17 , 36 , 7 ] , often leading to value overestimation and unstable