ASH: Agents that Self-Hone via Embodied Learning
Benjamin Schneider, Xavier Schneider, Victor Zhong, Sun Sun · 2026
Read ASH: Agents that Self-Hone via Embodied Learning by Benjamin Schneider, Xavier Schneider, Victor Zhong, Sun Sun (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-14211.txt.
Introduction
Schneider and three co-authors introduce ASH, an agentic system that self-hones in long-horizon worlds. Long-horizon visuomotor tasks challenge current methods, which lean on hand-engineered rewards or labeled demonstrations that do not scale. ASH instead learns a long-horizon policy from unlabeled internet video, with no reward shaping or expert annotation. In a self-improvement loop, ASH learns an Inverse Dynamics Model from its own trajectories when stuck. It then uses that model to extract supervision from relevant internet video, keeping key moments as memory.
Catalog record
| CREATOR | Benjamin Schneider, Xavier Schneider, Victor Zhong, Sun Sun |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 62403 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2605.14211
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
ASH: Agents that Self-Hone via Embodied Learning Benjamin Schneider, Xavier Schneider, Victor Zhong, Sun Sun Abstract Long-horizon embodied tasks remain a fundamental challenge in AI, as current methods rely on hand-engineered rewards or action-labeled demonstrations, neither of which scales. We introduce ASH , an agentic system that learns an embodied policy from unlabeled, noisy internet video, without reward shaping or expert annotation. ASH follows a self-improvement loop; when it gets stuck, ASH learns an Inverse Dynamics Model (IDM) from its own trajectories, and uses its IDM to extract supervision from relevant internet video. ASH uses unsupervised learning to identify key moments from large-scale internet video and retains them as long-term memory — allowing it to tackle long-horizon problems. We evaluate ASH on two complementary environments demanding multi-hour planning: Pokémon Emerald , a turn-based RPG, and The Legend of Zelda: The Minish Cap , a real-time action-adventure game. In both games, behavioral cloning, retrieval-augmented and zero-shot foundation-model baselines plateau, while ASH sustains progression across our 8-hour evaluation. ASH reaches an average of 11.2 / 12 11.2/12 milestones in Pokémon Emerald and 9.9 / 12 9.9/12 in Legend of Zelda, while the strongest baseline gets stuck in both environments at an average of 6.5 / 12 6.5/12 and 6.0 / 12 6.0/12 milestones, respectively. We demonstrate that self-improving agents are a scalable recipe for long-horizon embodied learning. 1 Introduction Developing agents capable of mastering long-horizon, open-ended tasks remains a fundamental challenge in artificial intelligence [ 18 , 17 ] . Unlike agents for short-horizon planning, long-horizon agents must navigate complex, open-ended worlds over millions of steps. This requires multi-hour planning, fine-grained motor control, and the ability to adapt to new visual contexts. Despite progress in embodied learning, current approaches struggle in these long-horizon settings. Behavioral Cloning (BC) methods [ 36 ] typically learn a policy from massive offline datasets. At this scale, assembling high-quality data is a significant challenge, requiring domain knowledge to curate data and extensive human annotation [ 3 ] . Moreover, much of the available data may be irrelevant to the agent’s current objectives, increasing both training compute and susceptibility to noise. Long-horizon tasks exacerbate these challenges [ 31 ] : a single trajectory may span hours, making it intractable to model dependencies across the full sequence. As a result, BC policies typically condition on short-term memory and struggle when a current decision depends on old information [ 23 ] . While Reinforcement Learning (RL) has demonstrated success in complex environments, it remains heavily dependent on dense reward engineering [ 28 , 15 , 3 ] . In a domain spanning tens of hours with diverse sub-tasks, manually designing a reward function for every intermediate milestone is brittle, domain-specific, and labor intensive. Like BC, modeling dependencies across hour-long sequences using RL is intractable. Instead, a common approach is to use short-term memory combined with “next sub-objective” conditioning [ 15 , 3 ] , which amounts to a human manually laying out a path for the agent to follow. These approaches have difficulty learning skills and knowledge beyond their human-engineered objectives, making them unsuitable for open-ended environments. Figure 1: ASH self-improves over the course of a multi-hour playthrough by retrieving and learning from relevant internet video. These skills compound, allowing ASH to achieve sustained progression. To address these limitations, we introduce ASH , an agentic system that follows a dynamic bootstrap loop. When ASH gets stuck, it searches for helpful demonstrations on the internet, and learns from them. ASH uses its current trajectories to retrieve relevant examples from online video and to update its Inverse Dynamics Model (IDM). ASH uses unsupervised learning to identify key moments from the retrieved video. ASH bootstraps its policy using these key moments as long-term memory and the retrieved video demonstrations labeled by the IDM. We evaluate ASH in two complementary long-horizon environments: Pokémon Emerald , a turn-based RPG combining strategic battling with open-ended exploration, and The Legend of Zelda: The Minish Cap , a real-time action-adventure game that requires puzzle solving and fine-grained motor control. Both require dozens of hours of decision-making across millions of distinct visual states, which makes them useful as long-horizon testbeds. ASH interacts with the games as a human does: by observing the screen and pressing buttons. In our evaluations, ASH progresses further than behavioral cloning [ 3 ] , retrieval-augmented [ 48 ] , and zero-shot foundation-model [ 29 ] baselines. Baselines plateau on intermediate milestones, while ASH continues to acquire new skills—battling trainers, catching Pokémon, exploring areas, and defeating enemies—across multi-hour horizons. After 8 hours, ASH achieves an average of 11.2/12 progression milestones in Pokémon Emerald and 9.9/12 milestones in Legend of Zelda. The strongest baseline gets stuck in both environments, achieving an average of 6.5 and 6.0 milestones, respectively. ASH’s main contributions are as follows: (1) We introduce dynamic bootstrapping for long-horizon embodied learning. (2) We propose long-term memory created from salient moments in large-scale internet video. (3) Through experimentation, we demonstrate that dynamic bootstrapping and long-term memory create a scalable recipe for embodied learning — capable of multi-hour planning, fine-grained motor control, and the ability to adapt to new visual contexts. 2 Related Work Internet-scale behavioral cloning. A large body of work trains embodied policies via BC on observation-only video at scale. VPT [ 3 ] creates pseudo-actions for thousands of hours of Minecraft video with an IDM trained on contractor-collected gameplay [ 36 ] , and remains the strongest reported Minecraft BC backbone [ 22 ] . A parallel imitation-from-observation literature recovers actions without an explicit IDM [ 37 , 11 , 32 , 5 ] . ASH departs from these approaches in two ways: it does not rely on a single statically curated training corpus or a fixed action-labeling model. Instead, ASH is an agentic system that uses the trajectories of its agents to update its IDM and retrieve data for policy bootstrapping. Retrieval-augmented decision-making. Retrieval has been used to give agents relevant context [ 21 , 43 , 33 ] , to condition policies on prior experience [ 19 , 13 ] , and to guide planning [ 47 ] . In these methods, retrieval selects a small number of relevant documents, trajectories, or examples to condition the agent during inference. ASH uses retrieval differently: it retrieves a task-relevant training set from a much larger internet-scale corpus, then updates its weights using that data. Self-improvement from model-generated data. A growing line of work bootstraps a learner from data produced by an earlier version of itself: STaR [ 45 ] and ReST [ 14 ] for reasoning, self-play for games [ 2 , 34 ] , and self-distillation in robotics [ 1 ] . ASH fits this template, but its improvement comes from retrieved internet video rather than from an external verifier or reward model, which are often unavailable in open-ended environments [ 12 ] . Long-horizon control in embodied agents. Long-horizon control requires agents to retain information that may become useful much later. However, full-history conditioning is computationally intractable, while short fixed-length memory is efficient but can lose information needed for planning [ 23 ] . Embodied BC and RL are dominated by bounded-memory policies, including recurrent policies that compress history into a