AgentOdyssey: Open-Ended Long-Horizon Text Game Generation for Test-Time Continual Learning Agents
Zheyuan Zhang ∗ * , Zehao Wen ∗ * , Alvin Zhang, Andrew Wang, Jianwen Xie, Daniel Khashabi ♡ \heartsuit , Tianmin Shu ♡ \heartsuit Johns Hopkins University AgentOdyssey.github.io · 2026
Read AgentOdyssey: Open-Ended Long-Horizon Text Game Generation for Test-Time Continual Learning Agents by Zheyuan Zhang ∗ * , Zehao Wen ∗ * , Alvin Zhang, Andrew Wang, Jianwen Xie, Daniel Khashabi ♡ \heartsuit , Tianmin Shu ♡ \heartsuit Johns Hopkins University AgentOdyssey.github.io (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2606-24893.txt.
Introduction
Zheyuan Zhang and six colleagues introduce AgentOdyssey, a benchmark that generates endless text games to test agents that keep learning while deployed. Most evaluation assumes learning stops at test time. Continual learning agents instead must explore, gain world knowledge, remember episodes, and plan far ahead, all during use. AgentOdyssey procedurally builds open ended text games with rich entities and world dynamics. Its tasks stretch over long horizons.
Evaluation goes beyond game progress. Diagnostic tests probe world knowledge acquisition, episodic memory, exploration of objects and actions, action diversity, and model cost. Diverse agent designs were run in the generated games. Results expose sharp limits in the key abilities. Stronger base models scale performance up, yet even the best agent stays far below human play. Short term memory helps several agent designs at once. Zhang and colleagues map which factors lengthen an agent's meaningful horizon, and how much headroom remains.
Catalog record
| CREATOR | Zheyuan Zhang ∗ * , Zehao Wen ∗ * , Alvin Zhang, Andrew Wang, Jianwen Xie, Daniel Khashabi ♡ \heartsuit , Tianmin Shu ♡ \heartsuit Johns Hopkins University AgentOdyssey.github.io |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 83 KB |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2606.24893
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
AgentOdyssey: Open-Ended Long-Horizon Text Game Generation for Test-Time Continual Learning Agents Zheyuan Zhang ∗ * , Zehao Wen ∗ * , Alvin Zhang, Andrew Wang, Jianwen Xie, Daniel Khashabi ♡ \heartsuit , Tianmin Shu ♡ \heartsuit Johns Hopkins University AgentOdyssey.github.io Abstract † † ∗ Equal contribution. ♡ Equal advising. Abstract. For agents to learn continuously from interaction with the world at test time, they must be able to explore effectively, acquire new world knowledge and skills, retain relevant episodic experiences, and plan over long horizons. To evaluate these key abilities of test-time continual learning agents, we introduce AgentOdyssey , a novel evaluation framework that procedurally generates open-ended text games with rich entities, world dynamics, and long-horizon tasks. Critically, AgentOdyssey goes beyond the conventional machine learning assumption that learning does not occur at test time by placing agents in a continuous, long-horizon setting that interleaves learning and inference throughout deployment. We further propose a multifaceted evaluation methodology that measures not only game progress but also offers diagnostic tests on world knowledge acquisition, episodic memory, object and action exploration, action diversity, and model cost. We evaluate diverse agent paradigms in the generated games. Our experimental results reveal critical limits in agents’ key abilities, as well as factors that influence their meaningful horizon. Although performance scales with stronger base models, even the top agent remains far below human performance, leaving substantial headroom for improvement. Among agent mechanisms, we find that short-term memory benefits multiple agent paradigms and is an important component of agent test-time training. Figure 1: Example player trajectory in an AgentOdyssey -generated game. AgentOdyssey games test five key abilities for test-time continual learning agents: exploration for finding the workbench, episodic memory for recalling the dropped trade item, world knowledge acquisition for avoiding dangerous travel times, skill learning for improving crafting via written recipes, and long-horizon planning for completing multi-step objectives such as crafting a key for trade. The trajectory shows these abilities working together over hundreds of steps to complete the quest. 1 Introduction Table 1: Comparison of text game environments for agents. Columns report, from left to right: World (open or closed), # E ntities (types of entities; O = objects, A = areas, N = NPCs), # A ctions (number of actions per game), # G oals (number of narrative goals per game), Cont amination (whether evaluation content may appear in training data), Scal ability (ability to generate larger or varied game environments), Stoch astic (presence of random dynamics), Dyn amic World (whether the game environment evolves independently of agent actions), Hor izon (typical episode length in steps). We use the game environment in Voyager [ Wang et al. 2023a ] as a text-based Minecraft setting, where agents receive world observations in textual form and produce executable code actions. For AgentOdyssey , we include the \bm{\infty} as the theoretical capability of the game generator. Environment World # E # A # G Cont. Scal. Stoch. Dyn. Hor. ALFWorld [ Shridhar et al. 2020 ] Close O,A 11 1 Yes No No No 50 Jericho [ Hausknecht et al. 2020 ] Close O, A, N 187.2 1 Yes No Yes No 100 ByteSized32 [ Wang et al. 2023b ] Close O 9.8 1 No Yes No No 12.8 Minecraft (Text) [ Wang et al. 2023a ] Open O, A, N \sim 30 0 Yes No Yes Yes \bm{\infty} AgentOdyssey Open O, A, N \bm{\infty} \bm{\infty} No Yes Yes Yes \bm{\infty} Advances in large language models (LLMs) have demonstrated strong reasoning and problem-solving capabilities in challenging domains such as mathematics and programming [ Hendrycks et al. 2021 , Chen et al. 2021 , Wei et al. 2022 , Kojima et al. 2022 , Roziere et al. 2023 , Bai et al. 2023 , Lightman et al. 2023 , Guo et al. 2025 ] . Building on these successes, there have been many LLM-based agentic frameworks for sequential decision-making and embodied tasks, where LLM-based agents perceive, plan, and act in interactive environments [ Yao et al. 2022b , Shinn et al. 2023 , Huang et al. 2022 , Ahn et al. 2022 , Singh et al. 2023 , Reed et al. 2022 , Driess et al. 2023 , Mu et al. 2023 , Xiang et al. 2023 , Zhang et al. 2024 ] . However, existing benchmarks and training paradigms for such agents remain inadequate to study realistic test-time learning. First, traditional benchmarks implicitly assume that learning does not happen at test time, serving instead as purely inference-time evaluations of static performance measures [ Li et al. 2024 , Cheng et al. 2025 ] . Second, even when learning is considered, it is not a continuous process during evaluation; rather, agents are trained over many episodes before evaluation [ Shridhar et al. 2020 , Côté et al. 2018 , Wang et al. 2025c , Feng et al. 2025 ] . Together, these paradigms obscure a central challenge faced by real-world agents: after initial training, learning must occur continuously throughout their lifetimes. In this work, we study test-time continual learning agents that interact with the world, continuously acquire the necessary new knowledge and skills, and meaningfully improve during test-time deployment. This contrasts with traditional continual learning, which typically assumes a non-interactive setting and a clear boundary between training and testing [ McCloskey and Cohen 1989 , Kirkpatrick et al. 2017 , Lopez-Paz and Ranzato 2017 , Shin et al. 2017 ] . Test-time continual learning can occur either without parametric updates, in which retrieval-based conditioning and in-context mechanisms drive learning, or with parametric updates, in which agents learn through test-time training. Drawing inspiration from human cognitive development [ Gopnik and Meltzoff 1997 , Spelke and Kinzler 2007 ] , we hypothesize that successful test-time continual learning agents require five key abilities: exploration , episodic memory , world knowledge acquisition , skill learning , and long-horizon planning . These abilities reinforce one another and jointly shape performance in long-horizon, non-resettable environments. Exploration and planning help agents gather novel experiences, from which they acquire world knowledge and skills that unlock future experiences, forming a positive feedback loop. Memory is central to this loop: past experiences, knowledge, and skills are often crucial for future success, making forgetting or poor generalization costly. Thus, memory cannot be a passive log of observations and actions; it must support learning by abstracting episodic experience into semantic knowledge and retaining causal, decision-relevant information for future planning. To study the key abilities of test-time continual learning agents, we introduce AgentOdyssey , a novel evaluation framework that procedurally generates open-ended text games with rich, long-horizon environments. As shown in Figure 1 , the generated games feature: (1) novel and diverse world knowledge about entities, such as areas, objects, and NPCs, as well as world dynamics that agents must explore and acquire; (2) rich game mechanics that encourage skill learning, such as note-taking; and (3) challenging long-horizon tasks that require agents to decompose objectives into subgoals, manage goals over time, and use episodic memory for more effective planning. Compared to video games [ Chen et al. 2024 , Hu et al. 2025 ] , computer tasks [ Tan et al. 2024 , Xie et al. 2024 , Wang et al. 2025a ] , and