Hero’s Journey : Testing Complex Rule Induction with Text Games
Anshun Asher Zheng, Kanishka Misra, David I. Beaver, Junyi Jessy Li · 2026
Read Hero’s Journey : Testing Complex Rule Induction with Text Games by Anshun Asher Zheng, Kanishka Misra, David I. Beaver, Junyi Jessy Li (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2606-02556.txt.
Introduction
Zheng and co-authors test whether language models can infer hidden rules and act on them. Their benchmark, HERO'S JOURNEY, uses text games. Agents must learn rules from demonstrations. They must then execute multi-step plans under those rules. Eight tasks span two induction families. Attribute and procedural induction are both covered.
Each task has four structural rule forms. Lexical grounding can be controlled. Identifiability conditions are set per task. Top models show real signs of rule induction. The skill is limited and uneven across tasks. Process execution adds a bottleneck of its own. Surface semantics barely matters by contrast. Steering methods aimed at induction help on attribute tasks. They give no reliable gain on procedural tasks. Procedural induction, the authors conclude, stays an open challenge.
Catalog record
| CREATOR | Anshun Asher Zheng, Kanishka Misra, David I. Beaver, Junyi Jessy Li |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 62 KB |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2606.02556
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Hero’s Journey : Testing Complex Rule Induction with Text Games Anshun Asher Zheng, Kanishka Misra, David I. Beaver, Junyi Jessy Li Abstract We introduce Hero’s Journey , a benchmark for rule induction in goal-directed episodic tasks, where agents must infer hidden rules from demonstrations and act on them through multi-step execution. Hero’s Journey 1 1 1 We release the code and the package under MIT license. covers eight tasks across attribute and procedural induction families, each with four structural rule forms, controllable lexical grounding, and identifiability conditions. Evaluating state-of-the-art LLMs, we find that models show evidence of rule induction, but the ability is limited and uneven across tasks. Meanwhile, process execution adds an execution bottleneck for models, whereas surface semantics has minimal effect. Induction-specific steering methods improve performance on attribute tasks but show no reliable gains on procedural tasks, suggesting the gap in procedural induction remains an open challenge. se counter=codeboxinput]codebox[2][] base=Listing \thetcbcounter : #2, listing only, #1 1 Introduction Figure 1: Overview of Hero’s Journey . 1 Rule interactions: four structural forms varying how entity attributes jointly determine the required item or process. 2 Attribute induction tasks: illustrated with A-Comp tasks (§ 3.2.1 ; each attribute independently governs a separate output dimension); entity class ( ranger, captain) and role ( prophet, chirurgeon). 3 Procedural induction tasks: illustrated with P-Comp tasks (§ 3.2.2 ; same structure): start, go_to_entity_location, buy_objects, perform_ritual, drink_potion, defeat_entity. 4 Identifiability: source demonstrations cover all combinations needed to recover the rule. 5 Agent loop: the agent reads demonstrations and acts in the environment. 6 Evaluations: ECSR measures task execution efficiency; RV measures whether the agent can verbalize the rule. 7 Results: ECSR and RV across all models; all underperform the human baseline. Somewhere beyond the dark forest, a captive waits to be freed. A warrior sets out, brave, but the only knowledge they carry is what others have left behind: scattered accounts of warriors who walked similar paths before, each facing a different guardian, each prevailing with a different weapon. The castle gates will not open until the guardian falls, and the guardian will not fall unless the warrior arrives with the right weapon. There is no oracle to consult; the armory is a day’s ride away and a wrong choice means starting over. The warrior must read the old accounts, find the pattern hiding in them, and commit. This simple scenario captures two challenges that, together, define the problem we study. The first is inductive : the rule connecting a guardian’s traits to the weapon that defeats it is never stated and must be recovered from past episodes. The second is procedural : acting on the rule requires executing a sequence of steps in the right order, where an incorrect inference forces the warrior to start again. From research agents that autonomously design and execute experiments ( Lu et al. 2026 ) , to coding agents that resolve real-world software issues ( Jimenez et al. 2024 ) , to web agents navigating complex multi-step tasks ( Zhou et al. 2024 ) , LLM agents must infer patterns from past examples and act on them. Despite rapid capability gains, LLMs continue to struggle with inductive reasoning ( Li et al. 2025 ) . For example, models in rule-learning settings frequently default to surface retrieval rather than structural induction ( He et al. 2025 ) . Existing benchmarks leave three aspects underexplored: (i) induction is not always situated in a multi-step execution setting where the inferred rule must drive a sequence of dependent actions (cf. Chollet 2019 ); (ii) evaluations typically test atomic mappings in isolation rather than more complex interaction between rules (cf. Rule 2020 ); and (iii) demonstrations may leave multiple rules consistent with the evidence, so it is unclear whether evaluation targets the specific rule under investigation (cf. Qiu et al. 2024 ; He et al. 2025 ). We refer interested readers to Chen et al. 2026 for a systematic review. To address these gaps, we introduce Hero’s Journey , an evaluation suite for rule induction in goal-directed episodic tasks. In each episode, the agent must infer the rules from demonstration episodes and apply the induced rule to novel test entities by executing a sequence of actions in the right order (Fig. 1 5 ). We instantiate Hero’s Journey with two families of tasks. Attribute induction tasks (Fig. 1 2 ; § 3.2.1 ) require inferring a hidden mapping from entity attributes to required items, with the mapping varying across four structural forms: additive, compositional, conditional, and override, each representing a different interaction between rules (Fig. 1 1 ). Procedural induction tasks (Fig. 1 3 ; § 3.2.2 ) require inferring hidden steps or ordering constraints within the goal procedure, with analogous structural variation. Each task comes with an identifiability condition (Fig. 1 4 ) that ensures the demonstration episodes are collectively sufficient to recover the target rule. Performance on Hero’s Journey is evaluated with two metrics for rule induction: ECSR (efficiency-calibrated success rate) captures whether models succeed efficiently (i.e., not via brute-force enumeration), and RV (rule verbalization) captures whether they can explicitly articulate the test rules. (Fig. 1 6 ). Across frontier proprietary and open models, humans outperform the strongest model on average by 13% on ECSR and 30% on RV, though the strongest models approach human ECSR on select attribute tasks (Fig. 1 7 ). While models are showing some reliance on rule induction to achieve tasks, the ability is still very limited and models may instead rely on contextual bias to succeed (§ 5 ). Notably, by embedding the tasks in a multi-step execution setting, surface semantics alone would not be effective. This extra complexity reveals further gaps in existing methodologies: steering methods yield only limited improvement and remain far below human-level performance (§ 6 ). 2 Background and Design Principles Inductive reasoning and its evaluation in LLMs. Inductive reasoning, deriving general rules from specific observations and applying them to novel instances, is a core capacity of intelligent agents ( Tenenbaum et al. 2011 ; Lake et al. 2015 ) . A related tradition in cognitive science has studied property induction : how agents generalize properties across entities sharing common category membership ( Osherson et al. 1990 ; Sloman 1993 ; Kemp and Jern 2014 ; Hayes and Heit 2018 ) ; language models have been also evaluated on tasks of this kind ( Misra et al. 2022 ; Bhatia 2024 ; Han et al. 2024 ; Padmanabhan et al. 2025 ) . More recently, it is found that LLMs in some settings perform explicit rule induction ( He et al. 2025 ; Lee et al. 2025 ) , though Li et al. 2025 reveal a systematic dissociation between the inductive and deductive stages: models often arrive at correct outputs without having abstracted the underlying rule. Existing benchmarks have probed abstract pattern induction across sequences, lists, and visual grids ( Chollet 2019 ; Rule 2020 ; Zhang et al. 2021 ) , and compositional generalization benchmarks have revealed systematic failures when applying learned rules to novel combinations ( Lake and Baroni 2018 ; Keysers et al. 2020 ; Kim and Linzen 2020 ; Ruis et al. 2020 ) . However, existing evaluation