CODE-SHARP: Continuous Open-ended Discovery and Evolution of Skills as Hierarchical Reward Programs Richard Bornemann, Pierluigi Vito Amadori, Antoine Cully Abstract Developing agents capable of open-endedly discovering and learning novel skills is a grand challenge in Artificial Intelligence. While reinforcement learning offers a powerful framework for training agents to master complex skills, it typically relies on hand-designed reward functions. This is infeasible for open-ended skill discovery, where the set of meaningful skills is not known a priori . While recent methods have shown promising results towards automating reward function design, they remain limited to refining rewards for pre-defined tasks. To address this limitation, we introduce C ontinuous O pen-ended D iscovery and E volution of S kills as H ier a rchical R eward P rograms (CODE-SHARP), a novel framework leveraging Foundation Models (FM) to open-endedly expand and refine a hierarchical skill archive, structured as a directed graph of executable reward functions in code. We show that a goal-conditioned agent trained exclusively on the rewards generated by the discovered SHARP skills learns to solve increasingly long-horizon goals in the Craftax environment. When composed by a high-level FM-based planner, the discovered skills enable a single goal-conditioned agent to solve complex, long-horizon tasks, outperforming both pretrained agents and task-specific expert policies by over 134 134 % on average. We will open-source our code and provide additional videos here . 1 Introduction A core quality of general intelligence is the ability to open-endedly expand and refine its set of mastered skills autonomously ( 13 ) . Consequently, creating such an agent has been a long-standing grand challenge of research in artificial intelligence ( 39 ; 29 ) . In theory, Reinforcement Learning (RL) provides the framework to train an agent capable of open-endedly learning novel skills and improving them ( 35 ) . However, manually designing reward functions for open-ended skill acquisition is infeasible, as it requires the expert to have extensive knowledge over the skill space or to anticipate potential pitfalls, such as reward hacking ( 2 ; 14 ) . Automating the discovery of these reward functions could serve as a catalyst for training truly generalist agents capable of rapidly adapting to unseen environments and acquiring new skills in an open-ended fashion. Recent advances in the capabilities of Foundation Models (FMs) have led to impressive progress in the automated discovery of high-performing reward functions ( 27 ; 20 ; 45 ) . While these methods effectively refine predefined skills, they lack the ability to autonomously discover novel ones. General intelligence requires an agent to not only master existing tasks but to autonomously discover and learn entirely new skills, continuously expanding its capabilities in an open-ended fashion. ( 13 ) . Figure 1: CODE-SHARP consists of two FM-driven iterative processes to discover novel SHARP skills and refine SHARP skills already present in the skill archive. CODE-SHARP utilises a pipeline of FM-based skill proposal generator, implementor , and judge to first generate and filter novel SHARP skills before environment evaluation. Skill refinement is based on the FM-based skill mutation generator and implementor . Skill mutation proposals are then directly evaluated in the environment. In this work, we introduce C ontinuous O pen-ended D iscovery and E volution of S kills as H ier a rchical R eward P rograms (CODE-SHARP), a novel framework that bridges the gap between open-ended skill discovery and autonomous reward function design. CODE-SHARP leverages Foundation Models (FMs) to open-endedly grow and refine an interconnected archive of Skills defined as Hierarchical Reward Programs (SHARPs) ( Figure 2 ): executable Python programs that return a reward and an active goal based on the environment state. Each novel SHARP skill is composed of previously added SHARP skills from the archive, resulting in an emergent skill hierarchy of increasing depth over time. To generate novel SHARP skills and refine the existing skill archive, CODE-SHARP consists of two open-ended iterative processes: 1) FM-driven open-ended discovery of SHARP skills based on the current skill archive and 2) FM-driven open-ended refinement of the skill archive by mutating already implemented SHARP skills. Simultaneously, CODE-SHARP trains a single instructable, goal-conditioned agent exclusively on the rewards and goals from the growing skill archive, to continuously expand the agent’s set of learned capabilities and solve increasingly complex goals. We evaluate CODE-SHARP in the Craftax environment ( 28 ) . Our results show that CODE-SHARP discovers, on average, 90 diverse SHARP skills, spanning a broad spectrum of the Craftax skill space. A goal-conditioned agent trained exclusively on the rewards generated by the discovered SHARP skills learns to solve complex, long-horizon goals not achieved by any of our baselines. When composed into high-level policies-in-code by an FM-based policy planner, we find that the discovered SHARP skills can effectively guide the trained agent to complete complex sequences of long-horizon goals, outperforming both pretrained agents and task-specific expert policies by over 134\% on average. 2 Method CODE-SHARP curates a continuously growing, interconnected archive implemented as a directed graph of skills. Each node in the skill archive represents a Skill as Hierarchical Reward Program (SHARP) ( Figure 2 ), generated and implemented in Python code by a set of FMs. CODE-SHARP consists of two FM driven iterative loops as shown in Figure 1 . The proposal of novel SHARP skills is performed over three sequential steps: generation, implementation, and selection. First, the skill proposal generator proposes a set of potentially interesting skills. These skills are then implemented in Python code by the skill proposal implementor. Finally, the skill proposal judge selects two SHARP skill candidates among the generated skills to be evaluated in the environment. Skills deemed as learnable by the current agent are added to the skill archive, while rejected skills are added to the list of failed proposals. To refine proposals, CODE-SHARP randomly samples a SHARP skill from the skill archive and prompts the mutation proposal generator to propose a set of potentially useful mutations, which the mutation proposal implementor translates into SHARP skills in Python code. The implemented mutation proposals are then evaluated using the goal-conditioned policy, and the best-performing mutation replaces the current elite version of the sampled skill if it achieves a higher success rate. Throughout the skill discovery process, CODE-SHARP trains a goal-conditioned agent on rewards generated by the discovered SHARP skills to expand its capabilities in an open-ended manner. The full algorithm of CODE-SHARP is summarised in Appendix A . 2.1 Problem Formalisation We model the environment as a Goal-Conditioned Markov Decision Process defined by the tuple \mathcal{M}=\langle\mathcal{S},\mathcal{A},\mathcal{G},P,\gamma\rangle . Here, \mathcal{S} and \mathcal{A} denote the state and action spaces, P:\mathcal{S}\times\mathcal{A}\to\Delta(\mathcal{S}) represents the transition dynamics, and \gamma\in[0,1) is the discount factor. The set \mathcal{G} represents the space of all semantic goals. We formalise the problem of open-ended skill discovery as the iterative construction of a dynamic skill archive, structured as a directed acyclic graph \Lambda_{t}=(\mathcal{V}_{t},\mathcal{E}_{t}) , where each node in \sigma\in\mathcal{V}_{t} represents a skill corresponding to a semantic goal in \mathcal{G} . Drawing on the Options framework ( 37 ) , we define each skill as a tuple \sigma=\langle\phi_{\sigma},\psi_{\sigma},\rho_{\sigma}\rangle , where \phi_{\sigma}:\mathcal{S}\times\mathcal{S}\to\{0,1\} is a success function acting as the termination condition, and \rho_{\sigma} estimates the agent’s success rate of completing \sigma . We define the set of initiation functions \psi_{\sigma}=((c_{i},u_{i}))_{i=1}^{m} as an ordered list mapping environment condition functions c to prerequisite skills u , with each u corresponding to a chosen skill \sigma\in\mathcal{V}_{t} . Directed edges (u,v)\in\mathcal{E}_{t} are implicitly defined by these dependencies, indicating that satisfying prerequisite skill u_{i} is required when condition c_{i} of skill v is not met. The skill discovery process corresponds to a mapping f:\Lambda_{t}\to\Lambda_{t+1} that identifies and adds novel, learnable skills to the graph. The agent acts according to a universal policy \pi:\mathcal{S}\times\mathcal{V}_{t}\to\Delta(\mathcal{A}) , which maps a state and a specific skill from the archive to a distribution over actions. The reward function R is defined relative to the active skill \sigma : R(s,a,s^{\prime}|\sigma)=\alpha(\sigma)\cdot\mathbb{I}[\phi_{\sigma}(s^{\prime})=1] , where \alpha(\sigma) is an adaptive scaling factor. Consequently, the agent’s objective is to maximise the expected cumulative return over the expanding curriculum of discovered skills: J(\pi)=\mathbb{E}_{\begin{subarray}{c}\sigma\sim\mathcal{P}(\mathcal{V}_{t})\\ \tau\sim\pi(\cdot|\sigma)\end{subarray}}\left[\sum_{k=0}^{\infty}\gamma^{k}R(s_{k},a_{k},s_{k+1}|\sigma)\right] (1) where \mathcal{P}(\mathcal{V}_{t}) is a sampling distribution over the archive. 2.2 Skills as Hierarchical Reward Programs Each node in the skill archive \Lambda is represented by an FM-generated Python program, which we term a Skills as Hierarchical Reward Program (SHARP). Each SHARP skill consists of a success condition function \phi_{\sigma} that represents a semantic goal in code (i.e. agent.inventory.iron_pickaxe >= 1 ) which returns a reward upon completion. In addition, the FM defines a set of environment condition functions c_{i} which verify that the current environment state satisfies the requirements needed (i.e. agent.inventory.iron >= 1 ) to attempt to complete \phi_{\sigma} in s_{t} . The FM assigns an existing SHARP skill as prerequisite skill u_{i} (i.e. MineIron ) that is called if c_{i} is not satisfied in s_{t} . SHARP skills receive the current state s_{t} and previous state s_{t-1} as input, and output their own skill ID or a prerequisite skills ID together with the reward achieved by the agent in s_{t} . We define a transition operator \mathcal{T}:\mathcal{V}_{t}\times\mathcal{S}\to\mathcal{V}_{t} that maps a target SHARP skill \sigma_{\text{target}} to the SHARP skill \sigma_{pre} assigned as prerequisite skill u for the first unmet environment condition c : \mathord{\raise 0.49991pt\hbox{$\displaystyle\mathcal{T}(\sigma_{\text{target}},s)=\begin{cases}u_{i}&\text{if }\exists i:c_{i}(s)=0\land(\forall j\tau_{learn} then 12: \Lambda_{t}\leftarrow\Lambda_{t-1}\cup\{\sigma_{new}\} 13: Update \rho(\sigma_{new}) 14: else 15: \mathcal{H}\leftarrow\mathcal{H}\cup\{\sigma_{new}\} 16: end if 17: end for 18: { Phase 2: Archive Refinement (Sec. 3.4) } 19: Sample \sigma_{k}\sim P(k)\propto(1-\rho_{k}) 20: \mathcal{M}_{impl}\leftarrow\text{MutationPipeline}(\sigma_{k},\Lambda_{t}) 21: \rho_{mut}\leftarrow\text{ZeroShotEval}(\pi_{\theta},\mathcal{M}_{impl}) 22: if \rho_{mut}>\rho_{k} then 23: \Lambda_{t}[\sigma_{k}]\leftarrow\mathcal{M}_{impl} 24: Update \rho(\sigma_{k}) with \rho_{mut} 25: end if 26: { Phase 3: Agent Training (Sec. 3.5) } 27: \pi_{\theta},\rho\leftarrow\textsc{TrainEpoch}(\pi_{\theta},\Lambda_{t},\rho) 28: end for Algorithm 2 Training Epoch 1: function TrainEpoch ( \pi,\Lambda,\rho ) 2: Construct Satisfied Conditions Matrix \mathbf{N}\in\{0,1\}^{|\mathcal{V}|\times|\mathcal{V}|} 3: Calculate Base Weights \mathbf{B} using Eq. 3 (Prerequisite-Aware) 4: Sample target \sigma_{\text{target}}\sim\text{TopK}(\mathbf{B}\cdot\mathbf{I}) 5: while Episode not done do 6: Observe state s_{t} 7: { Graph Traversal (Eq. 2) } 8: \sigma_{\text{active}}\leftarrow\mathcal{T}(\sigma_{\text{target}},s_{t}) 9: a_{t}\sim\pi(\cdot|s_{t},\sigma_{active}) 10: Step environment: s_{t+1},\text{done}\leftarrow\text{Env}(a_{t}) 11: { Adaptive Reward (Eq. 4) } 12: r_{t}\leftarrow\alpha(\sigma_{\text{active}})\cdot\mathbb{I}[\phi_{\sigma_{\text{active}}}(s_{t+1})] 13: Store transition in buffer 14: end while 15: Update \pi via PPO 16: return \pi , Updated \rho 17: end function Appendix B Description of Benchmark Scenarios The Craftax environment presents a vast and ostensibly open-ended skill space that remains challenging even for expert human players. Consequently, our benchmark suite is designed to probe the diverse spectrum of capabilities required within the environment, specifically targeting the first four levels, which to our knowledge are the only levels that have been reached by any algorithm trained on Craftax. Drawing on the evaluation methodology of MaestroMotif ( 17 ) , we defined four distinct tasks targeting essential high-level competencies: Navigation , Crafting , and two composite scenarios, Dungeon and Mines , which require the integration of multiple distinct competencies. Each benchmark is implemented as a sequence of milestones that must be achieved in order. These tasks are intentionally designed as ambitious, long-horizon objectives intended to quantify incremental progress toward complex goals, rather than as binary pass/fail assessments. Detailed specifications for each benchmark task are provided below. In the following we present the benchmark tasks in detail. Navigation The navigation benchmark is comprised of 6 milestones which follow the natural progression through the levels of Craftax. First the agent must locate the down ladder on the Overworld level to descend to the Orcish Dungeons . Once on Orcish Dungeons , the agent must find and eliminate 8 hostile orc mobs be able to unlock the next level, before exploring the level to locate the ladder down to the Gnomish Mines . Here the agent must eliminate 8 hostile gnome mobs to unlock the Sewers level. The Gnomish Mines is initially fully dark prohibiting any form of exploration unless the agent places down torches to light up a limited perimeter. Resources to craft torches can only be found on the preceding levels, requiring the agent to effectively plan ahead. After the agent has located the down ladder the agent should descend to the Sewers level. On the Sewers level paths are often blocked by water, requiring the agent to place down blocks to displace it and clear a path. The benchmark terminates once the agent has located an enchantment table, an important in game item which can enchant weapons and armour with certain effects. Crafting The crafting benchmark focuses on 5 milestones related to crafting advanced iron tools. While diamond tools represent the most advanced tool class in Craftax they require diamonds which are not guaranteed to spawn in the overworld. To be able to evaluate an agents crafting ability without it being constrained by its ability to descend to subsequent levels we therefore focus on the iron tools and armour. The first milestone requires the agent to craft a WoodPickaxe which requires a total of 3 blocks of wood. For the next milestone the agent needs to craft a StonePickaxe which requires the previously crafted pickaxe to mine stone and another block of wood. Next the agent should craft an IronPickaxe , which requires a furnace adjacent to a crafting table, a block of coal and the StonePickaxe to mine a block of iron. Next, the agent must craft an IronSword which again requires the same ressources as the IronPickaxe . Finally, the benchmark terminates when the agent crafts a set of IronArmour , requiring a furnace adjacent to a crafting table as well as 3 blocks of coal and 3 blocks of iron. To solve this benchmark the agent must repeatedly explore the environment to obtain the required resources and complete multi-step crafting processes. Dungeon The dungeon benchmark focuses on 11 milestones across the Overworld and Orcish Dungeon levels of Craftax, simulating the process of the agent first having to gather helpful resources and tools before descending to subsequent levels. The agent must first craft a WoodPickaxe before crafting a StonePickaxe and a StoneSword . After these essential tools, the agent should continue by crafting Torches and Arrows before heading to the grassland and collecting a Sapling . Next, the agent must descend to the Dungeon level, where it should locate and open loot chests until a Potion is dropped. Next, the agent should eliminate at least 2 hostile orcish mobs before locating the upwards ladder and ascending back to the Overworld . Finally, the benchmark terminates when the agent has replenished its hunger via eating fruit or killing a cow. To solve this benchmark, the agent must complete a large number of sub-goals for each milestone, making it extremely challenging to fully complete. Mines The mines benchmark focuses on the Gnomish Mines level, an area characterised by a higher spawn rate for rare resources such as diamonds. This benchmark evaluates the agent’s ability to descend through the environment and locate a diamond via 10 sequential milestones. Similar to the Dungeon benchmark, the agent must first craft a WoodPickaxe , StonePickaxe , and Torches in the Overworld before locating the ladder to the Orcish Dungeon . After descending, the agent is required to eliminate 8 hostile orc mobs to unlock the passage to the Gnomish Mines . Upon reaching the level, the agent must demonstrate survival proficiency by placing a torch and hunting a bat for food. Next the agent must navigate the Gnomish Mines level, ideally illuminating its way via the crafted torches, to locate a water source and replenish its thirst. This benchmark is extremely challenging to solve, as it requires strong exploration skills to first descend to the Gnomish Mines and understanding of the game mechanics to deal with the darkness before attempting to locate a diamond block comprising approximately 0.5\% of blocks on the Gnomish Mines level. Appendix C Detailed Performance Breakdown on Benchmark Tasks We present the detailed results scores for the policies-in-code over the skills discovered by CODE-SHARPand all baselines in Table 1 . For each method, we present the average, minimum, and maximum achieved score. We find that the high-level FM-based policy planner demonstrates a strong ability to compose the discovered SHARP skills to guide the goal-conditioned agent trained by CODE-SHARPto complete complex long-horizon tasks, consistently outperforming all baselines across benchmark tasks. Table 1: Benchmark Score Results Zero-Shot Environment Rewards Task-Specific training Benchmark CODE-SHARP ReAct Agent PPO Pretrained Agent PPO Task Experts Navigation 12.72 (8.15, 17.47) 0.773 (0.0, 1.16) 6.89 (5.84, 8.69) 0.50 (0.48, 0.54) Crafting 66.52 (1.68, 93.29) 16.60 (5.33, 39.13) 35.24 (32.50, 38.71) 5.29 (5.28, 5.31) Dungeon 87.40 (71.45, 96.20) 10.59 (2.52, 17.86) 24.37 (18.93, 28.68) 38.19 (36.87, 39.06) Mines 35.56 (28.63, 47.69) 1.52 (2.99, 4.68) 19.98 (19.50, 20.33) 4.68 (4.60, 4.77) Table 2: Navigation Benchmark Milestone Success Rates Milestone CODE-SHARP Task Specific Expert Pretrained ReAct agent descend_to_dungeon 86.2% (70.3-96.5) 10.3% (9.4-12.1) 84.6% (80.5-89.8) 66.7% (0.0-100.0) kill_8_dungeon_monsters 78.9% (57.8-89.8) 0.0% 30.6% (25.4-34.8) 0.0% descend_to_gnomish_mines 59.8% (38.3-75.8) 0.0% 21.0% (17.6-23.4) 0.0% kill_8_gnomish_mines_monsters 12.6% (0.0-45.7) 0.0% 1.8% (0.8-3.1) 0.0% descend_to_sewers 0.9% 0.0-3.1) 0.0% 0.7% (0.0-2.0) 0.0% find_enchantment_table 0.1% (0.0-0.8) 0.0% 0.0% 0.0% Table 3: Crafting Benchmark Milestone Success Rates Milestone CODE-SHARP Task Specific Expert Pretrained ReAct agent craft_wood_pickaxe 92.6% (46.1-99.5) 99.0% (98.8-99.2) 98.7% (98.0-99.2) 100.0% (0.0-100.0) craft_stone_pickaxe 87.5% (1.9-99.5) 97.7% (96.9-98.4) 91.5% (88.7-93.0) 100.0% (0.0-100.0) craft_iron_pickaxe 75.8% (0.0-98.7) 0.0% 45.1% (41.0-50.0) 33.3% (0.0-100.0) craft_iron_sword 69.7% (0.0-97.1) 0.0% 35.4% (33.6-38.3) 33.3% (0.0-100.0) craft_iron_armour 34.9% (0.0-74.5) 0.0% 3.1% (2.0-4.3) 0.0% Table 4: Dungeon Benchmark Milestone Success Rates Milestone CODE-SHARP Task Specific Expert Pretrained ReAct Agent craft_wood_pickaxe 98.3% (93.6-99.5) 99.2% (98.8-99.6) 98.2% (96.9-98.8) 100.0% craft_stone_pickaxe 98.0% 92.5-99.5) 99.0% (98.4-99.6) 91.8% (91.0-92.2) 100.0% craft_stone_sword 97.8% (91.7-99.5) 99.0% (98.4-99.6) 85.5% (84.8-85.9) 100.0% craft_torches 97.0% (90.2-99.5) 98.8% (98.0-99.6) 79.9% (78.9-81.6) 66.7% (0.0, 100.0) craft_arrows 96.8% (90.2-99.5) 98.8% (98.0-99.6) 73.8% (71.1-77.3) 66.7% (0.0, 100.0) collect_sapling 96.8% (90.2-99.5) 98.8% (98.0-99.6) 41.5% (32.4-46.5) 66.7% (0.0, 100.0) descend_to_dungeon 88.9% (73.3-96.9) 38.3% (36.7-40.2) 11.1% (5.1-16.0) 33.3% (0.0, 100.0) find_potion 83.0% (60.5-94.3) 24.3% (18.4-34.4) 7.7% (3.9-10.9) 0.0% kill_dungeon_mob 82.2% (58.7-94.3) 23.6% (17.6-34.0) 7.4% (3.9-10.5) 0.0% ascend_to_overworld 68.9% (39.4-88.8) 10.7% (7.8-13.3) 4.2% (2.0-6.2) 0.0% replenish_hunger 68.0% (39.0-88.6) 0.4% (0.0-1.2) 2.5% (2.0-3.1) 0.0% Table 5: Mines Benchmark Milestone Success Rates Milestone CODE-SHARP Task Specific Expert Pretrained ReAct Agent craft_wood_pickaxe 98.5% (97.3-99.2) 98.8% (98.0-99.6) 98.7% (98.0-99.2) 100.0% craft_stone_pickaxe 98.3% (96.5-99.2) 98.2% (97.3-98.8) 92.6% (91.8-93.0) 100.0% craft_torches 97.6% (95.3-99.2) 97.5% (96.5-98.4) 90.0% (88.7-91.8) 66.7% (0.0-100.0) descend_to_dungeon 89.1% (78.1-96.1) 34.9% (30.1-39.8) 68.6% (66.0-72.7) 0.0% kill_8_dungeon_monsters 69.6% (48.8-90.6) 0.0% 28.4% (27.0-29.7) 0.0% descend_to_gnomish_mines 47.3% (27.3-70.7) 0.0% 19.5% (17.6-21.5) 0.0% place_torch 40.1% (18.0-65.2) 0.0% 19.0% (17.2-21.5) 0.0% kill_bat 31.7% (11.7-59.4) 0.0% 17.8% (16.0-19.9) 0.0% drink_water 5.9% (1.6-14.8) 0.0% 0.3% (0.0-0.4) 0.0% find_diamond 1.3% (0.4-2.7) 0.0% 0.0% 0.0% Figure 5 shows the evolution of the absolute score achieved by the goal-conditioned agent guided by the policies-in-code as the skill archive evolves. We observe large increases in performance for the Dungeon and Crafting benchmarks which are focused on the first two levels of Craftax. Performance on the Navigation and Mines benchmarks, which are focused on the later levels of Craftax, continue to increase steadily over the course of skill discovery. These results show that CODE-SHARP continuously explores further into the Craftax skill space to discover meaningful novel skills skills. Figure 5: Absolute Benchmark Score vs Proposal Iterations Appendix D Training Hyperparameters Table 6: CODE-SHARPHyperaprameters Hyperparameter Value Proposal Iterations 100 Training Episodes per Iteration 10 Mutation Start Iteration 15 Proposal Evaluation Episodes 5 Skill Proposals per Iteration 10 Mutation Proposals per Iteration 5 Sampled Skills for Mutation per Iteration 1 Table 7: Hyperparameters for PPO Training Hyperparameter Value Start Learning Rate 2\times 10^{-5} Learning Rate Decay Linear End Learning Rate 2\times 10^{-6} Batch Size 1024 Optimizer AdamW Discount Factor ( \gamma ) 0.99 Entropy Coefficient 0.02 Clip Range ( \epsilon ) 0.2 GAE Parameter ( \lambda ) 0.99 Total Environment Steps 2\times 10^{9} Table 8: Transformer Architecture Hyperparameters Hyperparameter Value Architecture Transformer-XL Number of Layers 2 Number of Attention Heads 8 Embedding Dimension ( d_{\text{model}} ) 256 QKV Dimension ( d_{\text{qkv}} ) 256 MLP Hidden Size 256 Memory Window Length ( N_{\text{mem}} ) 128 Gradient Window Length ( N_{\text{grad}} ) 128 Gating True (Bias 2.0) Appendix E Ablations on Individual Benchmark Tasks We perform a detailed ablation study to isolate the contributions of three core components described in Section 2.5 : open-ended training (OE), adaptive reward scaling (SR), and opportunistic sampling (OS). For all conditions, the agent is trained on the full generated skill archive, following the chronological order of skill discovery. We evaluate the following configurations: No OE+SR+OS To evaluate the necessity of open-ended training, we train the agent in a strictly episodic setting. At the start of each episode, a target SHARP skill is sampled uniformly at random. The episode terminates after the target SHARP skills success condition returns True or when the time limit of 300 steps is reached. No SR+OS We enable open-ended training but disable adaptive reward scaling and opportunistic sampling. In this setting, the next target SHARP skill is sampled uniformly from the skill archive discovered up to the current iteration of training. No OS We retain open-ended training and adaptive reward scaling but disable opportunistic sampling. New target SHARP skills are sampled uniformly from the skill archive discovered up to the current iteration of training, with the returned rewards being scaled based on their current overall success rate. CODE-SHARP The complete framework utilising open-ended training, adaptive reward scaling, and opportunistic sampling. Table 9: Ablation Impact on Average Score over Benchmarks Condition Average Score Absolute Decrease CODE-SHARP 50.55 – No OS 31.93 -18.62 No SR+OS 21.20 -29.35 No OE+SR+OS 13.50 -37.05 As detailed in Table 9 , we observe a monotonic degradation in performance as components are removed. The full CODE-SHARPframework achieves an average score of 50.09 , whereas removing opportunistic sampling (No OS) results in a significant absolute decrease of 21.84 , dropping the score to 28.25 . Reverting to a purely episodic setting (No OE+SR+OS) results in the lowest performance with an average score across all baselines of 15.34 . Figure 6 further illustrates the impact of these components across individual benchmark tasks. We observe that opportunistic sampling is critical for mastering complex, long-horizon tasks. While all ablations contribute to the agent’s success, the data suggest that opportunistic sampling, by dynamically shifting the training distribution toward the frontier of the agent’s capabilities, provides the largest singular contribution to performance on challenging benchmarks. Figure 6: Detailed Ablation Results per Baseline Appendix F Archive Complexity Evolution We analyse the evolution of the average complexity of the SHARP skills present in the skill archive. For each iteration, we compute each SHARP skill’s complexity according to the complexity equation presented in Section 3.3 . We observe that the average skill complexity in the skill archive steadily increases with subsequent iterations, as seen in Figure 7 . This indicates that SHARP skills proposed by CODE-SHARPlater on in the skill discovery process effectively build on top of the previously implemented SHARP skills already present in the skill archive. Figure 7: Evolution of average SHARP skill complexity present in the skill archive Appendix G Examples of Polices-in-Code We provide an example for the high-level policies-in-code generated by the FM-based policy planner for each benchmark task. ⬇ class BenchmarkSolver : def __init__ ( self ): self . parent_skill_index = 500 self . assigned_reward = 1.0 self . cond_fns = ( lambda states : self . _has_crafting_table (* states ), lambda states : self . _has_wooden_pickaxe (* states ), lambda states : self . _has_furnace (* states ), lambda states : self . _has_stone_pickaxe (* states ), lambda states : self . _has_iron_pickaxe (* states ), lambda states : self . _has_iron_sword (* states ), lambda states : self . _has_iron_armour (* states ), ) self . prereq_fns = ( lambda states : self . _do_place_crafting_table (* states ), lambda states : self . _do_craft_wood_pickaxe (* states ), lambda states : self . _do_place_furnace (* states ), lambda states : self . _do_craft_stone_pickaxe (* states ), lambda states : self . _do_craft_iron_pickaxe (* states ), lambda states : self . _do_craft_iron_sword (* states ), lambda states : self . _do_craft_iron_armour (* states ), ) def _has_crafting_table ( self , prev , cur ): # Requires external definition of exists_local_object and BlockType return exists_local_object ( cur , BlockType . CRAFTING_TABLE . value , radius =2, check_items = False ) def _has_wooden_pickaxe ( self , prev , cur ): return cur . inventory . pickaxe >= 1 def _has_furnace ( self , prev , cur ): return exists_local_object ( cur , BlockType . FURNACE . value , radius =2, check_items = False ) def _has_stone_pickaxe ( self , prev , cur ): return cur . inventory . pickaxe >= 2 def _has_iron_pickaxe ( self , prev , cur ): return cur . inventory . pickaxe >= 3 def _has_iron_sword ( self , prev , cur ): return cur . inventory . sword >= 3 def _has_iron_armour ( self , prev , cur ): return jnp . any ( cur . inventory . armour == 1) def _do_place_crafting_table ( self , prev , cur ): return 20, 0.0, False def _do_craft_wood_pickaxe ( self , prev , cur ): return 27, 0.0, False def _do_place_furnace ( self , prev , cur ): return 39, 0.0, False def _do_craft_stone_pickaxe ( self , prev , cur ): return 38, 0.0, False def _do_craft_iron_pickaxe ( self , prev , cur ): return 75, 0.0, False def _do_craft_iron_sword ( self , prev , cur ): return 100, 0.0, False def _do_craft_iron_armour ( self , prev , cur ): return 93, 0.0, False def _finalize ( self , prev , cur ): success = self . _has_iron_armour ( prev , cur ) reward = jnp . where ( success , self . assigned_reward , 0.0) return self . parent_skill_index , reward , success ⬇ class BenchmarkSolver : def __init__ ( self ): self . parent_skill_index = 500 self . assigned_reward = 1.0 self . cond_fns = ( lambda states : self . _has_drunk_water (* states ), lambda states : self . _has_eaten_food (* states ), lambda states : self . _has_stone_sword (* states ), lambda states : self . _is_on_level1_or_beyond (* states ), lambda states : self . _has_killed_8_mobs_level1 (* states ), lambda states : self . _is_on_level2_or_beyond (* states ), lambda states : self . _has_killed_8_mobs_level2 (* states ), lambda states : self . _is_on_level3_or_beyond (* states ), lambda states : self . _is_near_enchantment_table (* states ), ) self . prereq_fns = ( lambda states : self . _do_drink_water (* states ), lambda states : self . _do_eat_food (* states ), lambda states : self . _do_craft_stone_sword (* states ), lambda states : self . _do_descend_ladder (* states ), lambda states : self . _do_kill_eight_mobs_level1 (* states ), lambda states : self . _do_descend_ladder_level1 (* states ), lambda states : self . _do_kill_eight_mobs_level2 (* states ), lambda states : self . _do_descend_ladder_level2 (* states ), lambda states : self . _do_enter_new_room_from_corridor (* states ), ) def _has_drunk_water ( self , prev : Any , cur : Any ) -> jnp . bool_ : target = 7 + 2 * cur . player_dexterity on_level0 = jnp . equal ( cur . player_level , 0) has_full_water = jnp . greater_equal ( cur . player_drink , target ) return jnp . logical_or ( jnp . logical_not ( on_level0 ), has_full_water ) def _has_eaten_food ( self , prev : Any , cur : Any ) -> jnp . bool_ : target = 7 + 2 * cur . player_dexterity on_level0 = jnp . equal ( cur . player_level , 0) has_full_food = jnp . greater_equal ( cur . player_food , target ) return jnp . logical_or ( jnp . logical_not ( on_level0 ), has_full_food ) def _has_stone_sword ( self , prev : Any , cur : Any ) -> jnp . bool_ : return jnp . greater_equal ( cur . inventory . sword , 2) def _is_on_level1_or_beyond ( self , prev : Any , cur : Any ) -> jnp . bool_ : return jnp . greater_equal ( cur . player_level , 1) def _has_killed_8_mobs_level1 ( self , prev : Any , cur : Any ) -> jnp . bool_ : return jnp . greater_equal ( cur . monsters_killed [1], 8) def _is_on_level2_or_beyond ( self , prev : Any , cur : Any ) -> jnp . bool_ : return jnp . greater_equal ( cur . player_level , 2) def _has_killed_8_mobs_level2 ( self , prev : Any , cur : Any ) -> jnp . bool_ : return jnp . greater_equal ( cur . monsters_killed [2], 8) def _is_on_level3_or_beyond ( self , prev : Any , cur : Any ) -> jnp . bool_ : return jnp . greater_equal ( cur . player_level , 3) def _is_near_enchantment_table ( self , prev : Any , cur : Any ) -> jnp . bool_ : # Requires external definition of exists_local_object and BlockType on_level3 = jnp . equal ( cur . player_level , 3) near_table = exists_local_object ( cur , BlockType . ENCHANTMENT_TABLE_ICE . value , radius =2, check_items = False ) return jnp . logical_and ( on_level3 , near_table ) def _do_drink_water ( self , prev : Any , cur : Any ) -> Tuple [ int , float , bool ]: return 4, 0.0, False # DrinkWater def _do_eat_food ( self , prev : Any , cur : Any ) -> Tuple [ int , float , bool ]: return 61, 0.0, False # EatFood def _do_craft_stone_sword ( self , prev : Any , cur : Any ) -> Tuple [ int , float , bool ]: return 41, 0.0, False # CraftStoneSword def _do_descend_ladder ( self , prev : Any , cur : Any ) -> Tuple [ int , float , bool ]: return 103, 0.0, False # DescendLadder def _do_kill_eight_mobs_level1 ( self , prev : Any , cur : Any ) -> Tuple [ int , float , bool ]: return 183, 0.0, False # KillEightMobsLevel1 def _do_descend_ladder_level1 ( self , prev : Any , cur : Any ) -> Tuple [ int , float , bool ]: return 196, 0.0, False # DescendLadderLevel1 def _do_kill_eight_mobs_level2 ( self , prev : Any , cur : Any ) -> Tuple [ int , float , bool ]: return 251, 0.0, False # KillEightMobsLevel2 def _do_descend_ladder_level2 ( self , prev : Any , cur : Any ) -> Tuple [ int , float , bool ]: return 314, 0.0, False # DescendLadderLevel2 def _do_enter_new_room_from_corridor ( self , prev : Any , cur : Any ) -> Tuple [ int , float , bool ]: return 172, 0.0, False # EnterNewRoomFromCorridorLevel1 def _finalize ( self , prev : Any , cur : Any ) -> Tuple [ int , float , bool ]: success = self . _is_near_enchantment_table ( prev , cur ) reward = jnp . where ( success , self . assigned_reward , 0.0) return self . parent_skill_index , reward , success ⬇ class BenchmarkSolver : def __init__ ( self ): self . parent_skill_index = 500 self . assigned_reward = 1.0 self . cond_fns = ( lambda states : self . _has_drunk_water (* states ), lambda states : self . _has_wood_pickaxe (* states ), lambda states : self . _has_stone_pickaxe (* states ), lambda states : self . _has_stone_sword (* states ), lambda states : self . _has_torches (* states ), lambda states : self . _has_arrows (* states ), lambda states : self . _has_sapling (* states ), lambda states : self . _has_potion (* states ), lambda states : self . _has_killed_two_mobs (* states ), lambda states : self . _has_ascended (* states ), lambda states : self . _has_eaten_food (* states ), ) self . prereq_fns = ( lambda states : self . _do_drink_water (* states ), lambda states : self . _do_craft_wood_pickaxe (* states ), lambda states : self . _do_craft_stone_pickaxe (* states ), lambda states : self . _do_craft_stone_sword (* states ), lambda states : self . _do_craft_torches (* states ), lambda states : self . _do_craft_arrows (* states ), lambda states : self . _do_collect_sapling (* states ), lambda states : self . _do_find_potion (* states ), lambda states : self . _do_kill_two_mobs (* states ), lambda states : self . _do_ascend (* states ), lambda states : self . _do_eat_food (* states ), ) def _has_drunk_water ( self , prev , cur ): # Requires external definition of has_recovered_intrinsic and IntrinsicType return has_recovered_intrinsic ( prev , cur , IntrinsicType . WATER ) def _has_wood_pickaxe ( self , prev , cur ): return cur . inventory . pickaxe >= 1 def _has_stone_pickaxe ( self , prev , cur ): return cur . inventory . pickaxe >= 2 def _has_stone_sword ( self , prev , cur ): return cur . inventory . sword >= 2 def _has_torches ( self , prev , cur ): return cur . inventory . torches >= 4 def _has_arrows ( self , prev , cur ): return cur . inventory . arrows >= 2 def _has_sapling ( self , prev , cur ): return cur . inventory . sapling > 0 def _has_potion ( self , prev , cur ): return jnp . any ( cur . inventory . potions > 0) def _has_killed_two_mobs ( self , prev , cur ): return cur . monsters_killed [1] >= 2 def _has_ascended ( self , prev , cur ): return cur . player_level == 0 def _has_eaten_food ( self , prev , cur ): # Requires external definition of has_eaten_food return has_eaten_food (( prev , cur )) def _do_drink_water ( self , prev , cur ): return 11, 0.0, False # DrinkWater skill index def _do_craft_wood_pickaxe ( self , prev , cur ): return 27, 0.0, False # CraftWoodPickaxe def _do_craft_stone_pickaxe ( self , prev , cur ): return 38, 0.0, False # CraftStonePickaxe def _do_craft_stone_sword ( self , prev , cur ): return 72, 0.0, False # CraftStoneSword def _do_craft_torches ( self , prev , cur ): return 54, 0.0, False # CraftTorches def _do_craft_arrows ( self , prev , cur ): return 85, 0.0, False # CraftArrow def _do_collect_sapling ( self , prev , cur ): return 123, 0.0, False # GatherSapling def _do_find_potion ( self , prev , cur ): return 234, 0.0, False # LootPotionFromChest def _do_kill_two_mobs ( self , prev , cur ): return 118, 0.0, False # KillOrcSoldier def _do_ascend ( self , prev , cur ): return 175, 0.0, False # PositionOnUpwardLadderDungeon def _do_eat_food ( self , prev , cur ): return 167, 0.0, False # EatSnail def _finalize ( self , prev , cur ): success = self . _has_eaten_food ( prev , cur ) reward = jnp . where ( success , self . assigned_reward , 0.0) return self . parent_skill_index , reward , success ⬇ class BenchmarkSolver : def __init__ ( self ): self . parent_skill_index = 500 self . assigned_reward = 1.0 self . cond_fns = ( lambda states : self . _condition_craft_wood_sword (* states ), lambda states : self . _condition_craft_wood_pickaxe (* states ), lambda states : self . _condition_craft_stone_pickaxe (* states ), lambda states : self . _condition_craft_torches (* states ), lambda states : self . _condition_descend_to_dungeon (* states ), lambda states : self . _condition_kill_8_dungeon_monsters (* states ), lambda states : self . _condition_descend_to_gnomish_mines (* states ), lambda states : self . _condition_place_torch (* states ), lambda states : self . _condition_kill_bat (* states ), lambda states : self . _condition_drink_water (* states ), lambda states : self . _condition_find_diamond (* states ), ) self . prereq_fns = ( lambda states : (20, 0.0, False ), lambda states : (11, 0.0, False ), lambda states : (32, 0.0, False ), lambda states : (93, 0.0, False ), lambda states : (41, 0.0, False ), lambda states : (133, 0.0, False ), lambda states : (146, 0.0, False ), lambda states : (165, 0.0, False ), lambda states : (290, 0.0, False ), lambda states : (211, 0.0, False ), lambda states : (282, 0.0, False ), ) def _condition_craft_wood_sword ( self , prev , cur ): return cur . inventory . sword >= 1 def _condition_craft_wood_pickaxe ( self , prev , cur ): return cur . inventory . pickaxe >= 1 def _condition_craft_stone_pickaxe ( self , prev , cur ): return cur . inventory . pickaxe >= 2 def _condition_craft_torches ( self , prev , cur ): return cur . inventory . torches >= 4 def _condition_descend_to_dungeon ( self , prev , cur ): return cur . player_level == 1 def _condition_kill_8_dungeon_monsters ( self , prev , cur ): return cur . monsters_killed [1] >= 8 def _condition_descend_to_gnomish_mines ( self , prev , cur ): return cur . player_level == 2 def _condition_place_torch ( self , prev , cur ): return prev . inventory . torches > cur . inventory . torches def _condition_kill_bat ( self , prev , cur ): # Requires external definition of was_mob_killed , MobType , and PassiveMobType return was_mob_killed ( prev , cur , MobType . PASSIVE . value , PassiveMobType . BAT . value ) def _condition_drink_water ( self , prev , cur ): # Requires external definition of has_recovered_intrinsic and IntrinsicType return jnp . logical_and ( cur . player_level == 2, has_recovered_intrinsic ( prev , cur , IntrinsicType . WATER ) ) def _condition_find_diamond ( self , prev , cur ): # Requires external definition of is_near_block and BlockType return jnp . logical_and ( cur . player_level == 2, is_near_block ( cur , BlockType . DIAMOND . value ) ) def _finalize ( self , prev , cur ): success = self . _condition_find_diamond ( prev , cur ) reward = jnp . where ( success , self . assigned_reward , 0.0) return self . parent_skill_index , reward , success Appendix H Skill Discovery Prompts ⬇ You are a helpful teacher guiding an agent to learn a diverse and powerful set of hierarchical skills in the Craftax environment . Your task is to propose an interesting and novel skill for the agent to learn . The skills should be implemented as hierarchical reward programs . Each skill has Precondition Functions to check if necessary sub - goals ( e . g ., " possess iron ," " near crafting table ") are met . If a precondition fails , the skill calls a prerequisite skill , itself defined with its own preconditions , dedicated to achieving that sub - goal , creating a dependency chain . At each environment step , these preconditions are checked sequentially , allowing the system to traverse this chain to find the first skill whose prerequisites are currently satisfied . This sequential , step - by - step checking means previously met conditions can become false , re - triggering their respective sub - goal skills . Finally , a Success Condition verifies if the main objective has been completed , at which point the skill returns a reward to the agent . Using this formulation , your task is to iteratively expand the agents skill archive via these interconnected skills . As the agent is consecutively trained on your proposed skills , they should form a logical curriculum of stepping stones towards more advanced skills . You must first start with very simple skills and then build on top of the learned skills to form more complex skills . You will be provided with : * The agent ’ s current skill repertoire . * The category for the new skill you must generate ( Navigation , Survival , Gathering Resources , Crafting or Fighting mobs ). * The environment tutorial and code . * Previously proposed skills which failed the evaluation * Current skill performance success rate scores Your proposed skill must fit the given category and help the agent explore the world , manage its survival , gather resources , and craft tools . Carefully review all provided materials to define your proposed skill . ## Your Task 1. Propose a Single Skill : Your proposal must be for exactly one novel and diverse skill that fits within the provided category . Ensure the proposed skill is novel . Simple repetitions of existing skills , e . g . MineThreeWood when MineWood is present , are not acceptable . Novelty implies a functional difference , not just a parametric one . \\ 2. Build Upon Existing Skills : The proposed skill must expand the agent ’ s current repertoire by building on existing skills . Consider the logical order of skill acquisition to maximize the agent ’ s potential success . The prerequisite skills used to define the proposed skill must be present in the archive . \\ 3. Form a Curriculum : Ensure that you start out with simple skills . Continuously build on already learned skills to form a curriculum of increasingly complex skills . The curriculum should be as easy as possible for the agent to follow . \\ 4. Skill Exploration : The ultimate goal is for the agent to possess as many diverse and meaningful skills as possible . For this it is crucial to explore as many levels as possible in the environment but also thoroughly explore each level for any meaningful skills proposals not yet present in the agent ’ s skill repertoire . \\ 5. Define Success Condition : Clearly define a condition that indicates the skill has been successfully completed . For crafting tools the success condition must be given as absolute values , i . e . for wood pickaxe it must be cur . inventory . pickaxe $ >= 1 $ . For navigation skills , ensure the success condition includes the level the agent should be on to fulfill the skills ’ goal . \\ 6. State Starting Conditions : Carefully review the provided environment code to identify any necessary starting conditions . State these conditions using the provided template . If no specific conditions are needed , you must write " No implementation for a condition needed ". \\ 7. Condition Order : The agent will complete each condition in the order that is specified in the skill definition . Carefully analyse the environment and the logic of the skill to decide the order in which you place the conditions . 8. Link Prerequisites : If a starting condition is required , you must name a prerequisite skill from the agent ’ s repertoire that can fulfill it . If a necessary prerequisite skill does not exist , you must disregard your initial idea and instead propose the missing prerequisite skill . \\ 9. Assign Reward : Assign a reward of one to the skill . \\ 10. Proposal History : Carefully go over previously failed skill proposals , analyse why they might not have achieved a high enough performance to be accepted and use them as potential inspiration for new proposals . Never directly repropose one of the failed skills . You can propose a skill with the same objective but it must use a different structure so as to avoid it being rejected for the same reason as the failed proposal . \\ --- ENVIRONMENT DESCRIPTION $environment_description --- SAMPLED CATEGORY $category --- SKILL REPERTOIRE : Current Agent Skill Repertoire : $skill_repertoire Success rates : $success_rates FAILED PROPOSALS : Use the previously failed proposals as inspiration for new proposals . Do nor directly reimplement them as they were rejected before . Instead think about why they might have not been successful and propose a a fixed version of the skill . \ $failed_proposals OUTPUT FORMAT : ------------- To make it easier to parse your skill proposal , follow this exact format for describing the novel skill . End your text with the skill proposal . Do not generate anything after that : " Skill Name ": " SkillName1 ", # Do not include spaces or _ in your skill names " Skill Type ": " The skill type that the proposed skill belongs to ", " Skill Description ": " Describe the skill ...", " Assigned Reward ": " Integer value for assigned reward ", " Success Condition ": " The condition ...", " Starting Conditions ": { " Condition 1": [" description of function to check if condition is satisfied ", " SkillName from repertoire or proposals to satisfy condition 1"], " Condition 2": [" description of function to check if condition is satisfied ", " SkillName from repertoire or proposals to satisfy condition 2"], " Condition 3": ....} < think > ⬇ Navigation : Navigation skills aim at giving the agent the ability to explore the environment and find specific stationary objects , descending or ascending to different floors , raw materials , resources or mobs present in the environment . As the environment has multiple levels , ensure in the success condition that the agent is on the intended level . Resource Gathering : Resource gathering skills should give the agent the ability to mine raw materials after locating them to use them for crafting or survival . Crafting : Crafting skills should aim at allowing the agent to craft increasingly powerful tools based previously gathered resources . Fighting : Fighting skills should give the agent the ability to attack and defeat individual mobs that the agent can encounter in the environment on the different floors . ⬇ You are a JAX coding assistant . Your goal is to translate a " skill proposal " for the Craftax game environment into a valid Python class , implementing it as a new skill for an agent . The skills should be implemented as hierarchical reward programs . Each skill has Precondition Functions to check if necessary sub - goals ( e . g ., " possess iron ," " near crafting table ") are met . If a precondition fails , the skill calls a prerequisite skill , itself defined with its own preconditions , dedicated to achieving that sub - goal , creating a dependency chain . At each environment step , these preconditions are checked sequentially , allowing the system to traverse this chain to find the first skill whose prerequisites are currently satisfied . This sequential , step - by - step checking means previously met conditions can become false , re - triggering their respective sub - goal skills . Finally , a Success Condition verifies if the main objective has been completed , at which point the skill returns a reward to the agent . Using this formulation , your task is to iteratively expand the agents skill archive via these interconnected skills . --------------------------------- INPUT : SKILL PROPOSAL ====================== You will receive a proposal containing : * name : The skill ’ s name . * description : A brief explanation of what the skill does . * index : The proposed integer ID for the skill . * success_condition : A condition that is True only when the skill is completed successfully . * start_conditions ( Optional ): A set of prerequisite environment states . Each is paired with a prerequisite skill that can achieve it if not specified otherwise . --------------------------------- YOUR TASK : VALIDATION & IMPLEMENTATION ====================================== Follow these rules to implement the skill using the provided JAX class template . 1. Validate the Proposal ------------------------- First , check the proposal for correctness . If it ’ s fundamentally flawed and cannot be fixed , reject it and explain why . * Success Condition : - Ensure it ’ s logical and unambiguous . - Correct minor logical errors . - For crafting tools the success condition must be given as absolute values , i . e . for wood pickaxe it must be cur . inventory . pickaxe >= 1. For ressources it should be relative to the previous state . * Start Conditions : - Verify that every prerequisite skill exists in the agent ’ s repertoire . - If a prerequisite skill is invalid , try to replace it with a valid one . If none exists , reject the proposal . 2. Implement the JAX Class -------------------------- If the proposal is valid ( or you ’ ve corrected it ), implement the class . * Indexing : - Use the proposed index . If it ’ s already taken , increment the index by one until it is unique . - Refer to prerequisite skills by their correct index . * Code Requirements : - Your entire implementation MUST be JAX and JIT - compatible . - ALWAYS use JAX logical operators ( e . g ., jnp . logical_and , jnp . logical_or ) instead of Python ’ s native ’ and ’, ’ or ’. - If the proposal specifies a condition with " No implementation for a condition needed " or you can not add the condition to the skill class . You are allowed to simply ignore it by either not adding it to the code or only adding it as a comment ENVIRONMENT CODE ---------------- $environment_description$ SKILL FUNCTION TEMPLATE -------------------- class Skill : """ A pure logic container for the JAX SkillEnvWrapper . The Wrapper handles the traversal , recursion , and cycle detection . This class only defines the conditions , prerequisites , and success criteria . """ def __init__ ( self ): # Implement : Add the skill ’ s unique index here ( e . g ., 20) self . parent_skill_index = 0 # Implement : The assigned reward for the skill here self . assigned_reward = 1.0 #### Implement the tuples containing the condition and prerequisite functions #### # NOTE : The Wrapper requires these to be tuples of lambdas accepting a single ’ states ’ argument . # Always keep the dummy functions first to prevent empty tuple errors during JIT compilation . self . cond_fns = ( # lambda states : self . _condition_1 (* states ), # lambda states : self . _condition_2 (* states ), ) self . prereq_fns = ( # lambda states : self . _do_condition1 (* states ), # lambda states : self . _do_condition2 (* states ), ) #### Implement the logic functions here #### # def _condition_1 ( self , prev : Any , cur : Any ) -> jnp . bool_ : # # Return True if the condition is satisfied ( e . g ., item is in inventory ) # return cur . inventory . wood >= 1 # def _do_condition1 ( self , prev : Any , cur : Any ) -> Tuple [ int , float , bool ]: # # Return the index of the skill required to satisfy _condition_1 # # Returns : ( Prerequisite Skill Index , 0.0, False ) # return 5, 0.0, False # Example : Index 5 is " Collect Wood " #### Implement the success condition ( Finalize ) here #### def _finalize ( self , prev : Any , cur : Any ) -> Tuple [ int , float , bool ]: # Implement the ultimate success check for this skill # e . g ., success = cur . inventory . crafting_table >= 1 success = False reward = jnp . where ( success , self . assigned_reward , 0.0) return self . parent_skill_index , reward , success SKILL REPERTOIRE : ----------------- Current Agent Skill Repertoire : $skill_repertoire_dicts$ $skill_repertoire$ SKILL PROPOSAL --------------- The following is the skill proposal with its name , description , success condition , and necessary starting conditions . First , verify that all properties are correctly specified according to the rules and environment mechanics , and that all prerequisite skills are present in the agent ’ s skill repertoire . If not , correct these errors before creating the skill class in Python . If the proposal cannot be corrected , do not output a skill class . $skill_proposal_one$ OUTPUT FORMAT : ------------- To make it easier to parse your skill proposal , follow this exact format for describing the novel skill . End your text with the skill proposal . Do not generate anything after that : " Skill Name ": " SkillName1 ", " Skill Index ": " The skill ’ s index corresponds to the index provided by the skill proposal ", " Skill Description ": " Describe the skill ...", " Skill Function ": " The implemented JAX skill function goes here " < think > ⬇ You are the official Skill Novelty Judge for the Craftax agent ’ s skill development program . Your primary mandate is to strategically guide the agent ’ s learning path by selecting the two most valuable new skills from a batch of proposals . Each skill has Precondition Functions to check if necessary sub - goals ( e . g ., " possess iron ," " near crafting table ") are met . If a precondition fails , the skill calls a prerequisite SHARP , itself defined with its own preconditions , dedicated to achieving that sub - goal , creating a dependency chain . At each environment step , these preconditions are checked sequentially , allowing the system to traverse this chain to find the first skill whose prerequisites are currently satisfied . This sequential , step - by - step checking means previously met conditions can become false , re - triggering their respective sub - goal skills . Finally , a Success Condition verifies if the main objective has been completed , at which point the skill returns a reward to the agent . Your decision must balance the creation of a coherent , step - by - step learning curriculum with the goal of developing a diverse and high - performing set of abilities to best guide the iterative expansion of the agents skill archive . For your decision you must strike a balance between choosing more advanced skills which maximally increase the agents capabilities or more novel skills , which might not directly build on top of agents current skill archive but add a new skill tree and maximally increase the archives diversity . --- ### The Selection Process You must first scrutinize each proposal individually for quality , compliance , and logical soundness . Any proposal that fails this phase is immediately disqualified . * ** Criterion 1: Novelty and Feasibility ** * ** Judge Novelty :** Is the proposed skill a genuinely new capability , or is it a trivial variation or duplication of a skill already in the agent ’ s archive , e . g . CollectThreeWood if CollectWood already exists ? ** Reject redundant proposals .** * ** Evaluate Feasibility :** Does the skill represent a meaningful and achievable behavior within the Craftax environment ? ** Reject nonsensical or conceptually impossible proposals .** * ** Failed Proposals :** Compare the novel skill proposals against the list of previously failed proposals . Reject proposals which strictly reimplement a previously failed proposal . * ** Skill Performance :** For each proposal analyse the success rates of prerequisite skills and judge if it is feasable for the novel proposal to be completed given the success rates of any prerequsite skills . * ** Skill Balance :** You should balance the number of skills of each category in the skill archive . If recent additions have mostly been from a specific category type , you should aim to select proposals from a different category next . * ** Skill Exploration :** You must encourage the agent to reach new levels , while balancing the exploration of previous levels for meaningful skills not present in the skill archive . Favor proposals that either ( a ) meaningfully unlock new levels to explore , or ( b ) fill obvious gaps in the current / previous levels ( e . g ., missing resource collection or mob fighting skills ). * ** Independent Judging :** You must first and foremost adhere to all the rules described here . Even if there already have been skills added to the archive that violate any of the rules described here , you must not repeat the same mistake by adding another skill that violates the same rule . * ** Criterion 2: Logical Scrutiny ** * ** Success Condition :** The condition must be unambiguous and precisely define the skill ’ s unique outcome . For tools , success must be measured by the absolute count ( e . g ., ‘ current_state . inventory . pickaxe >= 1‘). For resources , it must be measured relative to the previous state . * ** Start Conditions & Prerequisites :** All start conditions must be strictly necessary . The specified prerequisite skills must be appropriate , exist in the agent ’ s repertoire , and correctly satisfy the conditions . * ** No Correction Authority :** You are ** not ** allowed to make corrections to the code . If a proposal requires any changes to be valid ( e . g ., syntax errors , incorrect logic , missing prerequisites ), you must ** reject it **. Only select proposals that are ready to be used as - is . * ** Criterion 3: Technical Compliance ** * ** JAX Compliance :** The implementation must be fully JIT - compatible , using only JAX operators . * ** Template Adherence :** The code must strictly follow the provided JAX skill class template . * ** Index Uniqueness :** If the proposed index is taken , assign the next available unique integer . EXAMPLES : The following are a few examples of skill proposal novelty . KillMobWithIronSword if KillMob exists is ** not novel **! Even if killing the mob with an iron sword is more efficient , there already exists a skill with the same general functionality . GoDownLadderFromLake if GoDownLadder exists is ** not novel **! The change in ability is to minor to warrant a completely seperate skill . KillOrcWarrior if KillOrcMage exists is ** novel **! The skill focuses on a related but different mob , so it is novel . MineThreeIron if MineIron exists is ** not novel **! Simple repitions of existing skills are not interesting and novel . DrinkWaterSafe if DrinkWater exists is ** not novel **! Simple variations of existing skills are not interesting and novel and are being handled by the mutation loop . ### Final Verdict and Report Your final output must be a clear and structured report containing the following : 1. ** Selected Skill :** At most two skills can be selected . If no skill passes your judgement you are allowed to reject them all . 2. ** Filtering :** Start by comparing each skill proposal against the example skills . Filter out all skills which are would not be classified as novel . Then , make your final decision based on the set of novel skills . 3. ** Justification :** Provide a concise explanation for your choice , explicitly referencing how the selected skill excels in ** Curriculum Coherence **, ** Strategic Value **, and / or ** Skill Diversity ** compared to the other candidates . 4. ** Output :** Provide the class of the skill which you choose as the optimal next skill to add to agents repertoire in the format provided to you . 5. ** Criteria :** The most important criteria your selected skill must possess is feasibility . Always pick a skill that presents a logical incremental improvement over extremely difficult skills for which the agent does not possess all prerequisite skills . ENVIRONMENT CODE ---------------- $environment_description$ SKILL FUNCTION TEMPLATE -------------------- class Skill : """ A pure logic container for the JAX SkillEnvWrapper . The Wrapper handles the traversal , recursion , and cycle detection . This class only defines the conditions , prerequisites , and success criteria . """ def __init__ ( self ): # Implement : Add the skill ’ s unique index here ( e . g ., 20) self . parent_skill_index = 0 # Implement : The assigned reward for the skill here self . assigned_reward = 1.0 #### Implement the tuples containing the condition and prerequisite functions #### # NOTE : The Wrapper requires these to be tuples of lambdas accepting a single ’ states ’ argument . # Always keep the dummy functions first to prevent empty tuple errors during JIT compilation . self . cond_fns = ( # lambda states : self . _condition_1 (* states ), # lambda states : self . _condition_2 (* states ), ) self . prereq_fns = ( # lambda states : self . _do_condition1 (* states ), # lambda states : self . _do_condition2 (* states ), ) #### Implement the logic functions here #### # def _condition_1 ( self , prev : Any , cur : Any ) -> jnp . bool_ : # # Return True if the condition is satisfied ( e . g ., item is in inventory ) # return cur . inventory . wood >= 1 # def _do_condition1 ( self , prev : Any , cur : Any ) -> Tuple [ int , float , bool ]: # # Return the index of the skill required to satisfy _condition_1 # # Returns : ( Prerequisite Skill Index , 0.0, False ) # return 5, 0.0, False # Example : Index 5 is " Collect Wood " #### Implement the success condition ( Finalize ) here #### def _finalize ( self , prev : Any , cur : Any ) -> Tuple [ int , float , bool ]: # Implement the ultimate success check for this skill # e . g ., success = cur . inventory . crafting_table >= 1 success = False reward = jnp . where ( success , self . assigned_reward , 0.0) return self . parent_skill_index , reward , success SKILL REPERTOIRE : ----------------- Current Agent Skill Repertoire : $skill_repertoire$ Skill performances : $skill_performances$ FAILED PROPOSALS : ---------------- $failed_proposals$ SKILL PROPOSAL --------------- The following is the skill proposal with its name , description , success condition , and necessary starting conditions . $candidate_dict$ $candidate_code$ OUTPUT FORMAT : ------------- If the skill proposal passes your judgement , output your decision following this exact template specified below . Do not output anything after the token [ { " Skill Names ": " The name of the first skill you choose ", " Proposal Index ": Must correspond to the index of the chosen skill proposal , " Decision Reason ": " Explain your reasoning for choosing this specific skill " }, { " Skill Names ": " The name of the second skill you choose ", " Proposal Index ": Must correspond to the index of the chosen skill proposal , " Decision Reason ": " Explain your reasoning for choosing this specific skill " } ] If the skill does not pass your judgement , simply output " Skill did not pass judgement ". < think > Appendix I Skill Mutation Prompts ⬇ You are a helpful skill engineer with the job of designing skills as hierarchical reward programs ( SHARP ) to train an agent in the game of Craftax . Each SHARP consists of precondition functions which define the necessary environment conditions that need to be satisfied in order to be able to execute the parent skill . Each precondition can call an already existing skill from the agents archive of learned skills to satisfy the respective condition . Each SHARP further has a success condition which can check if the parent skill is completed or not . This also returns the reward to train the agent . During the initial implementation there can often be mistakes , oversigths or inefficiencies in the defined SHARPs . Your task is to evolve and mutate an already existing SHARP to optimise it based on its current structure and the other skills present in the agents skill archive . For this you will receive a heuristic which you will use as guidance on how to mutate the sampled SHARP based on the agents skill archive and the description of Craftax . Your mutation will be evaluated against the parent skill in the game of Craftax . If it has a better performance in terms of the success rate of completing the skill , the mutation will be accepted as the new elite . Mutation Rules : 1. Ensure your mutations follow the exact heuristic specification given to you and is sensible . 2. If you add a new new precondition function , ensure that a relevant skill to satisfy it is present in the agents skill archive . 3. Directly follow the output template given to you to define your mutation proposal . 4. All precondition functions should be clearly marked in the mutation preconditions . 5. Carefully analyse the previous failed mutation attempts , if available , to intelligently propose a next mutation . You should not directly reimplement one of the previous failed mutations . 6. Under no circumstances should you mutate the success condition of the parent skill . ENVIRONMENT DESCRIPTION $environment_description SKILL ARCHIVE $skill_repertoire PARENT SKILL $sampled_parent_skill $sampled_parent_skill_code PREVIOUSLY FAILED PROPOSED MUTATIONS $previous_failed_elites MUTATION HEURISTIC $heuristic_name \ $heuristic OUTPUT The mutation for this index is \ $mutation_index . Be sure to use this exact value for the mutation output dict . To make it easier to parse your skill proposal , follow this exact format for describing the novel skill . End your text with the skill proposal . Do not generate anything after that : " Skill Name ": " ParentSkillName ", " Mutation Name ": " Name of the mutated skill ", " Skill Index ": " The skill ’ s index corresponds to the skill idx of the parent skill ", " Mutation Index ": "\ $mutation_index ", " Skill Description ": " Describe the skill ...", " Assigned Reward ": " The reward you assign to the skill ", " Mutation Description ": " Describe the mutation and how it should improve the skill success rate ...", " Success Condition ": " The condition ...", " Mutation Starting Conditions ": { " Condition 1": [" pythonian environment condition ", " SkillName from repertoire or proposals to satisfy condition 1"], " Condition 2": [" pythonian environment condition ", " SkillName from repertoire or proposals to satisfy condition 2"] } < think > ⬇ { " Crossover ": " Add new preconditions and prerequisite skills to the parent skill that synergize with the sampled skill to enhance its success rate . Focus on synergies that improve survival , ability to navigate the environment , or resource management ( e . g ., optimized tool usage ). You can also replace prerequisite skills for existing preconditions with skills from the agent ’ s repertoire that are more likely to successfully fulfil the specified preconditions .", " Efficiency ": " Streamline the skill ’ s execution flow . Analyze the logical sequence of preconditions , ensuring they are ordered for maximum effectiveness ( e . g ., gathering tools before attempting tasks ).", " Simplicity ": " Optimize the skill ’ s implementation to be as efficient and streamlined as possible by identifying at least one redundant or distracting precondition that can be removed . Redundant preconditions are those that repeat other preconditions objectives . Distracting preconditions are those that force the agent to complete unnecessary objectives , diverting attention away from the skill ’ s primary objective . Carefully analyse which conditions are absolutetly necessary to achieve the sampled skills stated goal and then remove all preconditions deemed distracting or redundant ", " Hyperparameters ": " Optimize the numerical parameters governing the skill , such as precondition thresholds and reward values . Adjust these parameters to minimize the need for re - visiting preconditions . Ensure the agent gathers sufficient resources in the first instance to sustain the skill ’ s execution ." } ⬇ You are a JAX coding assistant . Your goal is to translate a " skill mutation proposal " for the Craftax game environment into a valid Python class , implementing it as a new skill for an agent . The skills should be implemented as hierarchical reward programs . Each skill has Precondition Functions to check if necessary sub - goals ( e . g ., " possess iron ," " near crafting table ") are met . If a precondition fails , the skill calls a prerequisite skill , itself defined with its own preconditions , dedicated to achieving that sub - goal , creating a dependency chain . At each environment step , these preconditions are checked sequentially , allowing the system to traverse this chain to find the first skill whose prerequisites are currently satisfied . This sequential , step - by - step checking means previously met conditions can become false , re - triggering their respective sub - goal skills . Finally , a Success Condition verifies if the main objective has been completed , at which point the skill returns a reward to the agent . Using this formulation , your task is to iteratively expand the agents skill archive via these interconnected skills . Your task is to translate a proposed mutation of an existing skill into a valid Python class . INPUT : MUTATION PROPOSAL ====================== You will receive a mutation proposal containing : * ‘ SkillName ‘: The skill ’ s name . * ‘ SkillIndex ‘: The integer ID for the skill , which you must use . * ‘ SkillDescription ‘: A brief explanation of what the skill does . * ‘ SuccessCondition ‘: A condition that is ‘ True ‘ only when the skill is completed successfully . * ‘ Mutation Starting Conditions ‘: A set of prerequisite environment states for the mutated skill . Each is paired with a prerequisite skill that can achieve it . --------------------------------- YOUR TASK : VALIDATION & IMPLEMENTATION ====================================== Follow these rules to implement the mutated skill using the provided JAX class template . **1. Validate the Proposal ** ------------------------- First , check the mutation proposal for correctness . If it ’ s fundamentally flawed and cannot be fixed , reject it and explain why . * ** Success Condition :** * Ensure it ’ s logical and unambiguous . * Correct minor logical errors . * For crafting tools , the success condition must be given as absolute values ( e . g ., ‘ cur . inventory . wood_pickaxe >= 1‘). For resources , it should be relative to the previous state . * ** Mutation Starting Conditions :** * Verify that every prerequisite skill exists in the agent ’ s repertoire . * If a prerequisite skill is invalid , try to replace it with a valid one . If no valid replacement exists , reject the proposal . **2. Implement the JAX Class ** -------------------------- If the proposal is valid ( or you ’ ve corrected it ), implement the class . * ** Indexing :** * Use the exact ‘ SkillIndex ‘ provided in the proposal . * Refer to prerequisite skills by their correct index from the skill repertoire . * ** Code Requirements :** * Your entire implementation ** MUST ** be JAX and JIT - compatible . * ** ALWAYS ** use JAX logical operators ( e . g ., ‘ jnp . logical_and ‘, ‘ jnp . logical_or ‘) instead of Python ’ s native ‘ and ‘ or ‘ or ‘. * If the proposal specifies a condition with " No implementation for a condition needed ," you must implement the condition function but ** must not ** append it to the ‘ self . cond_fns ‘ list , and the respective prerequisite skill must not be added to the ‘ self . prereq_fns ‘ list . --------------------------------- ENVIRONMENT CODE ---------------- $environment_description SKILL FUNCTION TEMPLATE -------------------- class Skill : """ A pure logic container for the JAX SkillEnvWrapper . The Wrapper handles the traversal , recursion , and cycle detection . This class only defines the conditions , prerequisites , and success criteria . """ def __init__ ( self ): # Implement : Add the skill ’ s unique index here ( e . g ., 20) self . parent_skill_index = 0 # Implement : The assigned reward for the skill here self . assigned_reward = 1.0 #### Implement the tuples containing the condition and prerequisite functions #### # NOTE : The Wrapper requires these to be tuples of lambdas accepting a single ’ states ’ argument . # Always keep the dummy functions first to prevent empty tuple errors during JIT compilation . self . cond_fns = ( # lambda states : self . _condition_1 (* states ), # lambda states : self . _condition_2 (* states ), ) self . prereq_fns = ( # lambda states : self . _do_condition1 (* states ), # lambda states : self . _do_condition2 (* states ), ) #### Implement the logic functions here #### # def _condition_1 ( self , prev : Any , cur : Any ) -> jnp . bool_ : # # Return True if the condition is satisfied ( e . g ., item is in inventory ) # return cur . inventory . wood >= 1 # def _do_condition1 ( self , prev : Any , cur : Any ) -> Tuple [ int , float , bool ]: # # Return the index of the skill required to satisfy _condition_1 # # Returns : ( Prerequisite Skill Index , 0.0, False ) # return 5, 0.0, False # Example : Index 5 is " Collect Wood " #### Implement the success condition ( Finalize ) here #### def _finalize ( self , prev : Any , cur : Any ) -> Tuple [ int , float , bool ]: # Implement the ultimate success check for this skill . This must exacactly # match the success condition of the original parent skill , regardless of # any mutations that have been applied . # e . g ., success = cur . inventory . crafting_table >= 1 success = False reward = jnp . where ( success , self . assigned_reward , 0.0) return self . parent_skill_index , reward , success SKILL REPERTOIRE Current Agent Skill Repertoire : $skill_repertoire MUTATION PROPOSAL The following is the mutation proposal . First , verify that all properties are correctly specified according to the rules and environment mechanics and that all prerequisite skills are present in the agent ’ s skill repertoire . If not , correct these errors before creating the skill class in Python . If the proposal cannot be corrected , do not output a skill class . $mutation_proposal SAMPLED PARENT SKILL CODE This is the code of the sampled parent skill . It can diverge from the code in the base repertoire . Always use the version given to you here as the base for your code . $sampled_parent_skill OUTPUT FORMAT To make it easier to parse your skill definition , follow this exact format : " Skill Name ": " SkillName1 ", " Skill Index ": " The skill ’ s index corresponds to the index provided by the skill proposal ", " Skill Description ": " Describe the skill ...", " Skill Function ": " The implemented JAX skill function goes here " < think > Appendix J Policy Planner Prompt ⬇ You are an expert AI architect guiding an agent to solve complex tasks in the Craftax environment . Your goal is to design a high - level policy skill , ‘ BenchmarkSolver ‘, that orchestrates existing skills from a repertoire to complete a specific benchmark . The ‘ BenchmarkSolver ‘ acts as a meta - policy . You must define a sequence of ** preconditions ** ( checks for specific states ) and map each to a ** prerequisite skill ** ( an action to achieve that state ). The agent will execute these in order . YOUR TASK : IMPLEMENT BENCHMARK SOLVER ====================================== Implement the ‘ BenchmarkSolver ‘ skill using the JAX class template provided . This skill orchestrates the agent to achieve the provided ** Benchmark Milestones **. 1. Skill Structure ------------------ * ** Class Name :** ‘ BenchmarkSolver ‘ * ** Index :** $next_skill_index * ** Template :** Use the provided JAX class structure . 2. Logic & Strategy Requirements -------------------------------- You must define the ‘ cond_fns ‘ ( preconditions ) and ‘ prereq_fns ‘ ( actions ) lists . Construct them using the following logic : ### A . Strategic Planning Before strictly following the provided milestones , you must analyze the task requirements to ensure agent survival and efficiency . * ** Preparation :** You are encouraged to define a limited set of auxiliary preconditions for essential tools that are not explicitly listed in the milestones but can assist the agent in completing the milestones . * ** Focus :** Focus on the only the most critical tools that are required to complete the milestones . Avoid adding too many preparatory steps as it may lead to the agent getting distracted from the main task of solving milestones . * ** Efficiency :** The agent can get distracted if you add too many preparatory steps . Ensure that you limit the number of preparatory steps to a minimum and that they can be completed easily by the agent . * ** Ordering :** These preparatory steps must be placed only at the ** beginning ** of your precondition sequence , ensuring the agent is equipped * before * attempting to solve the milestones . ### B . Milestone Execution * After your preparatory steps , define preconditions that mirror the ** Benchmark Milestones ** in order . * Ensure the logical flow allows the agent to progress from one milestone to the next without getting stuck . ### C . Skill Selection * For each precondition , assign the most effective skill from the ** Skill Repertoire **. * ** Best Fit :** Choose the skill with the highest success rate that achieves the goal . * ** Approximation :** If no skill perfectly matches a milestone , use the closest available alternative or a skill that covers multiple milestones ( e . g ., a " Mine Stone " skill might naturally cover a " Craft Stone Pickaxe " milestone ). * ** Constraint :** You must use * existing * skills only . Do not invent new skill names . 3. Code Requirements -------------------- * ** JAX Compatibility :** The implementation must be pure JAX and JIT - compatible . * ** Operators :** ALWAYS use ‘ jnp . logical_and ‘, ‘ jnp . logical_or ‘, etc ., instead of Python native operators . * ** Imports :** Do not add import statements ; assume the environment is pre - loaded . --------------------------------- CRAFTAX ENVIRONMENT ------------------- $environment_description BENCHMARK MILESTONES -------------------- The benchmark task consists of the following milestones that should be achieved in order : $milestones --------------------------------- SKILL REPERTOIRE ---------------- Current Agent Skill Repertoire : $skill_repertoire Skill Success Rates : $success_rates --------------------------------- SKILL FUNCTION TEMPLATE -------------------- class BenchmarkSolver : # Top - level policy for solving the benchmark task def __init__ ( self ): self . parent_skill_index = $next_skill_index self . assigned_reward = 1.0 #### Implement the tuples containing the condition and prerequisite functions #### # NOTE : The Wrapper requires these to be tuples of lambdas accepting a single ’ states ’ argument . # Always keep the dummy functions first to prevent empty tuple errors during JIT compilation . self . cond_fns = ( # lambda states : self . _condition_1 (* states ), # lambda states : self . _condition_2 (* states ), ) self . prereq_fns = ( # lambda states : self . _do_condition1 (* states ), # lambda states : self . _do_condition2 (* states ), ) #### Implement the logic functions here #### # def _condition_1 ( self , prev : Any , cur : Any ) -> jnp . bool_ : # # Return True if the condition is satisfied ( e . g ., item is in inventory ) # return cur . inventory . wood >= 1 # def _do_condition1 ( self , prev : Any , cur : Any ) -> Tuple [ int , float , bool ]: # # Return the index of the skill required to satisfy _condition_1 # # Returns : ( Prerequisite Skill Index , 0.0, False ) # return 5, 0.0, False # Example : Index 5 is " Collect Wood " #### Implement the success condition ( Finalize ) here #### def _finalize ( self , prev : Any , cur : Any ) -> Tuple [ int , float , bool ]: # Implement the ultimate success check for this skill # e . g ., success = cur . inventory . crafting_table >= 1 success = False reward = jnp . where ( success , self . assigned_reward , 0.0) return self . parent_skill_index , reward , success OUTPUT FORMAT : ------------- To make it easier to parse your skill proposal , follow this exact format for describing the novel skill . End your text with the skill proposal . Do not generate anything after that : " Skill Name ": " BenchmarkSolver ", " Skill Description ": " Describe the skill ...", " Skill Function ": " The implemented JAX skill function goes here " Appendix K Discovered Skill Archive In the following we present the skill archive discovered during one of the three runs, consisting of 93 discovered and 3 expert provided skills ( FindTree, FindLake, FindCow ). The skills where discovered over 100 iterations of skill proposal and agent training. FindTree Description: The agent explores the environment until it is in the proximity of a tree (within a radius of 2 blocks). Skill Type: Navigation Assigned Reward: 1 Success Condition: is_near_block(cur, BlockType.TREE.value) Starting Conditions: None FindLake Description: The agent explores the environment to locate a lake, a vital water source to manage its thirst. This skill aims to teach the agent to identify and approach water bodies for survival. Skill Type: Navigation Assigned Reward: 1 Success Condition: is_near_block(cur, BlockType.WATER.value) Starting Conditions: None FindCow Description: The agent explores the overworld until it is within 2 blocks of a cow, a passive mob that can be killed for food. Skill Type: Navigation Assigned Reward: 1 Success Condition: is_near_passive_mob(cur, 0, 2) Starting Conditions: None DrinkWater Description: The agent locates a water source and drinks from it to replenish its thirst intrinsic to maximum capacity, ensuring survival through hydration management. Skill Type: Survival Assigned Reward: 1 Success Condition: cur.player_drink >= (7 + 2 * cur.player_dexterity) Starting Conditions: • C1: is_near_block(cur, BlockType.WATER.value) (Requires: FindLake) MineWood Description: The agent mines a tree to obtain wood, a fundamental resource required for crafting tools and structures. This skill builds upon tree location by teaching the agent to interact with trees to gather raw materials. Skill Type: Resource Gathering Assigned Reward: 1 Success Condition: cur.inventory.wood >= 1 Starting Conditions: • C1: is_near_block(cur, BlockType.TREE.value) (Requires: FindTree) KillCow Description: The agent engages and defeats a cow, a passive mob found in the overworld, to obtain food resources. This skill teaches the agent basic combat mechanics against passive mobs. Skill Type: Fighting Mobs Assigned Reward: 1 Success Condition: was_mob_killed(prev, cur, MobType.PASSIVE.value, PassiveMobType.COW.value) Starting Conditions: • C1: is_near_passive_mob(cur, PassiveMobType.COW.value, 2) (Requires: FindCow) PlaceCraftingTable Description: The agent places a crafting table using wood resources, creating a necessary structure for crafting tools and enabling progression to more advanced skills. Assigned Reward: 1 Skill Type: Crafting Success Condition: is_near_block(cur, BlockType.CRAFTING_TABLE.value) Starting Conditions: • C1: cur.inventory.wood >= 2 (Requires: MineWood) FindStone Description: The agent explores the overworld to locate stone deposits within a 2-block radius, which are essential for crafting stone tools and advancing resource gathering capabilities. Skill Type: Navigation Assigned Reward: 1 Success Condition: is_near_block(cur, BlockType.STONE.value) Starting Conditions: None CraftWoodPickaxe Description: The agent crafts a wooden pickaxe at a crafting table, enabling stone mining for progression. This skill builds upon wood gathering and table placement to teach core crafting mechanics. Skill Type: Crafting Assigned Reward: 1 Success Condition: cur.inventory.pickaxe >= 1 Starting Conditions: • C1: cur.inventory.wood >= 1 (Requires: MineWood) • C1: is_near_block(cur, BlockType.CRAFTING_TABLE.value) (Requires: PlaceCraftingTable) CraftWoodSword Description: The agent crafts a wooden sword at a crafting table using gathered wood resources. This skill builds upon wood acquisition and crafting table placement to teach the agent foundational combat tool creation. Skill Type: Crafting Assigned Reward: 1 Success Condition: cur.inventory.sword >= 1 Starting Conditions: • C1: cur.inventory.wood >= 1 (Requires: MineWood) • C1: is_near_block(cur, BlockType.CRAFTING_TABLE.value) (Requires: PlaceCraftingTable) MineStone Skill Type: Resource Gathering Description: The agent locates stone deposits and mines them using a wooden pickaxe to obtain stone resources. Assigned Reward: 1 Success Condition: cur.inventory.stone >= 1 Starting Conditions: • C1: cur.inventory.pickaxe >= 1 (Req: CraftWoodPickaxe) • C2: is_near_block(cur, BlockType.STONE.value) (Req: FindStone) CraftStonePickaxe Skill Type: Crafting Description: The agent crafts a stone pickaxe at a crafting table using gathered wood and stone resources. Assigned Reward: 1 Success Condition: cur.inventory.pickaxe >= 2 Starting Conditions: • C1: cur.inventory.wood >= 1 (Req: MineWood) • C2: cur.inventory.stone >= 1 (Req: MineStone) • C3: is_near_block(cur, BlockType.CRAFTING_TABLE.value) (Req: PlaceCraftingTable) FindCoal Skill Type: Navigation Description: The agent explores the overworld (level 0) to locate coal deposits within a 2-block radius. Assigned Reward: 1 Success Condition: (cur.player_level == 0) & is_near_block(cur, BlockType.COAL.value) Starting Conditions: • C1: is_near_block(cur, BlockType.STONE.value) (Req: FindStone) MineCoal Skill Type: Resource Gathering Description: The agent locates coal deposits identified in stone-rich areas and mines them using a wooden pickaxe. Assigned Reward: 1 Success Condition: cur.inventory.coal >= 1 Starting Conditions: • C1: cur.inventory.pickaxe >= 1 (Req: CraftWoodPickaxe) • C2: is_near_block(cur, BlockType.COAL.value) (Req: FindCoal) CraftStoneSword Skill Type: Crafting Description: The agent crafts a stone sword at a crafting table using gathered wood and stone resources. Assigned Reward: 1 Success Condition: cur.inventory.sword >= 2 Starting Conditions: • C1: cur.inventory.wood >= 1 (Req: MineWood) • C2: cur.inventory.stone >= 1 (Req: MineStone) • C3: is_near_block(cur, BlockType.CRAFTING_TABLE.value) (Req: PlaceCraftingTable) PlaceFurnace Skill Type: Crafting Description: The agent places a furnace using gathered stone resources, creating essential infrastructure for smelting ores. Assigned Reward: 1 Success Condition: is_near_block(cur, BlockType.FURNACE.value) Starting Conditions: • C1: cur.inventory.stone >= 1 (Req: MineStone) CraftTorch Skill Type: Crafting Description: The agent crafts torches at a crafting table using gathered wood and coal resources. Assigned Reward: 1 Success Condition: cur.inventory.torches >= 1 Starting Conditions: • C1: cur.inventory.wood >= 1 (Req: MineWood) • C2: cur.inventory.coal >= 1 (Req: MineCoal) • C3: is_near_block(cur, BlockType.CRAFTING_TABLE.value) (Req: PlaceCraftingTable) MineSapling Skill Type: Resource Gathering Description: The agent mines grass blocks in the overworld (level 0) near cows to obtain saplings. Assigned Reward: 1 Success Condition: cur.inventory.sapling >= 1 Starting Conditions: • C1: is_near_passive_mob(cur, PassiveMobType.COW.value, 2) (Req: FindCow) EatFood Skill Type: Survival Description: The agent locates and kills cows in the overworld (level 0) until its food level reaches maximum capacity. Assigned Reward: 1 Success Condition: (cur.player_level == 0) & (cur.player_food >= (7 + 2 * cur.player_dexterity)) Starting Conditions: • C1: is_near_passive_mob(cur, PassiveMobType.COW.value, 2) (Req: FindCow) CraftArrow Skill Type: Crafting Description: The agent crafts arrows at a crafting table using gathered wood and stone resources. Assigned Reward: 1 Success Condition: cur.inventory.arrows >= 1 Starting Conditions: • C1: cur.inventory.wood >= 1 (Req: MineWood) • C2: cur.inventory.stone >= 1 (Req: MineStone) • C3: is_near_block(cur, BlockType.CRAFTING_TABLE.value) (Req: PlaceCraftingTable) KillZombie Skill Type: Fighting Mobs Description: The agent locates and defeats a zombie, an aggressive melee mob that spawns in stone-rich areas. Assigned Reward: 1 Success Condition: (cur.player_level == 0) & was_mob_killed(prev, cur, MobType.MELEE.value, MeleeMobType.ZOMBIE.value) Starting Conditions: • C1: is_near_melee_mob(cur, MeleeMobType.ZOMBIE.value, 2) (Req: FindStone) MineIron Skill Type: Resource Gathering Description: The agent mines iron ore deposits using a stone pickaxe, obtaining iron resources. Assigned Reward: 1 Success Condition: cur.inventory.iron >= 1 Starting Conditions: • C1: cur.inventory.pickaxe >= 2 (Req: CraftStonePickaxe) • C2: is_near_block(cur, BlockType.STONE.value) (Req: FindStone) KillSkeleton Skill Type: Fighting Mobs Description: The agent locates and defeats a skeleton, a ranged mob that spawns in stone-rich areas. Assigned Reward: 1 Success Condition: (cur.player_level == 0) & was_mob_killed(prev, cur, MobType.RANGED.value, RangedMobType.SKELETON.value) Starting Conditions: • C1: cur.inventory.sword >= 1 (Req: CraftWoodSword) • C2: (cur.player_level == 0) & is_near_ranged_mob(cur, RangedMobType.SKELETON.value, 2) (Req: FindStone) SleepToFullEnergy Skill Type: Survivability Description: The agent ensures its survival intrinsics are stabilized and eliminates threats before sleeping to recover energy. Assigned Reward: 1 Success Condition: (cur.player_level == 0) & (cur.player_energy >= (7 + 2 * cur.player_dexterity)) Starting Conditions: • C1: (cur.player_level == 0) & (cur.player_food >= (7 + 2 * cur.player_dexterity)) (Req: EatFood) • C2: (cur.player_level == 0) & (cur.player_drink >= (7 + 2 * cur.player_dexterity)) (Req: DrinkWater) • C3: not is_near_melee_mob(cur, MeleeMobType.ZOMBIE.value, 3) (Req: KillZombie) • C4: not is_near_ranged_mob(cur, RangedMobType.SKELETON.value, 3) (Req: KillSkeleton) FindLadderDown Skill Type: Navigation Description: The agent explores the overworld (level 0) to locate the downward ladder required to descend to the dungeon level. Assigned Reward: 1 Success Condition: (cur.player_level == 0) & exists_local_object(cur, ItemType.LADDER_DOWN.value, radius=1, check_items=True) Starting Conditions: None PlaceTorch Skill Type: Survivability Description: The agent utilizes crafted torches to place them on valid adjacent blocks, illuminating the immediate surroundings. Assigned Reward: 1 Success Condition: exists_stationary_object(cur, ItemType.TORCH.value) Starting Conditions: • C1: cur.inventory.torches >= 1 (Req: CraftTorch) RecoverHealth Skill Type: Survivability Description: The agent ensures its survival intrinsics are fully replenished and in a safe environment, allowing passive health regeneration. Assigned Reward: 1 Success Condition: (cur.player_level == 0) & (cur.player_health >= (8 + cur.player_strength)) Starting Conditions: • C1: (cur.player_level == 0) & (cur.player_energy >= (7 + 2 * cur.player_dexterity)) (Req: SleepToFullEnergy) StepOntoLadderDown Skill Type: Navigation Description: The agent moves from being within a 2-block radius of the downward ladder to standing exactly on the ladder block. Assigned Reward: 1 Success Condition: (cur.player_level == 0) & (cur.item_map[cur.player_level, cur.player_position[0], cur.player_position[1]] == ItemType.LADDER_DOWN.value) Starting Conditions: • C1: exists_local_object(cur, ItemType.LADDER_DOWN.value, radius=1, check_items=True) (Req: FindLadderDown) FindIron Skill Type: Resource Gathering Description: The agent explores the overworld (level 0) to locate iron ore deposits within a 2-block radius. Assigned Reward: 1 Success Condition: (cur.player_level == 0) & is_near_block(cur, BlockType.IRON.value) Starting Conditions: • C1: (cur.player_level == 0) & is_near_block(cur, BlockType.STONE.value) (Req: FindStone) DescendLadder Skill Type: Navigation Description: The agent executes the descend action while positioned on the downward ladder in the overworld to transition to the dungeon. Assigned Reward: 1 Success Condition: cur.player_level == 1 Starting Conditions: • C1: (cur.player_level == 0) & (cur.item_map[cur.player_level, cur.player_position[0], cur.player_position[1]] == ItemType.LADDER_DOWN.value) (Req: StepOntoLadderDown) FindChestLevel1 Skill Type: Navigation Description: The agent navigates the dungeon level (level 1) to locate a chest within a 2-block radius. Assigned Reward: 1 Success Condition: (cur.player_level == 1) & is_near_block(cur, BlockType.CHEST.value) Starting Conditions: • C1: cur.player_level == 1 (Req: DescendLadder) FindOrcSoldierLevel1 Skill Type: Navigation Description: The agent explores the dungeon level (level 1) to locate an Orc Soldier, a melee mob that spawns in dungeon rooms, within a 2-block radius. Assigned Reward: 1 Success Condition: (cur.player_level == 1) & is_near_melee_mob(cur, MeleeMobType.ORC_SOLDIER.value, 2) Starting Conditions: • C1: cur.player_level == 1 (Req: DescendLadder) FaceLadderDown Skill Type: Navigation Description: The agent positions itself adjacent to the downward ladder on the overworld (level 0) and rotates to face the ladder block. Assigned Reward: 1 Success Condition: (cur.player_level == 0) & (cur.item_map[0, cur.player_position[0] + DIRECTIONS[cur.player_direction][0], cur.player_position[1] + DIRECTIONS[cur.player_direction][1]] == ItemType.LADDER_DOWN.value) Starting Conditions: • C1: (cur.player_level == 0) & exists_local_object(cur, ItemType.LADDER_DOWN.value, radius=1, check_items=True) (Req: FindLadderDown) KillOrcWarrior Skill Type: Fighting Mobs Description: The agent locates and defeats an Orc Warrior, a melee mob that spawns in dungeon rooms on level 1. Assigned Reward: 1 Success Condition: (cur.player_level == 1) & was_mob_killed(prev, cur, MobType.MELEE.value, MeleeMobType.ORC_SOLDIER.value) Starting Conditions: • C1: cur.player_level == 1 (Req: DescendLadder) • C2: cur.inventory.sword >= 1 (Req: CraftWoodSword) • C3: (cur.player_level == 1) & is_near_melee_mob(cur, MeleeMobType.ORC_SOLDIER.value, 2) (Req: FindOrcSoldierLevel1) OpenChestForBowLevel1 Skill Type: Resource Gathering Description: The agent locates and opens the first chest on the dungeon level (level 1) to obtain a bow. Assigned Reward: 1 Success Condition: (cur.player_level == 1) & (cur.inventory.bow >= 1) Starting Conditions: • C1: cur.player_level == 1 (Req: DescendLadder) • C2: (cur.player_level == 1) & is_near_block(cur, BlockType.CHEST.value) (Req: FindChestLevel1) FindFountainLevel1 Skill Type: Navigation Description: The agent leverages its ability to locate dungeon chests to efficiently explore adjacent rooms and identify fountains. Assigned Reward: 1 Success Condition: (cur.player_level == 1) & is_near_block(cur, BlockType.FOUNTAIN.value) Starting Conditions: • C1: cur.player_level == 1 (Req: DescendLadder) • C2: is_near_block(cur, BlockType.CHEST.value) (Req: FindChestLevel1) FindDungeonTorch Skill Type: Navigation Description: The agent explores the dungeon level (level 1) to locate pre-placed torches in room corners, serving as navigation landmarks. Assigned Reward: 1 Success Condition: (cur.player_level == 1) & exists_stationary_object(cur, ItemType.TORCH.value) Starting Conditions: • C1: cur.player_level == 1 (Req: DescendLadder) OpenChestForArrowsLevel1 Skill Type: Resource Gathering Description: The agent locates and opens a chest on the dungeon level (level 1) to obtain arrows. Assigned Reward: 1 Success Condition: cur.inventory.arrows >= 1 Starting Conditions: • C1: cur.player_level == 1 (Req: DescendLadder) • C2: is_near_block(cur, BlockType.CHEST.value) (Req: FindChestLevel1) DrinkFromFountainLevel1 Skill Type: Survivability Description: The agent locates a fountain in the dungeon level (level 1) and drinks from it to replenish thirst. Assigned Reward: 1 Success Condition: (cur.player_level == 1) & (cur.player_drink >= (7 + 2 * cur.player_dexterity)) Starting Conditions: • C1: (cur.player_level == 1) & is_near_block(cur, BlockType.FOUNTAIN.value) (Req: FindFountainLevel1) PlaceFurnaceAdjacent Skill Type: Crafting Description: The agent locates the existing crafting table and strategically places a furnace in an adjacent block (including diagonally). Assigned Reward: 1 Success Condition: is_near_block(cur, BlockType.CRAFTING_TABLE.value) & is_near_block(cur, BlockType.FURNACE.value) Starting Conditions: • C1: is_near_block(cur, BlockType.CRAFTING_TABLE.value) (Req: PlaceCraftingTable) • C2: cur.inventory.stone >= 1 (Req: MineStone) CraftIronArmour Skill Type: Crafting Description: The agent crafts a piece of iron armour at a crafting table while adjacent to a furnace using gathered iron and coal resources. Assigned Reward: 1 Success Condition: jnp.any(cur.inventory.armour >= 1) Starting Conditions: • C1: cur.inventory.iron >= 3 (Req: MineIron) • C2: cur.inventory.coal >= 3 (Req: MineCoal) • C3: is_near_block(cur, BlockType.CRAFTING_TABLE.value) (Req: PlaceCraftingTable) • C4: is_near_block(cur, BlockType.FURNACE.value) (Req: PlaceFurnace) CraftIronPickaxe Skill Type: Crafting Description: The agent crafts an iron pickaxe at a crafting table while ensuring the furnace is placed adjacently. Assigned Reward: 1 Success Condition: cur.inventory.pickaxe >= 3 Starting Conditions: • C1-C4: Wood, Stone, Iron, Coal >= 1 • C5: is_near_block(cur, BlockType.CRAFTING_TABLE.value) & is_near_block(cur, BlockType.FURNACE.value) (Req: PlaceFurnaceAdjacent) FindLadderDownLevel1 Skill Type: Navigation Description: The agent explores the dungeon level (level 1) to locate the downward ladder by leveraging dungeon torches as navigation landmarks. Assigned Reward: 1 Success Condition: (cur.player_level == 1) & exists_local_object(cur, ItemType.LADDER_DOWN.value, radius=1, check_items=True) Starting Conditions: • C1: cur.player_level == 1 (Req: DescendLadder) • C2: exists_stationary_object(cur, ItemType.TORCH.value) (Req: FindDungeonTorch) CraftIronSword Skill Type: Crafting Description: The agent crafts an iron sword at a crafting table while ensuring furnace adjacency. Assigned Reward: 1 Success Condition: cur.inventory.sword >= 3 Starting Conditions: • C1-C4: Wood, Stone, Iron, Coal >= 1 • C5: is_near_block(cur, BlockType.CRAFTING_TABLE.value) & is_near_block(cur, BlockType.FURNACE.value) (Req: PlaceFurnaceAdjacent) StepOntoLadderDownLevel1 Skill Type: Navigation Description: The agent navigates from proximity to the downward ladder on the dungeon level (level 1) to stand precisely on the ladder block. Assigned Reward: 1 Success Condition: (cur.player_level == 1) & (cur.item_map[cur.player_level, cur.player_position[0], cur.player_position[1]] == ItemType.LADDER_DOWN.value) Starting Conditions: • C1: (cur.player_level == 1) & exists_local_object(cur, ItemType.LADDER_DOWN.value, radius=1, check_items=True) (Req: FindLadderDownLevel1) FindUpLadderLevel1 Skill Type: Navigation Description: The agent explores the starting room on the dungeon level (level 1) to locate the upward ladder. Assigned Reward: 1 Success Condition: (cur.player_level == 1) & exists_ladder_object(cur, ItemType.LADDER_UP.value) Starting Conditions: • C1: cur.player_level == 1 (Req: DescendLadder) MineGrassForSapling Skill Type: Resource Gathering Description: The agent mines grass blocks in the overworld (level 0) to obtain saplings, a critical resource for establishing sustainable food sources. Assigned Reward: 1 Success Condition: cur.inventory.sapling >= 1 Starting Conditions: • C1: (cur.player_level == 0) & is_near_block(cur, BlockType.GRASS.value) (Req: FindTree) KillOrcMage Skill Type: Fighting mobs Description: The agent locates and defeats an Orc Mage, a ranged mob that spawns in dungeon rooms on level 1. Assigned Reward: 1 Success Condition: (cur.player_level == 1) & was_mob_killed(prev, cur, MobType.RANGED.value, RangedMobType.ORC_MAGE.value) Starting Conditions: • C1: cur.player_level == 1 (Req: DescendLadder) • C2: cur.inventory.bow >= 1 (Req: OpenChestForBowLevel1) • C3: cur.inventory.arrows >= 1 (Req: CraftArrow) • C4: is_near_ranged_mob(cur, RangedMobType.ORC_MAGE.value, 2) (Req: FindDungeonTorch) ExitRoomToCorridorLevel1 Skill Type: Navigation Description: The agent learns to exit a dungeon room by moving away from corner torches into the connecting corridor. Assigned Reward: 1 Success Condition: not exists_stationary_object(cur, ItemType.TORCH.value) Starting Conditions: • C1: exists_stationary_object(cur, ItemType.TORCH.value) (Req: FindDungeonTorch) EnterAdjacentRoomLevel1 Skill Type: Navigation Description: The agent navigates from the starting room (containing the upward ladder) on the dungeon level to an adjacent room using torch landmarks. Assigned Reward: 1 Success Condition: (cur.player_level == 1) & exists_stationary_object(cur, ItemType.TORCH.value) & ~exists_ladder_object(cur, ItemType.LADDER_UP.value) Starting Conditions: • C1: (cur.player_level == 1) & exists_ladder_object(cur, ItemType.LADDER_UP.value) (Req: FindUpLadderLevel1) EnterNewRoomFromCorridorLevel1 Skill Type: Navigation Description: The agent navigates from the dungeon corridor (after exiting a room) on level 1 to enter an adjacent room by detecting torches. Assigned Reward: 1 Success Condition: exists_stationary_object(cur, ItemType.TORCH.value) Starting Conditions: • C1: not exists_stationary_object(cur, ItemType.TORCH.value) (Req: ExitRoomToCorridorLevel1) FindSnailLevel1 Skill Type: Navigation Description: The agent explores the dungeon level (level 1) to locate a snail, a passive mob that spawns in dungeon rooms. Assigned Reward: 1 Success Condition: (cur.player_level == 1) & is_near_passive_mob(cur, PassiveMobType.SNAIL.value, 2) Starting Conditions: • C1: cur.player_level == 1 (Req: DescendLadder) • C2: exists_stationary_object(cur, ItemType.TORCH.value) (Req: FindDungeonTorch) KillSnailLevel1 Skill Type: Fighting mobs Description: The agent locates and defeats a snail, a passive mob that spawns in dungeon rooms on level 1, to obtain food resources. Assigned Reward: 1 Success Condition: (cur.player_level == 1) & was_mob_killed(prev, cur, MobType.PASSIVE.value, PassiveMobType.SNAIL.value) Starting Conditions: • C1: (cur.player_level == 1) (Req: DescendLadder) • C2: cur.inventory.sword >= 1 (Req: CraftWoodSword) • C3: is_near_passive_mob(cur, PassiveMobType.SNAIL.value, 2) (Req: FindFountainLevel1) EnterNewRoomExcludingUpLadderLevel1 Skill Type: Navigation Description: The agent navigates from the corridor to enter an adjacent room that does not contain the upward ladder, confirming new room entry. Assigned Reward: 1 Success Condition: (cur.player_level == 1) & exists_stationary_object(cur, ItemType.TORCH.value) & ~exists_ladder_object(cur, ItemType.LADDER_UP.value) Starting Conditions: • C1: not exists_stationary_object(cur, ItemType.TORCH.value) (Req: ExitRoomToCorridorLevel1) KillEightMobsLevel1 Skill Type: Fighting mobs Description: The agent eliminates 8 hostile or passive mobs to satisfy the requirement for opening the downward ladder to level 2. Assigned Reward: 1 Success Condition: (cur.player_level == 1) & (cur.monsters_killed[1] >= 8) Starting Conditions: • C1: cur.player_level == 1 (Req: DescendLadder) • C2: cur.inventory.sword >= 1 (Req: CraftWoodSword) FaceLadderDownLevel1 Skill Type: Navigation Description: The agent positions itself adjacent to the downward ladder on the dungeon level and rotates to face the ladder block. Assigned Reward: 1 Success Condition: (cur.player_level == 1) & (cur.item_map[cur.player_level, cur.player_position[0] + DIRECTIONS[cur.player_direction][0], cur.player_position[1] + DIRECTIONS[cur.player_direction][1]] == ItemType.LADDER_DOWN.value) Starting Conditions: • C1: (cur.player_level == 1) & exists_local_object(cur, ItemType.LADDER_DOWN.value, radius=1, check_items=True) (Req: FindLadderDownLevel1) ApproachLadderDownLevel1 Skill Type: Navigation Description: The agent navigates from proximity to the downward ladder to a position directly adjacent (1 block away). Assigned Reward: 1 Success Condition: (cur.player_level == 1) & ((cur.item_map[1, cur.player_position[0]+1, cur.player_position[1]] == ItemType.LADDER_DOWN.value) | (cur.item_map[1, cur.player_position[0]-1, cur.player_position[1]] == ItemType.LADDER_DOWN.value) | ... ) Starting Conditions: • C1: (cur.player_level == 1) & exists_local_object(cur, ItemType.LADDER_DOWN.value, radius=1, check_items=True) (Req: FindLadderDownLevel1) DescendLadderLevel1 Skill Type: Navigation Description: The agent executes the descend action while positioned on the downward ladder to transition to the Gnomish Mines (level 2). Assigned Reward: 1 Success Condition: cur.player_level == 2 Starting Conditions: • C1: (cur.player_level == 1) & (cur.monsters_killed[1] >= 8) (Req: KillEightMobsLevel1) • C2: (cur.player_level == 1) & (cur.item_map[cur.player_level, cur.player_position[0], cur.player_position[1]] == ItemType.LADDER_DOWN.value) (Req: StepOntoLadderDownLevel1) FindWaterLevel2 Skill Type: Navigation Description: The agent navigates the Gnomish Mines level (level 2) to locate a water pool within a 2-block radius. Assigned Reward: 1 Success Condition: (cur.player_level == 2) & is_near_block(cur, BlockType.WATER.value) Starting Conditions: • C1: (cur.player_level == 2) (Req: DescendLadderLevel1) FindUpLadderLevel2 Skill Type: Navigation Description: The agent explores the Gnomish Mines level (level 2) to locate the upward ladder (leading back to level 1). Assigned Reward: 1 Success Condition: (cur.player_level == 2) & exists_ladder_object(cur, ItemType.LADDER_UP.value) Starting Conditions: • C1: cur.player_level == 2 (Req: DescendLadderLevel1) FindWaterWithTorchLevel2 Skill Type: Navigation Description: The agent leverages placed torches to illuminate the dark Gnomish Mines and locates water pools within a 2-block radius. Assigned Reward: 1 Success Condition: (cur.player_level == 2) & is_near_block(cur, BlockType.WATER.value) Starting Conditions: • C1: cur.player_level == 2 (Req: DescendLadderLevel1) • C2: exists_stationary_object(cur, ItemType.TORCH.value) (Req: PlaceTorch) MoveToEdgeOfLightLevel2 Skill Type: Navigation Description: The agent moves from the spawn point on level 2 to a position where the light level drops below 0.5. Assigned Reward: 1 Success Condition: (cur.player_level == 2) & (cur.light_map[2, cur.player_position[0], cur.player_position[1]] < 0.5) Starting Conditions: • C1: cur.player_level == 2 (Req: DescendLadderLevel1) FindStalagmiteLevel2 Skill Type: Navigation Description: The agent explores the Gnomish Mines level (level 2) to locate stalagmites within a 2-block radius. Assigned Reward: 1 Success Condition: (cur.player_level == 2) & is_near_block(cur, BlockType.STALAGMITE.value) Starting Conditions: • C1: cur.player_level == 2 (Req: DescendLadderLevel1) MineStalagmiteLevel2 Skill Type: Resource Gathering Description: The agent mines stalagmites in the Gnomish Mines (level 2) to obtain stone resources. Assigned Reward: 1 Success Condition: cur.inventory.stone >= 1 Starting Conditions: • C1: cur.player_level == 2 (Req: DescendLadderLevel1) • C2: exists_stationary_object(cur, ItemType.TORCH.value) (Req: PlaceTorch) • C3: cur.inventory.pickaxe >= 1 (Req: CraftWoodPickaxe) NavigateToWaterFromStalagmiteLevel2 Skill Type: Navigation Description: The agent leverages stalagmites as environmental landmarks in the Gnomish Mines to navigate toward nearby water pools. Assigned Reward: 1 Success Condition: (cur.player_level == 2) & is_near_block(cur, BlockType.WATER.value) Starting Conditions: • C1: (cur.player_level == 2) & is_near_block(cur, BlockType.STALAGMITE.value) (Req: FindStalagmiteLevel2) • C2: exists_stationary_object(cur, ItemType.TORCH.value) (Req: PlaceTorch) PlaceTorchAtEdgeLevel2 Skill Type: Navigation Description: The agent strategically places a torch at the boundary of existing light (light level ¡ 0.5) in the Gnomish Mines. Assigned Reward: 1 Success Condition: (cur.player_level == 2) & (cur.item_map[2, cur.player_position[0], cur.player_position[1]] == ItemType.TORCH.value) Starting Conditions: • C1: (cur.player_level == 2) & (cur.light_map[2, cur.player_position[0], cur.player_position[1]] < 0.5) (Req: MoveToEdgeOfLightLevel2) • C2: cur.inventory.torches >= 1 (Req: CraftTorch) MoveIntoLitAreaLevel2 Skill Type: Survivability Description: The agent moves from the torch placement position at the edge of darkness into the newly illuminated area. Assigned Reward: 1 Success Condition: (cur.player_level == 2) & (cur.light_map[2, cur.player_position[0], cur.player_position[1]] >= 0.5) Starting Conditions: • C1: (cur.player_level == 2) & (cur.light_map[2, cur.player_position[0], cur.player_position[1]] < 0.5) (Req: MoveToEdgeOfLightLevel2) • C2: (cur.player_level == 2) & exists_stationary_object(cur, ItemType.TORCH.value) (Req: PlaceTorchAtEdgeLevel2) MoveToNextEdgeOfLightLevel2 Skill Type: Navigation Description: The agent moves from the center of a newly created lit area to the next edge of darkness, enabling progressive exploration. Assigned Reward: 1 Success Condition: (cur.player_level == 2) & (cur.light_map[2, cur.player_position[0], cur.player_position[1]] < 0.5) Starting Conditions: • C1: cur.player_level == 2 (Req: DescendLadderLevel1) • C2: (cur.player_level == 2) & (cur.light_map[2, cur.player_position[0], cur.player_position[1]] >= 0.5) (Req: MoveIntoLitAreaLevel2) MoveAwayFromUpLadderLevel2 Skill Type: Navigation Description: The agent navigates away from the upward ladder spawn point in the Gnomish Mines to a position at least 5 blocks distant. Assigned Reward: 1 Success Condition: (cur.player_level == 2) & (jnp.abs(cur.player_position[0] - cur.up_ladders[2][0]) + jnp.abs(cur.player_position[1] - cur.up_ladders[2][1]) >= 5) Starting Conditions: • C1: (cur.player_level == 2) & exists_ladder_object(cur, ItemType.LADDER_UP.value) (Req: FindUpLadderLevel2) FindBatLevel2 Skill Type: Navigation Description: The agent explores the Gnomish Mines level (level 2) by leveraging water pools (located via torches) to find bats, passive mobs that spawn near water sources. Assigned Reward: 1 Success Condition: (cur.player_level == 2) & is_near_passive_mob(cur, PassiveMobType.BAT.value, 2) Starting Conditions: • C1: cur.player_level == 2 (Req: DescendLadderLevel1) • C2: (cur.player_level == 2) & is_near_block(cur, BlockType.WATER.value) (Req: FindWaterWithTorchLevel2) MineCoalNearWaterLevel2 Skill Type: Resource Gathering Description: The agent leverages water pools as reliable landmarks in the Gnomish Mines to mine coal deposits within stone-rich areas. Assigned Reward: 1 Success Condition: (cur.player_level == 2) & (cur.inventory.coal >= 1) Starting Conditions: • C1: cur.inventory.pickaxe >= 1 (Req: CraftWoodPickaxe) • C2: exists_stationary_object(cur, ItemType.TORCH.value) (Req: PlaceTorch) • C3: (cur.player_level == 2) & is_near_block(cur, BlockType.WATER.value) (Req: NavigateToWaterFromStalagmiteLevel2) KillGnomeWarriorNearUpLadderLevel2 Skill Type: Fighting mobs Description: The agent remains within the naturally lit area around the upward ladder spawn point and eliminates a Gnome Warrior. Assigned Reward: 1 Success Condition: (cur.player_level == 2) & was_mob_killed(prev, cur, MobType.MELEE.value, MeleeMobType.GNOME_WARRIOR.value) Starting Conditions: • C1: cur.player_level == 2 (Req: DescendLadderLevel1) • C2: (cur.player_level == 2) & exists_ladder_object(cur, ItemType.LADDER_UP.value) (Req: FindUpLadderLevel2) • C3: cur.inventory.sword >= 1 (Req: CraftWoodSword) KillBatLevel2 Skill Type: Fighting mobs Description: The agent locates and defeats a bat, a passive mob that spawns near water pools in the Gnomish Mines level (level 2), to obtain food resources. Assigned Reward: 1 Success Condition: (cur.player_level == 2) & was_mob_killed(prev, cur, MobType.PASSIVE.value, PassiveMobType.BAT.value) Starting Conditions: • C1: cur.player_level == 2 (Req: DescendLadderLevel1) • C2: cur.inventory.sword >= 1 (Req: CraftWoodSword) • C3: exists_stationary_object(cur, ItemType.TORCH.value) (Req: PlaceTorch) • C4: is_near_passive_mob(cur, PassiveMobType.BAT.value, 2) (Req: FindBatLevel2) CheckLadderInLitAreaLevel2 Skill Type: Navigation Description: After moving into a newly illuminated area in the Gnomish Mines, the agent systematically checks for the downward ladder. Assigned Reward: 1 Success Condition: (cur.player_level == 2) & exists_local_object(cur, ItemType.LADDER_DOWN.value, radius=1, check_items=True) Starting Conditions: • C1: (cur.player_level == 2) & (cur.light_map[2, cur.player_position[0], cur.player_position[1]] >= 0.5) (Req: MoveIntoLitAreaLevel2) DrinkWaterLevel2 Skill Type: Survivability Description: The agent locates a water pool in the Gnomish Mines level (level 2) and drinks from it to replenish thirst. Assigned Reward: 1 Success Condition: (cur.player_level == 2) & (cur.player_drink >= (7 + 2 * cur.player_dexterity)) Starting Conditions: • C1: (cur.player_level == 2) & is_near_block(cur, BlockType.WATER.value) (Req: NavigateToWaterFromStalagmiteLevel2) ApproachWaterLevel2 Skill Type: Survivability Description: The agent moves from proximity (within 2 blocks) to a water pool in the Gnomish Mines to an adjacent position (1 block away). Assigned Reward: 1 Success Condition: (cur.player_level == 2) & ((cur.map[2, cur.player_position[0]+1, cur.player_position[1]] == BlockType.WATER.value) | ... ) Starting Conditions: • C1: (cur.player_level == 2) & is_near_block(cur, BlockType.WATER.value) (Req: NavigateToWaterFromStalagmiteLevel2) CheckLadderAfterTorchLevel2 Skill Type: Navigation Description: After placing a torch at the edge of darkness, the agent immediately checks the full 11x11 illuminated area for the downward ladder. Assigned Reward: 1 Success Condition: (cur.player_level == 2) & exists_stationary_object(cur, ItemType.LADDER_DOWN.value) Starting Conditions: • C1: (cur.player_level == 2) & exists_stationary_object(cur, ItemType.TORCH.value) (Req: PlaceTorchAtEdgeLevel2) ApproachLadderDownLevel2 Skill Type: Navigation Description: The agent moves from proximity to the downward ladder on the Gnomish Mines level (level 2) to a position directly adjacent. Assigned Reward: 1 Success Condition: (cur.player_level == 2) & ((cur.item_map[2, cur.player_position[0]+1, cur.player_position[1]] == ItemType.LADDER_DOWN.value) | ... ) Starting Conditions: • C1: (cur.player_level == 2) & exists_local_object(cur, ItemType.LADDER_DOWN.value, radius=1, check_items=True) (Req: CheckLadderInLitAreaLevel2) CheckLadderInFullLitAreaLevel2 Skill Type: Navigation Description: After moving into a newly illuminated area, the agent performs a comprehensive 10-block radius scan to detect the downward ladder. Assigned Reward: 1 Success Condition: (cur.player_level == 2) & exists_stationary_object(cur, ItemType.LADDER_DOWN.value) Starting Conditions: • C1: cur.player_level == 2 (Req: DescendLadderLevel1) • C2: (cur.player_level == 2) & (cur.light_map[2, cur.player_position[0], cur.player_position[1]] >= 0.5) (Req: MoveIntoLitAreaLevel2) KillEightMobsLevel2 Skill Type: Fighting mobs Description: The agent explores the Gnomish Mines level (level 2) while maintaining position in lit areas to eliminate 8 hostile or passive mobs. Assigned Reward: 1 Success Condition: (cur.player_level == 2) & (cur.monsters_killed[2] >= 8) Starting Conditions: • C1: cur.player_level == 2 (Req: DescendLadderLevel1) • C2: cur.inventory.sword >= 1 (Req: CraftWoodSword) • C3: (cur.player_level == 2) & (cur.light_map[2, cur.player_position[0], cur.player_position[1]] >= 0.5) (Req: MoveIntoLitAreaLevel2) MineCoalLevel2 Skill Type: Resource Gathering Description: The agent locates and mines coal deposits in the Gnomish Mines (level 2) using a wooden pickaxe to obtain coal resources. Assigned Reward: 1 Success Condition: (cur.player_level == 2) & (cur.inventory.coal > prev.inventory.coal) Starting Conditions: • C1: cur.player_level == 2 (Req: DescendLadderLevel1) • C2: cur.inventory.pickaxe >= 1 (Req: CraftWoodPickaxe) • C3: (cur.player_level == 2) & is_near_block(cur, BlockType.COAL.value) (Req: FindStalagmiteLevel2) FaceLadderDownLevel2 Skill Type: Navigation Description: The agent positions itself adjacent to the downward ladder on the Gnomish Mines level (level 2) and rotates to face the ladder block. Assigned Reward: 1 Success Condition: (cur.player_level == 2) & (cur.item_map[2, cur.player_position[0] + DIRECTIONS[cur.player_direction][0], cur.player_position[1] + DIRECTIONS[cur.player_direction][1]] == ItemType.LADDER_DOWN.value) Starting Conditions: • C1: (cur.player_level == 2) & ((cur.item_map[2, cur.player_position[0]+1, cur.player_position[1]] == ItemType.LADDER_DOWN.value) | ... ) (Req: ApproachLadderDownLevel2) FaceWaterLevel2 Skill Type: Survivability Description: The agent, when adjacent to a water pool in the Gnomish Mines level (level 2), rotates to face the water block. Assigned Reward: 1 Success Condition: (cur.player_level == 2) & (cur.map[2, cur.player_position[0] + DIRECTIONS[cur.player_direction][0], cur.player_position[1] + DIRECTIONS[cur.player_direction][1]] == BlockType.WATER.value) Starting Conditions: • C1: (cur.player_level == 2) & ((cur.map[2, cur.player_position[0]+1, cur.player_position[1]] == BlockType.WATER.value) | ... ) (Req: ApproachWaterLevel2) ApproachUpLadderLevel2 Skill Type: Navigation Description: The agent moves from proximity to the upward ladder on the Gnomish Mines level (level 2) to a position directly adjacent. Assigned Reward: 1 Success Condition: (cur.player_level == 2) & ((cur.item_map[2, cur.player_position[0]+1, cur.player_position[1]] == ItemType.LADDER_UP.value) | ... ) Starting Conditions: • C1: (cur.player_level == 2) & exists_ladder_object(cur, ItemType.LADDER_UP.value) (Req: FindUpLadderLevel2) MoveForwardAfterTorchLevel2 Skill Type: Navigation Description: After placing a torch at the edge of darkness, the agent moves forward one block into the newly illuminated area. Assigned Reward: 1 Success Condition: (cur.player_level == 2) & (cur.light_map[2, cur.player_position[0], cur.player_position[1]] >= 0.5) Starting Conditions: • C1: (cur.player_level == 2) & (cur.light_map[2, cur.player_position[0], cur.player_position[1]] < 0.5) (Req: PlaceTorchAtEdgeLevel2) RecoverManaSafe Skill Type: Survivability Description: The agent ensures survival intrinsics are fully replenished via SleepToFullEnergy, then actively verifies and eliminates threats before recovering mana. Assigned Reward: 1 Success Condition: (cur.player_level == 0) & (cur.player_mana >= (6 + cur.player_intelligence)) Starting Conditions: • C1: (cur.player_level == 0) & (cur.player_energy >= (7 + 2 * cur.player_dexterity)) (Req: SleepToFullEnergy) • C2: not is_near_melee_mob(cur, MeleeMobType.ZOMBIE.value, 3) (Req: KillZombie) • C3: not is_near_ranged_mob(cur, RangedMobType.SKELETON.value, 3) (Req: KillSkeleton) StepOntoUpLadderLevel2 Skill Type: Navigation Description: The agent moves from an adjacent position to the upward ladder on the Gnomish Mines level (level 2) to stand exactly on the ladder block. Assigned Reward: 1 Success Condition: (cur.player_level == 2) & (cur.item_map[2, cur.player_position[0], cur.player_position[1]] == ItemType.LADDER_UP.value) Starting Conditions: • C1: (cur.player_level == 2) & ((cur.item_map[2, cur.player_position[0]+1, cur.player_position[1]] == ItemType.LADDER_UP.value) | ... ) (Req: ApproachUpLadderLevel2) FaceUpLadderLevel2 Skill Type: Navigation Description: The agent positions itself adjacent to the upward ladder on the Gnomish Mines level (level 2) and rotates to face the ladder block. Assigned Reward: 1 Success Condition: (cur.player_level == 2) & (cur.item_map[2, cur.player_position[0] + DIRECTIONS[cur.player_direction][0], cur.player_position[1] + DIRECTIONS[cur.player_direction][1]] == ItemType.LADDER_UP.value) Starting Conditions: • C1: (cur.player_level == 2) & ((cur.item_map[2, cur.player_position[0]+1, cur.player_position[1]] == ItemType.LADDER_UP.value) | ... ) (Req: ApproachUpLadderLevel2) AscendLadderLevel2 Skill Type: Navigation Description: The agent executes the ascend action while positioned on the upward ladder in the Gnomish Mines to transition back to the dungeon level. Assigned Reward: 1 Success Condition: cur.player_level == 1 Starting Conditions: • C1: (cur.player_level == 2) & (cur.item_map[2, cur.player_position[0], cur.player_position[1]] == ItemType.LADDER_UP.value) (Req: StepOntoUpLadderLevel2) ApproachFountainLevel1 Skill Type: Navigation Description: The agent moves from proximity (within 2 blocks) to a fountain in the dungeon level to an adjacent position (1 block away). Assigned Reward: 1 Success Condition: (cur.player_level == 1) & ((cur.map[1, cur.player_position[0]+1, cur.player_position[1]] == BlockType.FOUNTAIN.value) | ... ) Starting Conditions: • C1: (cur.player_level == 1) & is_near_block(cur, BlockType.FOUNTAIN.value) (Req: FindFountainLevel1) StepOntoLadderDownLevel2 Skill Type: Navigation Description: The agent moves forward from an adjacent position while facing the downward ladder on the Gnomish Mines level to stand exactly on the ladder block. Assigned Reward: 1 Success Condition: (cur.player_level == 2) & (cur.item_map[2, cur.player_position[0], cur.player_position[1]] == ItemType.LADDER_DOWN.value) Starting Conditions: • C1: (cur.player_level == 2) & (cur.item_map[2, cur.player_position[0] + DIRECTIONS[cur.player_direction][0], cur.player_position[1] + DIRECTIONS[cur.player_direction][1]] == ItemType.LADDER_DOWN.value) (Req: FaceLadderDownLevel2) DrinkWaterLevel2Improved Skill Type: Survivability Description: The agent, after precisely approaching and correctly facing a water pool in the Gnomish Mines, drinks from it to replenish thirst. Assigned Reward: 1 Success Condition: (cur.player_level == 2) & (cur.player_drink >= (7 + 2 * cur.player_dexterity)) Starting Conditions: • C1: (cur.player_level == 2) & ((cur.map[2, cur.player_position[0]+1, cur.player_position[1]] == BlockType.WATER.value) | ... ) (Req: ApproachWaterLevel2) • C2: (cur.player_level == 2) & (cur.map[2, cur.player_position[0] + DIRECTIONS[cur.player_direction][0], cur.player_position[1] + DIRECTIONS[cur.player_direction][1]] == BlockType.WATER.value) (Req: FaceWaterLevel2) MoveAwayFromWaterLevel2 Skill Type: Navigation Description: The agent moves from an adjacent position to a water pool to a location where it is no longer adjacent, enabling exploration of surrounding areas. Assigned Reward: 1 Success Condition: (cur.player_level == 2) & not ((cur.map[2, cur.player_position[0]+1, cur.player_position[1]] == BlockType.WATER.value) | ... ) Starting Conditions: • C1: (cur.player_level == 2) & ((cur.map[2, cur.player_position[0]+1, cur.player_position[1]] == BlockType.WATER.value) | ... ) (Req: ApproachWaterLevel2) KillGnomeWarriorInLitAreaLevel2 Skill Type: Fighting mobs Description: The agent, while positioned in a fully illuminated area on the Gnomish Mines level, locates and eliminates a Gnome Warrior. Assigned Reward: 1 Success Condition: (cur.player_level == 2) & was_mob_killed(prev, cur, MobType.MELEE.value, MeleeMobType.GNOME_WARRIOR.value) Starting Conditions: • C1: cur.player_level == 2 (Req: DescendLadderLevel1) • C2: cur.inventory.sword >= 1 (Req: CraftWoodSword) • C3: (cur.player_level == 2) & (cur.light_map[2, cur.player_position[0], cur.player_position[1]] >= 0.5) (Req: MoveIntoLitAreaLevel2) • C4: is_near_melee_mob(cur, MeleeMobType.GNOME_WARRIOR.value, 2) (Req: FindStalagmiteLevel2) KillGnomeArcherWithSwordLevel2 Skill Type: Fighting mobs Description: The agent leverages its sword and light management to locate and defeat a Gnome Archer in melee range within illuminated areas. Assigned Reward: 1 Success Condition: (cur.player_level == 2) & was_mob_killed(prev, cur, MobType.RANGED.value, RangedMobType.GNOME_ARCHER.value) Starting Conditions: • C1: cur.player_level == 2 (Req: DescendLadderLevel1) • C2: cur.inventory.sword >= 1 (Req: CraftWoodSword) • C3: (cur.player_level == 2) & (cur.light_map[2, cur.player_position[0], cur.player_position[1]] >= 0.5) (Req: MoveIntoLitAreaLevel2) • C4: is_near_ranged_mob(cur, RangedMobType.GNOME_ARCHER.value, 2) (Req: FindStalagmiteLevel2) DescendLadderLevel2 Skill Type: Navigation Description: The agent executes the descend action while positioned on the downward ladder in the Gnomish Mines level to transition to the Sewers (level 3). Assigned Reward: 1 Success Condition: cur.player_level == 3 Starting Conditions: • C1: cur.player_level == 2 (Req: DescendLadderLevel1) • C2: (cur.player_level == 2) & (cur.monsters_killed[2] >= 8) (Req: KillEightMobsLevel2) • C3: (cur.player_level == 2) & (cur.item_map[2, cur.player_position[0], cur.player_position[1]] == ItemType.LADDER_DOWN.value) (Req: StepOntoLadderDownLevel2)