OpenHA: A Series of Open-Source Hierarchical Agentic Models in Minecraft

Zihao Wang †, Muyao Li †, Kaichen He †, Xiangyu Wang, Zhancun Mu, Anji Liu, Yitao Liang · 2025

Read OpenHA: A Series of Open-Source Hierarchical Agentic Models in Minecraft by Zihao Wang †, Muyao Li †, Kaichen He †, Xiangyu Wang, Zhancun Mu, Anji Liu, Yitao Liang (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2509-13347.txt.

Introduction

Zihao Wang and 6 co-authors present OpenHA in a 2025 paper on hierarchical agents in Minecraft. Anyone building agents that learn end to end must choose an action space: the set of moves the agent can pick from. That choice matters a great deal, and nobody has settled it. Working in the open-ended world of Minecraft, the authors first compare the leading options on a large scale.

The comparison gives an awkward answer. No one action space wins everywhere. Which one works best depends heavily on the task. Their way out is called Chain of Action. An abstract action is not handed down as an order to a separate policy. Instead it serves as a middle step in the model's own reasoning, guiding the final action that can actually be carried out.

Catalog record

CREATORZihao Wang †, Muyao Li †, Kaichen He †, Xiangyu Wang, Zhancun Mu, Anji Liu, Yitao Liang
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE49 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2509-13347

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

OpenHA: A Series of Open-Source Hierarchical Agentic Models in Minecraft Zihao Wang †, Muyao Li †, Kaichen He †, Xiangyu Wang, Zhancun Mu, Anji Liu, Yitao Liang Abstract The choice of action representation is a critical yet unresolved challenge in developing capable, end-to-end trainable agents. This paper first presents a large-scale, systematic comparison of prominent action abstractions for Vision-Language-Action (VLA) or hierarchical agent models in the open-ended Minecraft. Our analysis reveals that no single action space is universally optimal; instead, the most effective abstraction is highly task-dependent, creating a dilemma for building generalist agents. To resolve this, we introduce Chain of Action (CoA), a novel framework that unifies high-level planning and low-level control within a single, monolithic VLA model. CoA treats an abstracted action not as a command for a separate policy, but as an intermediate reasoning step—akin to a chain of thought—that guides the generation of the final, executable action. Furthermore, we demonstrate that an All-in-One agent trained on a diverse mixture of action spaces using the CoA paradigm learns a more robust and generalizable policy. This unified agent achieves a new state-of-the-art, improving the overall task success rate over strong, specialized baselines. To foster reproducible research, we release the OpenHA (Open Hierarchical Agents) suite, which includes our comprehensive benchmark of over 800 distinct tasks, curated datasets, source code, and all pretrained model checkpoints at: https://github.com/CraftJarvis/OpenHA . \reportnumber 1 Introduction Autonomous agents, in their early designs, were predominantly constructed as modular systems leveraging pre-trained models, including Large Language Models (LLMs) ( Touvron et al. 2023 ; OpenAI 2023a ; Yang et al. 2025 ; Anthropic 2025 ) and Vision-Language Models (VLMs) ( Achiam et al. 2023 ; Wang et al. 2024b ; Team et al. 2023 ; Guo et al. 2025 ) . These agents often integrated a static foundation model with external modules—such as memory, planning, and tool use—to facilitate interaction within an environment ( Weng 2023 ; Yang et al. 2023 ; Wang et al. 2024e ; Cheng et al. 2024 ) . However, such modular pipelines cannot inherently undergo end-to-end training, resulting in a critical limitation: the agent’s performance is constrained by the capabilities of the pre-trained model ( Wang et al. 2024a ; Yu et al. 2025 ) . More importantly, the agent cannot learn and improve from its interactions with the environment ( Zhang et al. 2023 ; Li et al. 2025b ) . To address this constraint, recent advancements have turned toward the development of end-to-end pre-trained agent models that learn directly from interaction data ( Qin et al. 2025 ; openai 2025 ) . These models embed the capacity for action generation within the agent itself. Within this framework, two main methodologies for modeling an agent’s actions have emerged. The first, Vision-Language-Action (VLA) models ( Kim et al. 2024 ) , tokenize low-level actions—such as keyboard inputs or robotic movements—into sequences that are compatible with the vocabulary of language models. This enables the model to predict and generate low-level actions directly from the interaction context. However, a significant challenge arises from the semantic gap between the high-level abstractions expressed by language and the continuous, fine-grained nature of low-level actions. To overcome this issue, a second class of agent models adopts a hierarchical architecture ( Belkhale et al. 2024b ; Driess et al. 2023 ; Bjorck et al. 2025 ) . In these models, a high-level language model predicts an abstracted action, which is then decoded into a sequence of low-level actions by a learned action tokenizer ( Wang et al. 2024d ; Cai et al. 2024b ) . While this hierarchical approach separates reasoning and control, a key question arises: how can abstracted actions be effectively defined and formulated from the continuous stream of low-level experiences? Despite the widespread adoption of various action tokenizers ( Zhong et al. 2025a ) , a clear, standardized evaluation remains lacking. Most action tokenizers are tested on domain-specific benchmarks and proprietary datasets, making direct and fair comparisons difficult ( Belkhale et al. 2024a ; Gu et al. 2023 ; Kim et al. 2024 ) . This lack of standardization obscures a crucial question: which action representation is the most effective? To fill this gap, we introduce a unified benchmark for the open-ended environment of Minecraft ( Fan et al. 2022 ) , where we conduct a large-scale comparison of prominent abstracted action spaces across 1000 diverse tasks. Our empirical findings reveal that the effectiveness of an action tokenizer is highly task-dependent, with no single representation demonstrating universal superiority. Due to the traditional abstracted action being used only in hierarchical agent architecture, we further explored whether these abstracted actions are also beneficial for the VLA model. We introduce Chain-of-action (CoA) framework, which integrates the strengths of both hierarchical agents and VLA models. This observation motivates our primary contribution: the introduction of the Chain of Action (CoA) framework, which integrates the strengths of both hierarchical agents and VLA models. In CoA, action generation is structured as a two-step, auto-regressive process: first, the model generates a high-level abstracted action, which serves as an intermediate “thinking” or “reasoning”; then, conditioned on this thought, the model generates the final low-level action. For instance, to execute the command “chop a tree”, the CoA agent first predicts a grounding action such as Approach(object=<tree>, coordinate=[136, 287]) , followed by the sequence of low-level actions required to reach the position and perform the action. This approach effectively synergizes high-level reasoning with low-level control. In addition, inspired by the discovery that different abstracted actions excel in distinct task domains, we propose an All-in-One training strategy. The key insight is that, by grounding the model’s final output in low-level actions, CoA enables a unified prediction structure across all action spaces. This allows us to train a single agent on a composite dataset containing multiple action representations. The resulting agent can master a diverse range of actions, outperforming specialist models trained on a single type of action. The main contributions of this work are fourfold: • We provide a large-scale, systematic analysis demonstrating that the optimal abstracted action space is task-dependent, with different action representations excelling in distinct task domains. • We introduce the Chain of Action framework, which synergizes high-level reasoning and low-level control by using abstracted actions as intermediate plans. This novel approach bridges the gap between hierarchical agents and VLA models, offering superior performance compared to traditional dual-system architectures. • We demonstrate that training a single agent on mixed datasets of diverse action spaces enhances its ability to generalize across tasks, improving its overall decision-making capabilities. • We release a comprehensive suite of resources, including trained checkpoints, code, and datasets for various hierarchical agents, hierarchical VLA models and OpenHA model to support further research in action representation and generalization. 2 Abstracted Actions in Hierarchical Agents 2.1 Hierarchical Agent Structure Hierarchical agents model the agent’s decision-making process through a hierarchical policy decomposition. This process consists of two levels: first, a high-level pretrained LLM-based agentic model generates an abstracted action based on the task instruction and the current observation; second, a low-level policy acts as the action tokenizer and uses this abstracted action as guidance to produce a primitive, executable action in the environment. This two-level procedure

Read the complete text