Neuromorphic Control for 3D Navigation in Minecraft Using Genetic Algorithms

Eric Zipor · 2026

Read Neuromorphic Control for 3D Navigation in Minecraft Using Genetic Algorithms by Eric Zipor (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-02628.txt.

Introduction

Eric Zipor describes a student-style project that trains a controller for parkour movement in Minecraft, the block-based video game. Parkour play means crossing terrain with maximum efficiency, using frame-precise timing of sprinting, sneaking, and mouse direction to clear hard jumps. Human players treat the game's physics as a dynamic puzzle.

The project uses a genetic algorithm, a search method that breeds and selects candidate solutions, to set the weights of a neural network. The network reads inputs about block distances, terrain, and obstacles, and judges the best pathing choices. The abstract presents the design and its goal rather than finished results. It is a compact example of evolutionary training applied to a rich, three-dimensional game world.

Catalog record

CREATOREric Zipor
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE17 KB
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2605.02628

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Neuromorphic Control for 3D Navigation in Minecraft Using Genetic Algorithms Eric Zipor Abstract The popular 2009 voxel based videogame, Minecraft, contains several distinct disciplines. One of which is “parkour,” gameplay that focuses on traversing a world’s environment with maximum efficiency. The Minecraft online community has turned the game’s physics engine into dynamic puzzles, requiring players to masterfully manipulate motion mechanics through frame precise timing of keystrokes. Actions such as sprinting, sneaking, and mouse direction are all combined to clear specific difficult jumps. Through this project, we design a genetic algorithm to generate weights for a neural network to autonomously evaluate inputs for block distances, terrain, and obstacles to determine the most optimal pathing. I Introduction This project’s potential extends beyond the niche video-game scene. “Simulation-to-reality” (Sim2Real) is a growing trend in robotics, and although Minecraft is not a realistic movement engine, the fundamental concepts of trajectory, and spatial awareness are still present. By training a “brain” in a low stake virtual environment, we can develop upon robust navigation strategies that could eventually apply to real world unmanned systems such as drones and cars without the cost of material. Gamers infamously like to break the boundaries of what they once thought was impossible. Speedrunning is a sub-category that has captivated many audiences with pushing the limits on videogames to look for shortcuts and glitches that may benefit their world record run. Our approach however will not require the grinding and monotonous hours poured with the skill of a gamer. Instead, we are more akin to Tool-Assisted Speedrunning (TAS) and are interested in seeing the theoretical limits of perfection when the human factor is removed. For this project we will be using the Mineflayer API, a bot framework that enables automation and intelligence into the game. It’s simple Python bridge support (node.js module) allows bots to collect information and respond to their environment based on per in-game (IG) tick responses [6]. This sensor data will feed into a genetic algorithm, which will iteratively evolve the bot’s neural weights over thousands of generations to “breed” the best values for evaluating and clearing obstacles. Our goal is to take this one step further and apply an agent to a never before seen course and experiment it’s adaptiveness. To our knowledge, this has never been done before for Minecraft parkour. Pathing mods exist such as Baritone, however, they are deterministic A*-cartesian-style navigation, solving “get from A to B” with primitive traveling, not the desired optimization of momentum question we were asking [7]. We are also deliberately doing this in Mojang’s Minecraft: Java Edition rather than forking to a clone. Most prior work in evolutionary algorithims ports the problem to a simplified pygame or Unity environment so the physics can be sped up. We choose to train inside the real game engine, on real server ticks, against the actual IG physics. This choice later cost wall-clock training speed but gave a genuine Sim2Real experience. Fig. 1: Parallel generation of bots navigating a randomized string of blocks and gaps on a local server. Spawning on bedrock (left), bots jump across or fall into the void, with the end goal of landing on the red wool (right). II Setup To make the environment reproducible across runs we host a Vanilla Minecraft server, version 1.19.4, with no gameplay-altering mods. The server runs on a single laptop on a local area network, while bot clients connect from a separate desktop machine over the LAN. The world spawn is fixed to a coordinate and has a several in-game rules changes that would otherwise interfere with training (server.properties). Fall damage is turned off so a failed jump does not require a respawn animation, ‘doImmediateRespawn‘ is enabled, ‘randomTickSpeed‘ is set to zero so world elements do not change underneath the bots, and ‘sendCommandFeedback‘ is disabled to keep the server log clean. All bots are added to a single team with ‘collisionRule‘ toggled never and ‘friendlyFire‘ false so they do not push and are invisible to each other. Each bot receives the same set of sensor readings every server tick and emits the same four control outputs, which Mineflayer translates into keypresses (forward, jump, strafe-left, strafe-right) and a mouse direction (yaw delta). From the server’s perspective each bot is indistinguishable from a human player. The last piece of infrastructure in the world is a chain of command blocks in the game driven by entity markers tagged ‘block_pos‘ and ‘rng_source‘ that randomize the layout of the obstacles between generations. The description of that critical enhancement is referenced later in Section V, the driving mechanism for our random curriculum learning strategy. III Neural Network Design III-A Input Layer Our first input layer prototype encoded the bot’s surroundings as a voxel scan: a fixed 3D window of blocks around the bot, flattened into a one-hot vector. This gave the policy “perfect” information, however, the input layer would balloon past over a thousand neurons and the per-tick cost of querying every block from the Mineflayer world view dominated the digestion loop. The genetic algorithm needed to evaluate hundreds of policies per generation, so a single bot taking even milliseconds longer per tick could compound cost and error across a training run. To solve this, we replaced the voxel grid search with simple raycasting. Each ray is cast from the bot’s eye position out to a maximum distance of 10 blocks and returns a normalized hit value 1 - (hit_distance / max_distance), or zero if the ray misses completely. Fig. 2: Example of occlusion detection via voxel status using ray casting from the sensor (grey cube) to the measured point (red dot) [3]. Raycasting has two distinct advantages. First, the input vector shrinks from thousands of neurons to just 19, which is a small tractable search space for the genetic algorithm. Second, raycasting more closely mirrors what a human parkour player actually has access to which is a line-of-sight (LOS) read on the geometry directly in front of them, not an oracle map of every block existing. The bot, like the human player, cannot foresee obstacles outside its forward hemisphere. The agent has to commit to a trajectory and discover the consequences. The final input array consists of 33 neurons, 19 rays panning 180 degrees forward, seeking ground, ledge, and ceiling detection, the goal’s offset as an (X,Y,Z) vector from the bot’s respective frame; the bot’s velocity, if on-ground flag, the distance to the goal, and finally, an internal clock to encourage efficiency. III-B Hidden Layer During development, the model architecture remained a single-hidden-layer multilayer perceptron (MLP) with 32 neurons. Increasing the number of hidden neurons could help mitigate underfitting by providing additional learning capacity, however, excessive neurons may lead to overfitting, expanding the genetic algorithm search space, and increase per-tick Mineflayer API call cost. Ultimately, 32 hidden neurons was good enough to showcase expressivity during experiments. III-C Output Layer The output layer only has four necessary neurons! Three of the four outputs are binary, on or off actions: strafe left, strafe right, and jump (on a keyboard setup: buttons “A”,“D”, and “Space bar”). The fourth output is a continuous mouse yaw delta. We mix the activation functions on the output layer. Sigmoid squashes a real number into the

Read the complete text