PokeRL: Reinforcement Learning for Pokémon Red

Dheeraj Reddy Mudireddy, Sai Patibandla · 2026

Read PokeRL: Reinforcement Learning for Pokémon Red by Dheeraj Reddy Mudireddy, Sai Patibandla (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2604-10812.txt.

Introduction

Mudireddy and Patibandla present PokeRL, training reinforcement learning agents for early Pokemon Red tasks. Pokemon Red offers sparse rewards, partial observability, and quirky controls . Recent PPO agents clear two gyms with heavy reward shaping, yet training stays brittle, with agents falling into loops, menu spam, or wandering. PokeRL contributes a loop-aware PyBoy wrapper with map masking, anti-loop and anti-spam layers, and a dense hierarchical reward design. The authors argue modeling failure modes like loops and spam is a necessary step toward full Pokemon League champion agents.

Catalog record

CREATORDheeraj Reddy Mudireddy, Sai Patibandla
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE25146
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2604.10812

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

PokeRL: Reinforcement Learning for Pokémon Red Dheeraj Reddy Mudireddy, Sai Patibandla Abstract Pokémon Red is a long-horizon JRPG with sparse rewards, partial observability, and quirky control mechanics that make it a challenging benchmark for reinforcement learning. While recent work has shown that PPO agents can clear the first two gyms using heavy reward shaping and engineered observations, training remains brittle in practice, with agents often degenerating into action loops, menu spam, or unproductive wandering. In this paper we present PokeRL, a modular system that trains deep reinforcement learning agents to complete early-game tasks in Pokémon Red, including exiting the player’s house, exploring Pallet Town to reach tall grass, and winning the first rival battle. Our main contributions are a loop-aware environment wrapper around the PyBoy emulator with map masking, a multi-layer anti-loop and anti-spam mechanism, and a dense hierarchical reward design. We argue that practical systems like PokeRL, which explicitly model failure modes such as loops and spam, are a necessary intermediate step between toy benchmarks and full “Pokémon League champion” agents. Code is available at https://github.com/reddheeraj/PokemonRL . I Introduction Classic game environments have been central to the progress of deep reinforcement learning, from Atari and Go to NetHack and Minecraft. These domains combine high-dimensional observations with long time horizons and have repeatedly revealed the limitations of standard RL algorithms in the presence of sparse rewards and deceptive local optima. Pokémon Red occupies a particularly interesting point in this landscape. Unlike reflex-oriented arcade games, it is a story-driven JRPG where progress depends on coordinated navigation, interaction, and turn-based combat, executed over tens of thousands of timesteps. The player must leave their house, traverse Pallet Town and nearby routes, collect a starter Pokémon, engage in wild battles to gain experience, and defeat scripted opponents. From an RL perspective, this yields: • Extreme rewards assignment : Key rewards such as catching a Pokémon or winning a battle may occur only after long sequences of actions. • High branching factor : Even in the opening minutes there are many movement options and interaction opportunities. • Partial observability : The agent sees only the current 2D screen, while crucial variables (HP, map ID, party status) live in hidden memory. • Non-standard controls : Movement requires a double press (turn then step), and multiple menu buttons exist that are useless or harmful for learning. Recent work has shown that PPO agents can complete roughly the first fifth of Pokémon Red by using a custom Gym environment, dense reward shaping, and an additional “visited mask” observation channel [ 6 ] . However, reproducing these results is non-trivial. In our own attempts, we observed several recurring pathologies: agents exploiting shaping rewards by looping, spamming the A button or the no-op action, spinning in place because the environment mishandled double-press movement, and wandering within a small region without meaningful exploration. PokeRL is our attempt to build a more robust and transparent environment for early-game Pokémon Red. Instead of only tuning rewards, we explicitly encode loop detection, spam penalties, and spatial memory into the environment. We then structure training as a curriculum over three sequences: exiting the house, reaching tall grass from Pallet Town, and winning the first rival battle. The result is not a full game-solving agent, but a working system that reliably acquires non-trivial behaviors and surfaces many of the design trade-offs involved in turning “RL + emulator” into a practical research platform. II Motivation and Problem Setting II-A Why Pokemon Red? Pokémon Red is a natural testbed for long-horizon RL because it embeds several sub-tasks commonly studied in isolation: • Navigation and exploration in a 2D grid-like overworld. • Inventory and resource management through items and Pokémon party composition. • Turn-based combat with discrete actions, stochastic outcomes, and non-trivial strategy as shown in Figure 1(c) . (a) Red in his house (b) Exploring Pallet Town (c) First Battle Fig. 1 : Frames of selected objectives in the game. The opening segment of the game provides a compact yet challenging subset of these skills: 1. House exit : Learn basic movement and door interaction to leave Red’s bedroom and house. 2. Early exploration : Navigate Pallet Town and Route 1 to reach tall grass and trigger scripted encounters. 3. First rival battle : Execute viable battle actions to defeat the rival’s Pokémon given a fixed starter. Our project focuses on these three tasks because they are short enough to allow experimentation on a single device and rich enough to expose the core issues of loop, sparse rewards, and bad exploration. II-B Challenges in Vanilla RL Setup Preliminary experiments with a simple PPO agent wrapped around PyBoy highlighted five concrete failure modes: • Infinite action loops : The agent discovered local cycles (for example, pacing between two tiles) that gave small positive movement rewards without progress. • Reward sparsity and instability : When only rare events were rewarded, episode returns fluctuated wildly, and policies failed to improve. • Button spam : Action distributions collapsed onto A, Start, or no-op, producing pointless menus or idling. • Incorrect movement semantics : Treating a single direction press as “move one tile” caused the agent to spin in place instead of walking. • Exploration without memory : Without an explicit notion of visited states the agent repeatedly revisited already explored areas. PokeRL is designed as an answer to these five issues. III Related Work Deep RL in video games began with agents that achieved human-level performance on Atari games by learning directly from pixels with score-based rewards. Games such as Montezuma’s Revenge exposed the limitations of this approach: rewards were extremely sparse, and naive exploration strategies failed to reach them. Work on intrinsic motivation and Go-Explore [ 2 ] addressed this by explicitly rewarding novelty and maintaining state memories to revisit promising situations. Until recently, little academic literature existed on applying RL to Pokémon. For Pokémon specifically, Pleines et al [ 5 ] introduced the first comprehensive deep RL agent for Pokémon Red, using PPO [ 9 ] over a PyBoy-based Gym environment. They engineered a reward function with more than 25 components and augmented the observation with a 72×80 visited mask channel, enabling an agent to progress up to the second gym in Cerulean City, beating two major gym leaders in pokemon battles . This was a milestone achievement: as of February 2025, an RL policy (with < 10 million parameters) could beat Pokémon Red’s early game to an extent with only a few simplifications to the game logic. They showed that certain reward bonuses could backfire, as agents learned to exploit them in unintended ways. Their ablation studies showed both the necessity and risk of dense shaping: certain rewards encouraged grinding or skipping mandatory events when tuned poorly. Parallel community projects have applied RL to other Pokémon titles, such as Pokémon Gold, and constructed tools like poke-env [ 7 ] , which interfaces RL algorithms with the Pokémon Showdown battle simulator. These battle-only settings provide cleaner reward signals (win or loss) and smaller action spaces, and have been pushed to near human-parity using offline RL and large language model policies. For example, PokeLLMon (2024) [ 3 ] integrated a large language model as the decision-making policy and reached human-parity in

Read the complete text