GEAR : Genetic AutoResearch for Agentic Code Evolution

Ahmadreza Jeddi, Minh Ngoc Le, Hakki C. Karaimer, Konstantinos G. Derpanis, Babak Taati · 2026

Read GEAR : Genetic AutoResearch for Agentic Code Evolution by Ahmadreza Jeddi, Minh Ngoc Le, Hakki C. Karaimer, Konstantinos G. Derpanis, Babak Taati (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-13874.txt.

Introduction

Ahmadreza Jeddi and four colleagues introduce GEAR, gene auto-study, swapping single-path search over one program with crowd-based search over many study states. GEAR keeps a set of strong trial answers, selects parents by yield, novelty and spread, and tries new ideas through tweaks and mixing. Under the same compute budget and setup, all three versions beat the auto-study rival. While the rival settles into one local optimum, GEAR keeps finding gains over longer runs.

Catalog record

CREATORAhmadreza Jeddi, Minh Ngoc Le, Hakki C. Karaimer, Konstantinos G. Derpanis, Babak Taati
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE35092
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2605.13874

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

GEAR : Genetic AutoResearch for Agentic Code Evolution Ahmadreza Jeddi, Minh Ngoc Le, Hakki C. Karaimer, Konstantinos G. Derpanis, Babak Taati Abstract Autonomous research agents such as AutoResearch can now propose, run, and commit machine learning experiments without human supervision. However, their outer-loop search is largely single-incumbent hill climbing : the agent edits one program, evaluates it, and retains the result only if it improves over the current best. We argue that this strategy prematurely discards valuable search signal, including complementary local optima, partially successful ideas, and accumulated insights from diverse research directions. We introduce GEAR (GEnetic AutoResearch), a drop-in search controller that replaces single-incumbent hill climbing with a population-based frontier search over research states. GEAR maintains a bounded population of elite nodes, selects parents using a composite score that balances estimated productivity, local novelty, and global coverage, and expands the frontier through mutation and crossover. Each node in the search graph preserves code changes, reflections, and performance statistics that inform future expansion decisions. We study three variants: (i) GEAR-Prompt , where the LLM agent manages population dynamics through natural-language instructions alone; (ii) GEAR-Fixed , which externalizes the genetic search policy into a fixed programmatic controller; and (iii) GEAR-Evolve , which treats the controller itself as a mutable artifact and explicitly decides at each iteration whether to run an experiment or modify the search policy. Under identical environments and compute budgets, all three GEAR variants outperform the AutoResearch baseline and achieve lower validation bits-per-byte. Beyond final performance, GEAR exhibits a distinct search dynamic: while the baseline quickly converges to a single local optimum, GEAR variants continue discovering improvements over longer horizons. These results suggest that equipping autonomous research agents with explicit population structure and mutable search policies can meaningfully extend their capacity for sustained progress. Project page: https://genetic-autoresearch.github.io/ † † footnotetext: * Equal contribution. 1 Introduction Figure 1 : Genetic AutoResearch (GEAR) variants all surpass the AutoResearch baseline’s plateau after 100 experiments. Light grey dots are discarded experiments. Bold dots mark new global bests. GEAR-Fixed and GEAR-Prompt converge to similar quality, while GEAR-Evolve pulls ahead from roughly experiment 25 onward and eventually reaches the lowest bpb of the four. Automating machine learning research is shifting from simple code generation toward long-horizon experimental systems that can modify code, run training jobs, inspect outcomes, and decide what to try next ( Lu et al. 2024 ; Yamada et al. 2025 ; Toledo et al. 2025 ; Chen et al. 2026a ; Qu and Lu 2026 ; Borthwick et al. 2026 ) . Among recent systems, AutoResearch ( Karpathy 2026 ) is a particularly simple and functional implementation of this idea, with a single LLM agent iteratively editing a training script, running under a fixed budget, and retaining changes that improve validation bits-per-byte (bpb) on a single GPU. Running Autoresearch without any human intervention for an extended period of time yields numerous additive improvements ( Karpathy 2026 ) . The simplicity of this design, however, reveals a structural limitation. The standard AutoResearch loop is effectively single-incumbent hill climbing : there is one active artifact, one latest improvement, and a soft memory of prior attempts encoded only in logs or conversational context. This is often sufficient for local progress, but it is a weak representation of research as a search process. Real research typically spans multiple directions of investigation. One direction may be strongest on raw performance, another may be leaner or more efficient, and a third might be temporarily uncompetitive yet contain an idea worth revisiting. A single incumbent cannot represent these alternatives explicitly, and the ideas they embody are lost once discarded. To this end, we propose GEAR (Genetic AutoResearch), a search controller designed as a drop-in replacement for the single-incumbent loop in AutoResearch-style systems. GEAR preserves the original training environment and scalar objective but replaces keep-or-discard hill climbing with a search graph and a bounded frontier of best search nodes. Each node stores the code change and results, alongside its parentage, mutation type, and other descriptive metadata about how productive it has been as a source of future children, what it has tried and learned, and search statistics such as expansion count and improvement. The search graph is expanded through artifact mutation (proposing a code change from a single parent) and semantic crossover (synthesizing a child that combines complementary ideas from two parent nodes). We study three variants of increasing structure: (i) GEAR-Prompt , where the LLM agent manages population dynamics through natural language instructions alone, (ii) GEAR-Fixed , which externalizes the search policy into a fixed programmatic controller that implements parent selection, promotion, and bookkeeping, and (iii) GEAR-Evolve , which additionally permits the agent to modify the controller code under logging and reproducibility constraints. We evaluate all three variants on the same language modeling task used by AutoResearch and under identical compute budgets. All GEAR variants consistently outperform the AutoResearch baseline. More importantly, they exhibit a qualitatively different search dynamic: while the baseline quickly exhausts its useful search directions and settles into a local optimum, GEAR continues to discover improvements over longer horizons. This sustained progress reflects the value of maintaining a frontier of diverse elite states rather than committing to a single incumbent. By preserving partially successful ideas, revisiting complementary branches, and recombining useful changes, GEAR avoids the premature convergence that limits hill-climbing agents. Our main contributions are as follows: 1. GEAR , a drop-in genetic search policy that replaces single-incumbent hill climbing with a bounded frontier of elite research states expanded through mutation and crossover, requiring no changes to the underlying training environment or evaluation harness (§ 3.2 ). 2. A systematic dissection of genetic search mechanisms in autonomous research agents, showing how mutation enables diverse local exploration while crossover composes complementary discoveries across branches. We instantiate this policy in three forms: natural-language prompting, a fixed programmatic controller, and a self-modifying controller that evolves its own search policy (§ 3.3 ). 3. Empirical evidence that population-based frontier search extends the horizon of autonomous discovery. While AutoResearch rapidly converges to a local optimum, GEAR variants continue making progress throughout the search budget, with the evolved controller producing the strongest sustained improvement (§ 4 ). 2 Related Work 2.1 Autonomous and Long-Horizon ML Research Agents Recent work on autonomous research has progressed from code assistance and paper drafting toward systems that execute substantial portions of the research loop end to end. Systems such as AI Scientist Lu et al. 2024 ; Yamada et al. 2025 , Agent Laboratory Schmidgall et al. 2025 , AI-Researcher Tang et al. 2025 , InternAgent Feng et al. 2026 , AI co-scientist Gottweis et al. 2025 , and EvoScientist Lyu et al. 2026 study increasingly complete pipelines spanning literature review, hypothesis generation, code writing, experimentation, analysis, and manuscript preparation. A closely related thread focuses more specifically on machine learning engineering and long-horizon search in code space. AIDE Jiang et al. 2025 formulates MLE as tree search over candidate code solutions, while methods like MLE-STAR Nam et al. 2025 and ML-Master Liu et al. 2025 ; Zhu et al. 2026 augment this with memory, retrieval and targeted component-wise refinement. AIRA Toledo et al. 2025 makes the search-policy view explicit by studying how operator design and search

Read the complete text