DeltaEvolve: Accelerating Scientific Discovery through Momentum-Driven Evolution
Jiachen Jiang 1 Tianyu Ding 2 Zhihui Zhu 1 · 2026
Read DeltaEvolve: Accelerating Scientific Discovery through Momentum-Driven Evolution by Jiachen Jiang 1 Tianyu Ding 2 Zhihui Zhu 1 (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2602-02919.txt.
Introduction
Automated science discovery drives a 2026 paper by Jiachen Jiang and 2 co-authors. Language-model evolutionary systems propose and improve candidate programs over many rounds. Earlier systems fed the full code history back as context. That wastes space and buries the key ideas behind routine implementation details.
Their framework, DeltaEvolve, stores structured semantic deltas instead. A delta records how and why a change between versions affected performance. Deltas are organized in a multi-level database with progressive disclosure, cutting input tokens further. The authors frame evolution as an expectation-maximization process. Tests across scientific domains followed. DeltaEvolve found better solutions than full-code systems while using fewer tokens for context.
Catalog record
| CREATOR | Jiachen Jiang 1 Tianyu Ding 2 Zhihui Zhu 1 |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 78 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2602-02919
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
DeltaEvolve: Accelerating Scientific Discovery through Momentum-Driven Evolution Jiachen Jiang 1 Tianyu Ding 2 Zhihui Zhu 1 Abstract LLM–driven evolutionary systems have shown promise for automated science discovery, yet existing approaches such as AlphaEvolve rely on full-code histories that are context-inefficient and potentially provide weak evolutionary guidance. In this work, we first formalize the evolutionary agents as a general Expectation–Maximization framework, where the language model samples candidate programs (E-step) and the system updates the control context based on evaluation feedback (M-step). Under this view, constructing context via full-code snapshots constitutes a suboptimal M-step, as redundant implement details dilutes core algorithmic ideas, making it difficult to provide clear inspirations for evolution. To address this, we propose 𝙳𝚎𝚕𝚝𝚊𝙴𝚟𝚘𝚕𝚟𝚎 \mathop{\mathtt{DeltaEvolve}}\limits , a momentum-driven evolutionary framework that replaces full-code history with structured 𝚜𝚎𝚖𝚊𝚗𝚝𝚒𝚌 𝚍𝚎𝚕𝚝𝚊 \mathop{\mathtt{semantic\ delta}}\limits capturing how and why modifications between successive nodes affect performance. As programs are often decomposable, 𝚜𝚎𝚖𝚊𝚗𝚝𝚒𝚌 𝚍𝚎𝚕𝚝𝚊 \mathop{\mathtt{semantic\ delta}}\limits usually contains many effective components which are transferable and more informative to drive improvement. By organizing 𝚜𝚎𝚖𝚊𝚗𝚝𝚒𝚌 𝚍𝚎𝚕𝚝𝚊 \mathop{\mathtt{semantic\ delta}}\limits through multi-level database and progressive disclosure mechanism, input tokens are further reduced. Empirical evaluations on tasks across diverse scientific domains show that our framework can discover better solution with less token consumption over full-code-based evolutionary agents. 1 Introduction LLMs have facilitated automated science discovery across diverse domains such as mathematical optimization ( 8 ; 10 ; 25 ) , physical systems ( 13 ) , and molecular discovery ( 3 ) . Although different tasks, the common core objective is to discover a high-performing object that satisfy desirable quantitative properties. Fundamentally, all these tasks can be represented as the search problem: solutions are difficult to synthesize directly but computationally easy to evaluate. Recent works ( 18 ; 14 ) mainly choose to search in the code space, since code is sufficiently expressive to represent complex algorithms, aligns well with LLM pretraining, and enables automatic evaluation through execution. For simple problems, single-turn generation conditioned on problem specifications can already be effective. However, complex tasks typically require interaction with execution environments and iterative refinement based on feedback. This has led to self-evolving agents such as FunSearch ( 20 ) and AlphaEvolve ( 15 ) , which iteratively generate programs, execute them, analyze feedback signals, and produce improved variants. These approaches can be interpreted as a form of test-time scaling, instead of relying solely on internal reasoning tokens, the performance improves by allocating more computation to accumulating history and feedback across iterations. Despite their success, existing evolving agents face two fundamental limitations: • Limited context window. Extensive multi-turn inference incurs prohibitive computational cost and latency, preventing effective utilization of the full evolution history ( 4 ) . As a result, methods such as AlphaEvolve ( 15 ) retain only a small subset of high-quality solutions (e.g., top-performing or diverse programs). However, for complex systems, individual programs are already long, leaving limited capacity to reuse historical information. • Insufficient evolutionary guidance. Complete programs often entangles core algorithmic ideas with extensive implementation and control logic which are irrelevant to the underlying strategy, making it hard for LLMs to isolate the truly useful components from vast code histories. Therefore, full codes would not explicitly capture transferable patterns of success or failure that could guide future iterations, causing missed successful patterns or repeated failures. This motivates our research question: how can we provide stronger evolutionary guidance while staying within a limited context budget? To rigorously address this, we first formalize the evolutionary agents within a general Expectation–Maximization (EM) framework. Optimization proceeds by alternating two steps: in the E-step, the language model samples candidate programs conditioned on the current context; in the M-step, the system updates the control context using evaluation feedback to maximize the objective. From this perspective, the standard practice of representing history via static full-code snapshots constitutes a suboptimal M-step: while it captures the current state, it obscures which changes led to performance gains or losses, weakening the learning signal. Figure 1 : Illustration of search dynamics under existing methods and \mathop{\mathtt{DeltaEvolve}}\limits . Existing methods store and reuse full programs, whereas \mathop{\mathtt{DeltaEvolve}}\limits stores \mathop{\mathtt{semantic\ delta}}\limits that capture what changed and why it worked, forming a momentum-like memory that provides more informative guidance for reuse. We hypothesize that within a fixed task, the modifications that affect performance in earlier iterations often capture task-level structures and may remain informative for guiding updates to different programs in later iterations. These semantic changes are more informative for guiding future updates than the static solutions themselves. Based on this hypothesis, we propose \mathop{\mathtt{DeltaEvolve}}\limits , a momentum-driven evolutionary framework designed to optimize the M-step. As illustrated in Figure 1 , instead of storing full-code snapshots in the memory bank, we propose to use \mathop{\mathtt{semantic\ delta}}\limits to record the core semantic modifications from a parent node to its offspring and their resulting impact on performance. The memory bank stores \mathop{\mathtt{semantic\ delta}}\limits between pairs of nodes. As these deltas capture what changed across successive iterations and guide future updates, they form a directional signal analogous to the momentum term in optimization. To implement \mathop{\mathtt{semantic\ delta}}\limits efficiently, we augment each node with additional structures: a delta summary (Level-1) and delta plan details (Level-2), alongside the existing full code (Level 3), forming a pyramid-structured Multi-Level Database . Then the Progressive Disclosure Sampler dynamically decides the level of detail to expose for historical nodes based on relevance and recency: the current parent node is presented as full code for editing, while historical inspiration nodes are compressed into delta summaries or delta plan details. This approach exposes the logic of past improvements without wasting tokens on their implementation details. Figure 2 : Comparison between AlphaEvolve and \mathop{\mathtt{DeltaEvolve}}\limits (ours) on black-box optimization over 100 iterations. The x-axis shows cumulative consumed tokens across all iterations, and y-axis shows the best achieved score. Point size indicates the number of top and diverse nodes included in the context. \mathop{\mathtt{DeltaEvolve}}\limits consistently achieves higher scores with fewer tokens. Our evaluation prioritize two critical metrics: solution quality and token consumption. As illustrated in Figure 2 , on the black-box optimization task, \mathop{\mathtt{DeltaEvolve}}\limits consistently achieves higher scores with significantly fewer input tokens across varying context hyperparameters (e.g., number of top and diverse programs). To ensure generality and robustness, we further validate our framework across five diverse domains. Extensive experiments confirm that \mathop{\mathtt{DeltaEvolve}}\limits discovers solutions of comparable or superior quality to state-of-the-art baselines, all while reducing total token consumption by approximately 36.79% on average. Overall, our work makes the following contributions: • General Evolutionary Framework. We formalize the evolution as an Expectation–Maximization (EM) process, identifying updating the context (the M-Step) as the critical bottleneck in existing systems. • DeltaEvolve. We propose a momentum-driven framework that replaces static full-code snapshots with \mathop{\mathtt{semantic\ delta}}\limits . This approach utilizes a progressive disclosure mechanism to maximize context utility. • Empirical Efficiency : Through comprehensive evaluations across five scientific domains, we demonstrate that \mathop{\mathtt{DeltaEvolve}}\limits achieves superior solution quality while reducing token consumption compared to state-of-the-art full-code baselines. 2 Related Works Evolutionary Coding Agents. The integration of LLMs with evolutionary algorithms has produced powerful frameworks like AlphaEvolve ( 15 ) and OpenEvolve ( 21 ) , which established the core viability of LLM-driven program evolution. Subsequent works have significantly