Knowing in Advance When an Evolutionary Outer Loop Will Not Help: A Pre-Registered Cheap-Baseline Screening Rule

Ramchand Kumaresan · 2026

Read Knowing in Advance When an Evolutionary Outer Loop Will Not Help: A Pre-Registered Cheap-Baseline Screening Rule by Ramchand Kumaresan (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2606-29119.txt.

Introduction

Kumaresan offers a cheap screening rule for deciding whether an evolutionary outer loop is worth building for a neural network project. Such loops, which evolve network weights or structure over generations, cost hundreds of times more than the gradient training inside them. Whether they beat a simple single-shot method is usually learned only after paying.

His rule is computed before any implementation, at a phase-zero gate, and was pre-registered, its threshold fixed in advance. It forms a recovery ratio from the best single-shot gain and the best gain of any rival method. If the ratio reaches ninety percent, skip the outer loop. In two cases from a within-lab series, the gate fired and the loops were abandoned. In one, the apparent win came from a static substrate change, not evolution. On one project the gate cost tens of GPU-hours and screened out hundreds more.

Catalog record

CREATORRamchand Kumaresan
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE27 KB
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2606.29119

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Knowing in Advance When an Evolutionary Outer Loop Will Not Help: A Pre-Registered Cheap-Baseline Screening Rule Ramchand Kumaresan Abstract We introduce a pre-registered screening rule that decides, before any implementation , whether an evolutionary / population / lifecycle outer loop over neural-network parameters or structure is worth building. Such outer loops cost 10 2 10^{2} – 10 3 × 10^{3}\times their gradient inner loop, yet whether they beat a cheap single-shot alternative is usually discovered only after the expense is paid. Our rule computes, at a Phase-0 gate, a single number—the recovery R = s / G R=s/G , the best single-shot gradient/curvature statistic’s gain s s divided by the best gain G G of any cheap method evaluated—and prescribes skipping the outer loop when R ≥ 90 % R\geq 90\% . We validate the rule on a within-lab series of pre-registered outer-loop bets (two analyzed cases plus a disclosed file drawer): in both analyzed cases a static or single-shot computation captured the effect on the project’s own metric, the gate fired ( R ≈ 1.0 R\approx 1.0 in both cases; ≈ 0.95 {\approx}\,0.95 under a stricter metric on one), and the outer loop was abandoned—including one case where a companion factorial decomposition localizes the apparent win to a static substrate change with the evolutionary lifecycle contributing no detectable gain. On one project the gate cost ∼ 50 {\sim}50 – 70 70 GPU-hours and screened out an estimated ∼ 400 + {\sim}400{+} GPU-hours (first cell only) plus weeks of implementation—a ∼ 6 {\sim}6 – 8 × 8\times saving. The rule is prospectively falsifiable: a task with R < 90 % R<90\% where the outer loop still fails to beat single-shot would refute it. 1 Introduction Evolutionary, population-based, and lifecycle outer loops that search over network parameters, masks, routes, or structure are an attractive idea: they can optimize objectives a gradient inner loop cannot directly serve. They are also expensive—commonly 10^{2} – 10^{3}\times the cost of the inner loop they wrap. The practical question a lab faces is therefore not “ can an outer loop help?” but “ how do you know, cheaply and in advance, whether it will? ” This paper answers that question with a pre-registered discipline and reports the outcome of applying it to our own work. The contribution is methodological , not a new algorithm; the individual negative results the discipline produced are, on their own, unsurprising (see Section 2 ). What is new is the packaged, evidenced, prospectively-falsifiable decision procedure . Contributions. • A Cheap-Baseline-Falsification gate (Section 3 ): a pre-registered Phase-0 protocol that runs the cheapest plausible non-mechanism explanations (random, single-shot heuristic, static/non-evolutionary) on the same metric the paper will use , before the outer loop is built. • An operational screening rule (Section 3 ): skip the outer loop when recovery R=s/G\geq 90\% , where s is the best single-shot gradient/curvature statistic’s gain and G is the best gain of any cheap method evaluated. The 90\% threshold is a conservative convention placed below the observed kills, and the rule is prospectively falsifiable. • Validation on a within-lab series (Sections 4 – 5 ): two analyzed cases, alongside a disclosed file drawer. In both analyzed cases the gate screened out the outer loop, including a case whose submitted companion factorial isolates the outer loop as the non-contributor. (We do not claim independence—Section 8 .) • A GPU-hours-saved ledger (Section 6 ) quantifying what the gate buys. We are explicit about what this paper is not : it is not a claim that evolutionary outer loops never help. Section 7 states the converse—the condition under which they should help—as a hypothesis with a named, pre-registered successor design whose first screen is now reported; no claim here depends on its unrun branches. 2 Related Work and Positioning We position against three lines of work; none is contested by this paper, which is why our individual negatives are unsurprising and our contribution is the screening procedure rather than any one result. Evolution strategies on smooth objectives. ES is a competitive black-box optimizer when gradients are unavailable or the landscape is rugged ( 15 ; 19 ; 12 ) . On smooth, differentiable objectives the inner optimizer already serves, ES is known to be at best competitive and often worse: 25 show ES estimates the gradient of a Gaussian-smoothed objective, so when the per-minibatch gradient is already accessible its overhead is unwarranted, and 13 find ES competes with gradient-based search only on small problems. Our cases sit in this regime, and the screening rule operationalizes detecting it in advance. This is live terrain: ES is being actively re-proposed for LLM fine-tuning ( 16 ) and its failure modes actively studied ( 1 ; 7 ) —which is precisely why a pre-build gate is useful, since it tells you, before committing to an ES outer loop, whether a cheap baseline already recovers the frontier. Importance-based mask selection / pruning. Magnitude pruning ( 6 ) , iterative magnitude pruning with rewind ( 4 ; 5 ) , connection sensitivity ( 11 ) , and Fisher-information importance ( 8 ; 20 ) produce strong masks from single-shot or cheap iterative statistics. 18 go further: randomly reshuffling which weights are kept within each layer leaves final accuracy unchanged, so only the per-layer sparsity ratios carry information and the mask identity does not—the direct analog of our random- \kappa control. That these rival searched masks is established; our mask-identity result (Section 5.2 ) is a confirmation in a bilevel-constrained-fitness setting, not a new claim. Parameter-space constraints for ES on LLMs (concurrent). 17 study parameter-space constraints (magnitude anchoring) for ES on LLM fine-tuning—concurrent work in the same family, orthogonal to mask-identity selection. Our mask-identity baseline includes the generic anchored-weight-decay mechanism —a plain L_{2} pull of the adapter toward its warm checkpoint ( \gamma\sum\|W-W_{\text{warm}}\|^{2} ), which we ran as a baseline, not a reimplementation of their specific formulation; we used their reported range only as a starting estimate for the single strength hyperparameter \gamma . This generic anchored weight decay ties the best single-shot mask method within noise—a positioning point, not a contest. “Is the search worth it?” decision procedures. The closest formal relatives decide whether an expensive search is worthwhile rather than how to run it. 21 formalize a non-inferiority test and a “tuning risk”—the performance lost by leaving a hyperparameter at its default instead of tuning it—and find defaults are often non-inferior; 2 likewise recommend which hyperparameters are worth tuning. In neural architecture search, random search is a competitive baseline ( 14 ) , the search phase often fails to beat random selection ( 24 ) , and relative-improvement-over-random has been proposed as the only honest NAS metric ( 22 ) . The decisive difference is when the cheap baseline is consulted: in all of this work it is a post-hoc comparator used to ground or debunk a method after it is built. Ours is a pre-registered, pre-build go/no-go gate, with a committed recovery-ratio threshold, that decides whether to construct the outer loop at all. The closest pre-build intuition we found—running random search first to gauge a search space’s potential ( 9 ) —is stated as informal advice, with no recovery ratio, threshold, or

Read the complete text