TadA-Bench: A Million-Variant Benchmark for Future-Round Discovery Toward Agentic Protein Engineering
Jin Gao, Juntu Zhao, Zirui Zeng, Jiaqi Shen, Junhao Shi, Dukun Zhao, Yuming Lu, Dequan Wang · 2026
Read TadA-Bench: A Million-Variant Benchmark for Future-Round Discovery Toward Agentic Protein Engineering by Jin Gao, Juntu Zhao, Zirui Zeng, Jiaqi Shen, Junhao Shi, Dukun Zhao, Yuming Lu, Dequan Wang (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2606-02624.txt.
Introduction
Gao and co-authors build TadA-Bench, a benchmark for protein engineering that replays a real lab campaign. Agentic systems should pick future wet-lab experiments, not just fit old data. The bench replays 31 rounds of TadA directed evolution. It holds a million variants. Chronology is preserved. Models see earlier rounds and must rank variants that appear only later.
DNA, RNA, and protein views are aligned for each variant. A pipeline called Seq2Graph unifies noisy enrichment measures. It produces consistent activity labels across rounds. Random-split controls show strong interpolation skill. Future-round ranking is much weaker. Finite-budget candidate selection is weaker still. Controlled tests suggest evolutionary coverage beats local data density as a signal. Data and code are released on Hugging Face and GitHub. The bench gives agentic protein engineering a reproducible test bed.
Catalog record
| CREATOR | Jin Gao, Juntu Zhao, Zirui Zeng, Jiaqi Shen, Junhao Shi, Dukun Zhao, Yuming Lu, Dequan Wang |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 57 KB |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2606.02624
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
TadA-Bench: A Million-Variant Benchmark for Future-Round Discovery Toward Agentic Protein Engineering Jin Gao, Juntu Zhao, Zirui Zeng, Jiaqi Shen, Junhao Shi, Dukun Zhao, Yuming Lu, Dequan Wang Abstract AI for scientific discovery is entering an agentic era, where protein-engineering systems are expected to prioritize future wet-lab experiments rather than merely fit static measurements. We introduce TadA-Bench, a million-variant wet-lab replay benchmark from 31 TadA directed-evolution rounds for future-round discovery toward agentic protein engineering. TadA-Bench preserves the campaign chronology and defines a fixed-data replay task: given earlier experimental rounds, models rank variants that appear only in later rounds. It provides aligned DNA, RNA, and protein views, and uses Seq2Graph, a graph-based label-unification pipeline, to reconcile noisy enrichment measurements into consistent cross-round activity labels. Random-split controls show strong interpolation, but future-round ranking and finite-budget candidate selection are much weaker. Controlled analyses suggest that evolutionary coverage is more informative than local data density, positioning TadA-Bench as a reproducible wet-lab replay substrate for future-round discovery toward agentic protein engineering; the data and code are released on Hugging Face and GitHub . † † affiliationnotice: † Corresponding authors. 1 Introduction AI for science is moving toward iterative, agent-like workflows, in which models are expected to participate in scientific decision-making rather than passively fit static datasets. Protein engineering is a representative frontier: an assistant may need to read wet-lab histories, use analysis tools, rank candidate mutations, and help decide what to test next. Such workflows require a data substrate with chronological replayability, exploration scale, and cross-round label consistency. This setting calls for benchmarks that turn real experimental campaigns into reproducible past-to-future discovery problems before agent deployment. Figure 1 : TadA-Bench overview. The benchmark frames protein engineering as fixed-data future-round discovery: models observe earlier wet-lab rounds and rank later-round variants. The design combines chronological replayability, million-scale variant coverage, and cross-round activity consistency. Seq2Graph builds a shared activity scale from multi-round NGS selections using within-round rankings, overlap anchors, inconsistency elimination, and activity assignment. Evaluation contrasts random-split interpolation with future-round ranking and finite-budget selection, showing that interpolation on known data does not ensure reliable future-round discovery. We introduce TadA-Bench as a concrete fixed-data implementation of this substrate. As summarized in Figure 1 , TadA-Bench preserves chronological replayability by turning 31 rounds of TadA directed evolution into a task where models use earlier variants and measurements to rank later-round variants. It provides exploration scale through a million-variant campaign and supports cross-round label consistency by aligning DNA, RNA, and protein views through Seq2Graph , a graph-based pipeline that reconciles noisy enrichment measurements using within-round rankings and cross-round sequence overlaps. Constructing the benchmark requires turning noisy multi-round sequencing into a stable label space. Selection data contain batch effects, noisy enrichment estimates, repeated variants, and local pairwise inconsistencies, so Seq2Graph serves as a data-integration pipeline rather than a graph-learning contribution. It uses graph edges for score propagation and consistency correction; recorded rounds, not graph paths, define chronological order. Seq2Graph combines within-round ordering with shared-variant anchors, producing sequence-defined activity scores instead of labels for planned designs or ancestry claims. Using TadA-Bench, we find that representative biological language models struggle with future-round discovery. On the protein view, a random-split control reaches strong rank correlation, indicating that the label space is learnable under interpolation. Under future-round evaluation, however, DNA, RNA, and protein models degrade sharply, and finite-budget candidate selection remains weak. This gap shows that interpolation over known experimental distributions is not sufficient for prioritizing variants that emerge in later wet-lab rounds. The challenge exposed by TadA-Bench is therefore not merely an abstract distribution shift, but a concrete bottleneck for the sequence-ranking component of iterative protein-engineering workflows. To test whether this gap is an artifact of a minimal probing protocol, we include targeted adaptation and discovery-mode checks. Full fine-tuning and prompt tuning give the encoders more task-specific flexibility, while finite-budget candidate prioritization asks how many true high-activity future variants a lab-facing agent would recover under a limited testing budget. The gap persists, indicating that the main difficulty is future-round prioritization from recorded evidence rather than the choice of a lightweight regression head. Our controlled data analyses further suggest that future-round discovery benefits more from evolutionary coverage than from dense local sampling in matched-size analyses. We present this as an empirical observation on TadA-Bench rather than a universal law, but it reinforces the need to preserve campaign structure when building benchmarks for protein discovery. This distinction matters because matched-size controls separate broad campaign coverage from repeated sampling around already observed regions. The benchmark remains fixed-data and reproducible: it isolates past-to-future ranking without evaluating planning policies, tool-use strategies, or autonomous wet-lab execution. This scope keeps TadA-Bench focused on the practical campaign decision of which variants should be prioritized next. It also gives model developers a stable test of whether representations use recorded experimental history before proposal or acquisition layers are added. This keeps comparisons tied to a shared wet-lab history rather than rerun-specific experimental variation. The release follows the same principle: fixed splits, aligned sequence views, baseline protocols, and code let users compare models on one recorded campaign without rerunning wet-lab experiments. Contributions. • We introduce TadA-Bench , a 31-round, million-variant wet-lab benchmark for future-round replay with aligned DNA, RNA, and protein views. • We develop Seq2Graph , a graph pipeline that unifies noisy multi-round enrichment into activity labels. Overlap anchors support label unification, not ancestry inference. • We show that representative biological language models interpolate random splits yet fail on future-round ranking and finite-budget selection. Adaptation checks with full fine-tuning and prompt tuning do not close the gap. • We find that evolutionary coverage is more informative than local data density on TadA-Bench. The result highlights coverage-aware benchmark design. 2 Related Work 2.1 Protein Activity Benchmark While structural benchmarks in protein research have achieved notable success ( 30 ; 18 ; 4 ; 49 ) , functional benchmarks are still in nascent stages. These benchmarks are primarily categorized into two groups ( 44 ) : biophysical properties and deep mutational scanning (DMS) data. Benchmarks for biophysical properties ( 1 ; 46 ; 51 ; 31 ; 36 ; 42 ) include enzymatic activity, fluorescence, thermodynamics, and solubility, but their broad focus limits their utility for precise future-round evaluation. DMS benchmarks ( 15 ; 21 ; 17 ) use large-scale mutagenesis and high-throughput sequencing to provide detailed views of local fitness landscapes. ProteinGym and FLIP aggregate diverse DMS datasets into broad fitness-prediction benchmarks ( 32 ; 8 ; 38 ) , and ProteinBench broadens evaluation across protein foundation-model tasks ( 49 ) . TadA-Bench is complementary to these broad-coverage resources ( 9 ; 2 ; 33 ) : instead of maximizing the number of protein families or assay types, it follows one TadA engineering campaign in depth, preserving 31 wet-lab rounds, a fixed future-round split, and cross-round measurement consistency for evaluating whether models can rank variants from later experimental rounds in a single-campaign replay setting. 2.2 Base Editing Dataset Current datasets focusing on deaminase enzyme optimization in base editing are relatively sparse and often fragmented. Most available resources ( 10 ; 47 ; 28 ; 40 ; 45 ; 23 ) emphasize the optimization of base editors through their interactions with CRISPR-associated proteins (Cas), single-guide RNAs (sgRNAs), or