Steering Generative Models with Experimental Data for Protein Fitness Optimization
Jason YangCaltech, Wenda ChuCaltechDaniel KhalilCaltechRaul AstudilloCaltechBruce J. WittmannMicrosoft CorporationFrances H. ArnoldCaltechYisong YueCaltech · 2025
Read Steering Generative Models with Experimental Data for Protein Fitness Optimization by Jason YangCaltech, Wenda ChuCaltechDaniel KhalilCaltechRaul AstudilloCaltechBruce J. WittmannMicrosoft CorporationFrances H. ArnoldCaltechYisong YueCaltech (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2505-15093.txt.
Introduction
Jason Yang and co-authors study how to steer protein-generating models with small amounts of lab data, in a paper from 2025. Protein design searches a vast space of letter sequences for one with the best measured traits. New generative models can propose sequences, but real fitness comes from slow lab tests, so labels are few.
The team used only hundreds of labeled sequence and fitness pairs. They compared ways to guide generation, including classifier guidance and posterior sampling, across several diffusion models. They also wove guidance into an adaptive picking scheme akin to Bayesian optimization. Plug-and-play guidance beat options such as reinforcement learning on language models. The paper gives practical advice for real design campaigns with tight lab budgets.
Catalog record
| CREATOR | Jason YangCaltech, Wenda ChuCaltechDaniel KhalilCaltechRaul AstudilloCaltechBruce J. WittmannMicrosoft CorporationFrances H. ArnoldCaltechYisong YueCaltech |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 64 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2505-15093
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Steering Generative Models with Experimental Data for Protein Fitness Optimization Jason YangCaltech, Wenda ChuCaltechDaniel KhalilCaltechRaul AstudilloCaltechBruce J. WittmannMicrosoft CorporationFrances H. ArnoldCaltechYisong YueCaltech Abstract Protein fitness optimization involves finding a protein sequence that maximizes desired quantitative properties in a combinatorially large design space of possible sequences. Recent developments in steering protein generative models (e.g diffusion models, language models) offer a promising approach. However, by and large, past studies have optimized surrogate rewards and/or utilized large amounts of labeled data for steering, making it unclear how well existing methods perform and compare to each other in real-world optimization campaigns where fitness is measured by low-throughput wet-lab assays. In this study, we explore fitness optimization using small amounts (hundreds) of labeled sequence-fitness pairs and comprehensively evaluate strategies such as classifier guidance and posterior sampling for guiding generation from different discrete diffusion models of protein sequences. We also demonstrate how guidance can be integrated into adaptive sequence selection akin to Thompson sampling in Bayesian optimization, showing that plug-and-play guidance strategies offer advantages compared to alternatives such as reinforcement learning with protein language models. $\dagger$ $\dagger$ footnotetext: These authors contributed equally to this work. 1 Introduction Proteins, sequences of amino acids, can be optimized for useful properties such as binding affinity, catalytic activity, or stability, numerically quantified as “fitness.” However, the protein optimization problem is challenging: The design space of proteins is vast, as a protein of length M can be constructed in 20^{M} different ways; only a negligible fraction of possible protein sequences are functional ( Romero & Arnold 2009 ) ; and most experimental wet-lab assays can only provide 10^{2}-10^{3} fitness labels per round. As a result, researchers often rely on directed evolution, an iterative process that incrementally improves protein fitness ( Packer & Liu 2015 ) . In each round of directed evolution, a protein is mutated, the fitness of the variants is measured, and the most beneficial variant is retained for the next iteration. However, this approach can be slow (often only one mutation is accumulated in each round), and it is often inefficient–as it is limited to a local search of very similar protein sequences. Figure 1: Comparison of steered generation for protein optimization (SGPO) to other ML-assisted workflows for protein engineering. (A) SGPO involves initializing a generative prior model to sample sequences with high natural likelihoods and steering that model with assay-labeled fitness data. Optimization is difficult because the design space is massive, and the throughput of wet-lab fitness assays (Erlenmeyer flask icon) is low, so adaptive learning across multiple iterations is beneficial. Previous methods have utilized generative models such as (B) fully zero-shot methods that sample highly natural sequences but do not utilize labeled fitness data or (C) those that only utilize labeled fitness. (D) Alternatively, supervised approaches involve enumerating to calculate fitness predictions for all variants in a design space, limiting them to optimizing few residues (i.e. N<9 ). In recent years, there has been strong interest in developing machine learning (ML)-assisted methods to more efficiently optimize protein fitness ( Yang et al. 2019 ; Wittmann et al. 2021a ; Hie & Yang 2022 ; Yang et al. 2024b ) . Many recent studies have focused on generative approaches that combine both unlabeled and labeled data for protein design. Broadly, these methods achieve conditional generation by steering generative priors of natural protein sequences ( Freschlin et al. 2022 ) with fitness data, which can be incorporated into an adaptive optimization cycle ( Hie & Yang 2022 ) . These S teered G eneration for P rotein O ptimization (SGPO) methods are advantageous because they do not face the individual limitations of other methods (Fig. 1 A, Table 1 ). Firstly, SGPO utilizes labeled data, which is important for fitness goals that deviate from natural function (e.g. enzymes being engineered for non-native activities ( Arnold 2018 ; Yang et al. 2025 ) ), unlike previous studies utilizing only generative priors of natural sequences, e.g. for antibody design ( Generative: Zero-Shot , Fig. 1 B, Hie et al. 2023 ). Secondly, generative priors ( Wu et al. 2021 ; Hsu et al. 2024 ) sample sequences with high evolutionary likelihoods and potentially higher fitness, which is advantageous compared to approaches only using labeled data ( Brookes et al. 2019 ; Brookes & Listgarten 2020 ; Song & Li 2024 ; Stanton et al. 2022 ; Gupta & Zou 2019 ; Hie & Yang 2022 ) ( Generative: Adaptive , Fig. 1 C). Finally, SGPO is not limited to a small design space, unlike many supervised ML-assisted directed evolution (MLDE) approaches, which require enumerating to calculate fitness predictions for all variants in the design space ( Supervised , Fig. 1 D) ( Wu et al. 2019 ; Wittmann et al. 2021b ; Yang et al. 2025 ; Li et al. 2024a ; Vornholt et al. 2024 ; Jiang et al. 2024 ; Hsu et al. 2022 ; Ding et al. 2024 ; Hawkins-Hooker et al. 2024 ; Zhao et al. 2024a ; Thomas et al. 2025 ) . Table 1: SGPO is a general approach for protein fitness optimization that doesn’t face the individual limitations of other strategies. Namely, SGPO utilizes zero-shot knowledge from the natural distribution of proteins, can be guided by assay-labeled fitness data, and can optimize many residues ( N ) simultaneously. Beyond those listed here, there are many other studies that combine different elements of these approaches. Approach Prior Information Used? Assay Fitness Used? Scales to large N ? Protein Examples (non-exhaustive) SGPO ✓ ✓ ✓ Lisanza et al. 2024 ; Widatalla et al. 2024 ; Stocco et al. 2024 ; Nisonoff et al. 2024 ; Blalock et al. 2025 ; Goel et al. 2024 Generative: Zero-Shot ✓ \times ✓ Hie et al. 2023 Generative: Adaptive \times ✓ ✓ Brookes et al. 2019 ; Brookes & Listgarten 2020 ; Song & Li 2024 ; Stanton et al. 2022 Supervised ✓ ✓ \times Wittmann et al. 2021b ; Ding et al. 2024 ; Hawkins-Hooker et al. 2024 ; Zhao et al. 2024a Despite these advantages, SGPO methods remain limited in their applicability to real-world fitness optimization–specifically considering two classes of methods: guiding discrete diffusion models ( Nisonoff et al. 2024 ; Stark et al. 2024 ; Klarner et al. 2024 ; Gruver et al. 2023 ; Lisanza et al. 2024 ; Goel et al. 2024 ) and reinforcement learning (RL) by finetuning models such as protein language models (PLMs) ( Ruffolo & Madani 2024 ; Widatalla et al. 2024 ; Stocco et al. 2024 ; Blalock et al. 2025 ) . The limitations of prior work are summarized as follows: (1) Few previous studies have explored steering with few ( 10^{2}-10^{3} ) labeled sequences ( Lisanza et al. 2024 ; Stocco et al. 2024 ) for protein optimization based on real fitness data, e.g. activity or fluorescence, rather than computational surrogates ( Lisanza et al. 2024 ; Blalock et al. 2025 ) . (2) Most studies only evaluate one type of generative prior and steering strategy, so it is unclear how different combinations perform in practice. (3) Most existing approaches fail to incorporate principles from adaptive optimization, such as uncertainty-aware exploration (e.g., Bayesian optimization), which have shown