Repurposing Protein Language Models for Latent Flow–Based Fitness Optimization Amaru Caceres Arroyo, Lea Bogensperger, Ahmed Allam, Michael Krauthammer, Konrad Schindler, Dominik Narnhofer Abstract Protein fitness optimization is challenged by a vast combinatorial landscape where high-fitness variants are extremely sparse. Many current methods either underperform or require computationally expensive gradient-based sampling. We present CHASE, a framework that repurposes the evolutionary knowledge of pretrained protein language models by compressing their embeddings into a compact latent space. By training a conditional flow-matching model with classifier-free guidance, we enable the direct generation of high-fitness variants without predictor-based guidance during the ODE sampling steps. CHASE achieves state-of-the-art performance on AAV and GFP protein design benchmarks. Finally, we show that bootstrapping with synthetic data can further enhance performance in data-constrained settings. † † affiliationnotice: Equal contribution 1 Introduction Figure 1 : Fitness–diversity–novelty chart across protein optimization methods for GFP Medium. CHASE achieves high fitness while maintaining diversity and novelty compared to existing baselines, whereas alternative methods tend to favor individual objectives at the expense of others. Figure 2 : Overview of CHASE. (1) Discrete protein sequences are encoded by a pretrained pLM and (2) projected into a compact latent manifold via a compression block. (3) A conditional generative model learns the probability path between noise and these embeddings, conditioned on time t and fitness f . (4) At inference, high-fitness embeddings are sampled and mapped back to sequence space through a decompressor and a discrete decoder. Gray blocks denote modules trained in our framework. Proteins are essential macromolecules whose primary amino acid sequences dictate their three-dimensional structures, which in turn determine their biological functions—such as stability, binding affinity, and catalytic efficiency. Protein fitness optimization aims to enhance these properties by navigating the fitness landscape, a vast combinatorial space of potential sequences. Because the number of possible sequences grows exponentially with length, identifying variants with meaningful biological functionality remains a significant challenge ( 11 ) . Traditional directed evolution ( 29 ) mimics natural selection through iterative laboratory cycles; however, despite its success, it is fundamentally limited in its ability to efficiently explore this high-dimensional and often sparse landscape. To overcome these limitations, a variety of in-silico approaches have been proposed to computationally navigate the fitness landscape and prioritize only the most promising candidates for experimental validation. These strategies range from evolutionary greedy algorithms ( 39 ; 32 ; 33 ) as well as gradient-based sampling methods such as Gibbs With Gradients (GWG) ( 9 ; 7 ) and its smoothed variants ( 18 ) , which utilize local fitness gradients to explore discrete sequence spaces. However, navigating these landscapes remains challenging because protein sequences are discrete and rugged due to epistatic interactions, where a mutation’s effect depends on its amino acid context ( 36 ) . Recent methods like Variational Latent Generative Protein Optimization (VLGPO) ( 2 ) address these challenges by embedding proteins into a continuous latent space. While this enables smoother exploration via flow matching ( 22 ) , current approaches rely on predictor-guided trajectors during sampling, which can be computationally demanding if they have to be backpropagated through ( 5 ) . Furthermore, these models rely on variational autoencoders (VAEs) that learn latent spaces de novo, failing to leverage the rich, evolutionary priors captured by large-scale pre-trained protein language models (pLMs) ( 21 ) . In this work, we propose Conditional High-fitness Amino-acid Sequence Enhancement (CHASE) , a framework that repurposes the expressive embedding space of pLMs into a compact latent representation inspired by the Hourglass architecture in ( 19 ) , which leverages the structural priors implicitly encoded in pLMs ( 27 ) . Our approach is summarized graphically in Figure 2 . Specifically, we make the following contributions: • We demonstrate that pretrained pLMs can be repurposed into a compressed, continuous latent space capable of capturing the fine-grained fitness effects of mutations within a specific protein family. • We develop a framework that adapts conditional flow matching and classifier-free guidance to learn the distribution of latent protein sequences, effectively eliminating the need for external predictor models during sampling. • We achieve state-of-the-art performance on two protein fitness optimization benchmarks, generating candidates with competitive fitness while maintaining sequence diversity and novelty. • We propose a data augmentation and bootstrapping strategy that expands the limited training set with synthetic sequence-fitness pairs, further enhancing the search for high-fitness variants. 2 Related Work Traditional and Surrogate-Based Optimization. The foundational paradigm for protein engineering is directed evolution ( 29 ) , which navigates fitness landscapes through physical mutagenesis and screening. This method, which can be viewed as a random walk from a wild-type starting sequence, is inherently resource-intensive: the majority of random mutations are deleterious, and the throughput of physical assays cannot scale to the exponential complexity of the sequence-to-function mapping. To mitigate these costs, surrogate models are increasingly employed to guide the search more efficiently ( 33 ; 28 ; 23 ) . For instance, Adalead ( 33 ) utilizes a black-box predictor to inform a greedy algorithm, prioritizing mutations with the highest predicted fitness gains. Similarly, LatProtRL frames the search as a reinforcement learning problem, simulating active learning by perturbing the latent space of a large pLM to identify high-fitness actions ( 20 ) . Recently, EvolvePro was introduced ( 16 ) , a powerful active learning framework which combines the computational capacity to navigate high-dimensional search spaces with empirical feedback to ground the model’s predictions. Discrete Generative Models. Efficient optimization requires in-silico methods capable of proposing high-fitness candidates without immediate experimental feedback. As protein sequences are discrete, many generative approaches model this representation directly. Sampling-based methods, such as the walk-jump algorithm ( 10 ; 8 ) , learn distributions over one-hot encodings using a single noise level and a Tweedie step for recovery via Markov chain Monte Carlo (MCMC) sampling. This framework was later extended to incorporate gradient guidance within the noisy manifold ( 14 ) . Parallel efforts, such as GWG ( 9 ; 7 ) and its variant Gibbs sampling with Graph-based Smoothing (GGS) ( 18 ) , introduce gradient-informed MCMC for discrete sequences. These methods train a separate predictor model on limited datasets to generate guiding gradients that steer the sampling process. GGS specifically incorporates graph-based smoothing to regularize the inherently noisy fitness landscape, providing more stable optimization trajectories. Optimization in Continuous Latent Spaces. As sequence length increases, the search space expands exponentially, leading recent research to explore continuous latent representations to facilitate optimization ( 26 ) . For example, latent directed evolution (LDE) ( 23 ) maps sequences to a smoothed latent space, enabling gradient ascent-based optimization guided by a learned fitness predictor. Similarly, VLGPO ( 2 ) leverages the predictors established in ( 18 ) to perform informed posterior sampling, executing gradient steps toward high-fitness regions within the continuous manifold during the generative process. More recently, generative property enhancer (GPE) ( 25 ) introduced a formulation that implicitly guides generation from low-fitness to high-fitness samples through conditional density estimation. Protein Language Representations. The efficacy of these methods relies heavily on the quality of the underlying sequence representation. pLMs , such as ESM2 ( 21 ) , have emerged as the standard for capturing complex evolutionary patterns and implicitly learning the structural regularities of the protein space from massive sequence databases ( 27 ) . Recent frameworks like ProtFlow ( 19 ) leverage these pretrained pLMs by operating within a compressed latent embedding space, enabling efficient de novo design. While ProtFlow demonstrates the power of latent-space generation, it primarily focuses on unconditional backbone and sequence generation; our work extends this paradigm by introducing direct fitness-conditioning directly into the flow-matching objective in the compressed pLM latent space. Building on these advancements, we propose CHASE , a framework that leverages the rich, continuous pLM embeddings of ESM2 ( 21 ) to establish an expressive representation space. Inspired by the architectural insights of ( 19 ) , we project these embeddings into a compressed manifold where fitness information is directly integrated into the flow-matching objective. By learning the latent distribution conditioned on these fitness values, CHASE enables fast sequence generation with targeted high-fitness while preserving diversity and novelty. Furthermore, our approach supports controllable fitness sampling and facilitates synthetic data augmentation, which provides a mechanism for efficient model bootstrapping in data-constrained regimes. 3 Method 3.1 Protein Optimization Let \mathcal{V} denote the protein vocabulary consisting of 20 canonical amino acids and associated special tokens. A protein sequence of length L is defined as x=(x_{1},\dots,x_{L})\in\mathcal{V}^{L} . We define a fitness landscape as a mapping g:\mathcal{V}^{L}\to\mathbb{R} , thereby assigning a scalar value g(x) to each sequence x representing its biological performance. The goal of protein fitness optimization is to identify new sequences x that maximize this property, which is achieved by introducing mutations, i.e. substituting amino acids in a starting sequence with any variant in \mathcal{V} to enhance the fitness value f . In this work, we focus on a constrained benchmark setting where we only have access to a limited subset of sequence-fitness pairs \mathcal{A}\subset\mathcal{A}_{\text{GT}} ( 18 ) . This training set \mathcal{A} is restricted to narrow fitness ranges and contains only variants that maintain a significant distance from the highest-performing sequences in the full, unknown ground-truth dataset \mathcal{A}_{\text{GT}} . Our modeling and data augmentation are performed exclusively on \mathcal{A} ; the ground-truth landscape \mathcal{A}_{\text{GT}} is reserved solely for final evaluation to evaluate the fitness of the generated sequences using an oracle trained on \mathcal{A}_{\text{GT}} . Our framework, CHASE , is shown in Figure 2 . First, we establish a continuous representation space by mapping discrete sequences x\in\mathcal{V}^{L} to high-dimensional pLM embeddings h\in\mathbb{R}^{L\times D} (1), which are subsequently compressed into a low-dimensional latent manifold z\in\mathbb{R}^{l\times d} (2). Then we train a conditional flow matching model to learn the distribution p(z\mid f) within this compressed space (3). Finally, we sample z given a target fitness f and reconstruct the resulting sequence x^{\prime} (4). We explain the details of each step in the following sections. 3.2 Latent Manifold Construction The VAE framework ( 17 ; 12 ) provides a principled approach to mapping discrete protein sequences into a continuous latent space. A VAE learns a probabilistic mapping between data x and a lower‐dimensional latent representation z by maximizing the Evidence Lower Bound (ELBO) : \mathcal{L}(x;\theta)=\mathbb{E}_{q_{\theta}(z\mid x)}[\log p_{\theta}(x|z)]-\beta\,\mathrm{KL}\!\left(q_{\theta}(z|x)\,\|\,p(z)\right). (1) This objective balances reconstruction accuracy with latent space regularization, since the Kullback-Leibler (KL) divergence encourages the approximate posterior q_{\theta}(z|x) to remain close to a prior p(z)=\mathcal{N}(0,I) . In CHASE , we implement this framework as a two-stage process (steps (1) and (2) in Figure 2 ). First, we repurpose a pretrained ESM2-8M encoder \mathcal{E} ( 21 ) without further finetuning to map the discrete input x into a continuous embedding h\in\mathbb{R}^{L\times D} . The corresponding decoder \mathcal{D} is trained via cross-entropy loss to reconstruct sequence logits from h . Second, as even the smallest ESM2 embeddings ( D=320 ) are high-dimensional and prone to activation sparsity ( 35 ) , we employ an additional compressor \mathcal{C} and decompressor \mathcal{R} pair adopted from ( 19 ) . This architecture reduces both the feature dimension to d\ll D and the sequence length to l