AMix-1 : A Pathway to Test-Time Scalable Protein Foundation Model

2025

Read AMix-1 : A Pathway to Test-Time Scalable Protein Foundation Model (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2507-08920.txt.

Introduction

Lv and co-authors introduce AMix-1, a large model for proteins built to scale up both in training and at test time. The model rests on a Bayesian flow network design and a systematic training plan. Their 2025 paper first sets a scaling law, then watches structural understanding emerge as the model grows to 1.7 billion parameters.

A key trick feeds the model aligned sequences from related proteins, letting it read deep evolutionary signals and design new proteins inside one framework. With it, the team designed a variant of one protein with up to fifty times the activity of the natural form. A test time method mimics directed evolution in the computer, gaining more as the checking budget grows. The authors frame the system as groundwork for lab linked protein design.

Catalog record

YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE87 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2507-08920

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

AMix-1 : A Pathway to Test-Time Scalable Protein Foundation Model Abstract We introduce AMix-1 , a powerful protein foundation model built on Bayesian Flow Networks and empowered by a systematic training methodology, encompassing pretraining scaling laws, emergent capability analysis, in-context learning mechanism, and test-time scaling algorithm. To guarantee robust scalability, we establish a predictive scaling law and reveal the progressive emergence of structural understanding via loss perspective, culminating in a strong 1.7-billion model. Building on this foundation, we devise a multiple sequence alignment (MSA)-based in-context learning strategy to unify protein design into a general framework, where AMix-1 recognizes deep evolutionary signals among MSAs and consistently generates structurally and functionally coherent proteins. This framework enables the successful design of a dramatically improved AmeR variant with an up to 50 × 50\times activity increase over its wild type. Pushing the boundaries of protein engineering, we further empower AMix-1 with an evolutionary test-time scaling algorithm for in silico directed evolution that delivers substantial, scalable performance gains as verification budgets are intensified, laying the groundwork for next-generation lab-in-the-loop protein design. 1]Shanghai Artificial Intelligence Laboratory 2]Generative Symbolic Intelligence Lab (GenSI), Tsinghua University 3]Institute for AI Industry Research (AIR), Tsinghua University \correspondence [email protected] \projectpage https://gensi-thuair.github.io/AMix-1/ Figure 1 : An overview of AMix-1 . AMix-1 is built upon a pathway methodology of scaling law, emergent ability, in-context learning, and test-time scaling, and is validated on both in silico evaluation and wet-lab assay. 1 Introduction Advanced artificial intelligence models, such as AlphaFold [ 1 , 2 , 3 ] and ESM [ 4 , 5 , 6 ] , have significantly facilitated a wide range of protein research, including structure prediction [ 1 , 2 , 3 , 7 , 8 , 9 ] , inverse folding [ 10 ] , functional property prediction [ 11 , 12 ] , mutational effect estimation [ 13 , 14 , 15 ] , and de novo protein design [ 6 , 16 ] . However, these models lack a unified, scalable, and systematic methodology akin to that of cutting-edge large language models (LLMs) [ 17 , 18 , 19 , 20 , 21 , 22 , 23 ] , whose capabilities consistently improve with increased scale in data, model size, and compute. Consequently, the systematic trading of compute for performance has revealed unprecedented emergent intelligence, as models continue to improve with larger computational investments, both during pretraining [ 24 , 25 , 26 ] and at inference [ 27 , 28 , 29 ] . This observation motivates a fundamental question: As large language models begin to exhibit traits of artificial general intelligence, how can we craft protein foundation models under a converging methodology grounded in scaling law characterization, emergent capability analysis, in-context learning mechanism, and test-time scaling algorithms, to unlock scalable and universal protein design of high fidelity? In this study, we propose AMix-1 , a pathway methodology for scalable protein design ( Figure 1 ). We first characterize a predictable scaling law for Bayesian Flow Networks [ 30 ] and reveal that structural understanding emerges naturally as training progresses, with which we pretrained a powerful 1.7B AMix-1 . Then we leverage its in-context learning capability with multiple sequence alignments to unify structure- and function-guided protein design into a general framework. With AMix-1 , we successfuly design a novel AmeR variant validated through wet-lab experiments, achieving a 50\times activity increase over the wild type. To push the limits further, we fledge AMix-1 with an evolutionary test-time scaling (TTS) algorithm for in silico directed evolution, allowing plug-and-play metrics and delivering strong performance gains with increasing verification budgets. We summarize the main contributions of our study as follows: • We propose a systematic paradigm, including scaling laws, structural emergent ability, in-context learning, and test-time scaling, for crafting protein foundation models, emphasizing a road map towards scalable protein design; • We are the first to characterize scaling law and emergent ability for an advanced generative model, i.e., Bayesian Flow Network, and pretrain a family of powerful models, AMix-1 , scaling from 8 million to 1.7 billion parameters; • We introduce a novel evolutionary test-time scaling algorithm by leveraging AMix-1 ’s in-context learning for unified protein design, amplifying its effectiveness in in silico directed evolution, delivering rapid gains as verification budgets grow; • We design a high-activity AmeR variant with a 50\times improvement over the wild type, showcasing AMix-1 ’s real-world impact on functional protein engineering. 2 Protein Modeling with Bayesian Flow Network Protein modeling with bayesian flow networks (BFNs) [ 30 ] introduces a generative framework that learns the continuous parameters of protein sequence distributions through iterative Bayesian updates, rather than directly modeling discrete amino acid sequences. This process iteratively refines a set of noisy samples initiated from an uninformative prior {\bm{\theta}}_{0} , producing successive posteriors {\bm{\theta}}_{i} that gradually incorporate more information and increase certainty. The generative process is modeled as a message-passing protocol between sender distribution and receiver distribution , where the sender’s visibility is confined to the sample space, while the receiver infers messages based on its interpretation of samples and parameters. During each communication round, the sender distribution p_{\text{s}}(\mathbf{y}\mid\mathbf{x};\alpha) perturbs protein sequences \mathbf{x} with a noise scheduler \alpha and sends the noisy protein sequences \mathbf{y} to the receiver distribution, which mirrors a forward diffusion mechanism. Specifically, for the protein sequence distribution, the sender distribution can be formulated as follows: p_{\text{s}}(\mathbf{y}\mid\mathbf{x};\alpha)=\mathcal{N}\left(\mathbf{y}\mid\alpha(K\mathbf{e}(\mathbf{x})-\mathbf{1}),\alpha K\mathbf{I}\right) (1) where \mathbf{e}(\mathbf{x}) is the one-hot encoding of the protein sequence \mathbf{x} , flattened into a vector, with each position encoded as a K -dimensional vector, and K is the number of amino acid types. Here \alpha is derived from the schedule \beta(t) , where \beta_{1}\in R^{+} is the final noise level at t=1 : \alpha=\beta(t)=\beta_{1}t^{2} (2) On the other hand, the receiver distribution employs a neural network {\bm{\phi}} to approximate the sender distribution with its current belief {\bm{\theta}}_{i-1} . Concretely, the network {\bm{\phi}} takes the belief state {\bm{\theta}}_{i-1} as input, and the corresponding distribution p_{\text{in}}(\cdot|{\bm{\theta}}) is termed the input distribution . The network output {\bm{\phi}}({\bm{\theta}}_{i-1}) is an updated distribution p_{\text{out}}(\cdot|{\bm{\phi}}({\bm{\theta}}_{i-1})) over the sample space, termed the output distribution . Integrating the output distribution with the noise scheduler \alpha yields the receiver distribution as follows: p_{\text{r}}(\mathbf{y}_{i}\mid{\bm{\theta}}_{i-1},{\bm{\phi}},\alpha_{i})=\mathbb{E}_{p_{\text{out}}(\mathbf{x}^{\prime}|{\bm{\phi}}(\bm{{\bm{\theta}}}_{i-1}))}p_{\text{s}}\left(\mathbf{y}_{i}\mid\mathbf{x}^{\prime};\alpha_{i}\right) (3) Figure 2 : Illustration of Bayesian Flow Networks (BFNs). The network {\bm{\phi}} takes as input the distribution p_{\text{in}} and generates the output distribution p_{\text{out}} . The training objective is to minimize the Kullback-Leibler (KL) divergence between the sender distribution p_{s} and the receiver distribution p_{r} , both of which are perturbed by additive noise \alpha_{i} . Therefore, the final training objective integrates the KL divergence between sender and receiver distributions across timesteps and trajectories, yielding the following loss: \displaystyle\mathcal{L}({\bm{\phi}})=\mathbb{E}_{\mathbf{x}\sim p_{\text{data}}}\mathbb{E}_{\Pi_{i=1}^{n}p_{\text{s}}(\mathbf{y}_{i}|\mathbf{x},\alpha_{i})}D_{KL}\left(p_{\text{s}}(\mathbf{y}_{i}|\mathbf{x},\alpha_{i})||p_{\text{r}}(\mathbf{y}_{i}\mid{\bm{\theta}}_{i-1},{\bm{\phi}},\alpha_{i})\right), (4) Leveraging the Bayesian update function to model the information gathering process, which iteratively refines priors into data-informed posteriors as follows: {\bm{\theta}}_{i}=h({\bm{\theta}}_{i-1},\mathbf{y}_{i},\alpha_{i})=\frac{e^{\mathbf{y}}{\bm{\theta}}_{i-1}}{\sum_{k=1}^{K}e^{\mathbf{y_{k}}}({\bm{\theta}}_{i-1})_{k}} (5) In practice, we use the continuous categorical values of output distribution {\bm{\phi}}(\bm{{\bm{\theta}}}_{i-1}) without sampling to directly update {\bm{\theta}} bypassing the sampling of noisy data needed for Bayesian update, which is similar to the noise reduced sampling method used in MolCRAFT [ 31 ] . The effective sampling method is described in appendix Algorithm 2 . We also provide Figure 2 to help the readers visually grasp the key

Read the complete text