Phylogenetic Corrections and Higher-Order Sequence Statistics in Protein Families: The Potts Model vs MSA Transformer

Kisan Khatri, Ronald M. Levy, Allan Haldane · 2025

Read Phylogenetic Corrections and Higher-Order Sequence Statistics in Protein Families: The Potts Model vs MSA Transformer by Kisan Khatri, Ronald M. Levy, Allan Haldane (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2503-00289.txt.

Introduction

Kisan Khatri, Ronald Levy, and Allan Haldane compared this 2025 pair of models that learn from protein family alignments. Related proteins share letter patterns shaped by physics and by ancestry. The Potts model reads pairs of positions with a clear energy formula. The MSA Transformer, a neural model, may capture deeper patterns and family history in one go.

The team tested which model better rebuilds high-order patterns in the alignments. The verdict hinged on how ancestry links were handled. Raw alignments carry family-tree echoes that mimic physics signals. With explicit corrections for those echoes, the simpler Potts model beat the transformer at spotting true physical couplings. The study is a caution about hidden ancestry in protein data.

Catalog record

CREATORKisan Khatri, Ronald M. Levy, Allan Haldane
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE27 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2503-00289

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Phylogenetic Corrections and Higher-Order Sequence Statistics in Protein Families: The Potts Model vs MSA Transformer Kisan Khatri, Ronald M. Levy, Allan Haldane Abstract Recent generative learning models applied to protein multiple sequence alignment (MSA) datasets include simple and interpretable physics-based Potts covariation models and other machine learning models such as MSA-Transformer (MSA-T). The best models accurately reproduce MSA statistics induced by the biophysical constraints within proteins, raising the question of which functional forms best model the underlying physics. The Potts model is usually specified by an effective potential including pairwise residue-residue interaction terms, but it has been suggested that MSA-T can capture the effects induced by effective potentials which include more than pairwise interactions and implicitly account for phylogenetic structure in the MSA. Here we compare the ability of the Potts model and MSA-T to reconstruct higher-order sequence statistics reflecting complex biological sequence constraints. We find that the model performance depends greatly on the treatment of phylogenetic relationships between the sequences, which can induce non-biophysical mutational covariation in MSAs. When using explicit corrections for phylogenetic dependencies, we find the Potts model outperforms MSA-T in detecting epistatic interactions of biophysical origin. Introduction - Machine learning models have made great strides in predicting the functional and structural properties of proteins based on large protein sequence datasets, including the subclass of generative protein sequence models (GPSM) that learn an underlying sequence probability distribution P(S) from a Multiple Sequence Alignment (MSA) to generate new synthetic protein sequences S . To function well, GPSMs must capture amino acid patterns in the MSA that implicitly encode information about physical constraints on proteins, enabling the design and detection of hidden protein properties from sequence data Levy et al. 2017 ; Bepler and Berger 2021 . This raises the question of the GPSM functional form that best describes protein biophysics and how to measure this. Here we examine two leading GPSMs in recent focus, the Potts model Weigt et al. 2009 ; Morcos et al. 2011 ; Mora and Bialek 2011 ; Haldane et al. 2016 ; Haldane and Levy 2019 ; Levy et al. 2017 ; Trinquier et al. 2021 and the MSA Transformer (MSA-T) Rao et al. 2021a ; Lupo et al. 2022 . We suggest that certain statistical characteristics of individual protein families provide the most reliable metrics to measure GPSM performance if they exclude the biasing effects of phylogenetic relationships between sequences, which do not originate from the fundamental biophysical properties of the protein family. We find that the Potts model better captures such a statistic, the higher-order MSA statistics which arise due to the potentially large epistatic networks within proteins, when the impact of phylogenetic relationships is carefully accounted for. Figure 1: Phylogenetic relationships between sequences in an MSA result in a spurious mutational correlation due to common ancestry, for R/V and Q/T combinations at the illustrated column-pair. Sequence pairs greater than an identity cut-off (diverging to the left of the gray dotted line) approximate i.i.d. samples, so that identity filtering to retain 3 sequences labeled in blue gives an unbiased sample showing no correlation with equal frequency of R/V, Q/V and Q/T combinations. Effect of Phlyogenetic relationships on GPSM evaluation - Mutational covariation in protein sequences provides a highly informative statistical signal that accurate GPSMs must capture. These arise from multiple factors including: 1) High-fitness sequence motifs and epistasis (mutational cooperativity) underlying biophysical function lead to compensatory mutation pairing patterns between residues. 2) Phylogentic relationships due to recent speciation or gene duplication can distort covariation patterns, causing “excess” counts of patterns from evolutionarily related sequence clusters as illustrated in Fig. 1 , and 3) Statistical noise due to the limited number of available distinct natural sequences used during model inference introduces finite-sampling statistical variation in estimated covariation and in GPSM accuracy. The latter two can be considered nuisance factors in uncovering the underlying biophysics driving the first factor of fitness-induced covariation. These covariation signals are measured from MSAs of individual protein families, which consist of homologous proteins with shared function and overall structure. Despite their quite different architectures and complexity, the Potts model and MSA-T exhibit fundamental commonalities in their design to account for these patterns Martin et al. 2024 . The Potts model is an interpretable, physics-inspired machine learning model fit to a single protein family, inspired by spin-glasses. A Potts model for protein family F has probability distribution P(S\mid\theta_{F})\propto\exp(-\sum_{i<j}J^{ij}_{s_{i}s_{j}}) for sequences S to evolve, with characters s_{i} at position i , and pairwise “coupling” parameters \theta_{F}=\{J^{ij}_{\alpha\beta}\} measuring the favorability of having residues \alpha,\beta at positions i,j in a sequence. It models complex higher-order correlations through networks of pairwise interactions. In contrast, MSA-T is a deep learning LeCun et al. 2015 masked language neural network attention model Vaswani et al. 2017 trained on MSAs of all available protein families Rao et al. 2021b with about 10^{4} times more parameters, recently used for protein structure and evolution analyses Hawkins-Hooker et al. 2021 ; Hong et al. 2022a ; Ma et al. 2023 ; Almalki and Liao 2024 ; Cuturello et al. 2024 ; Chen et al. 2024a ; Chen et al. 2024b . The Potts model has been found to have superior generative accuracy to some other GPSMs including Variational Auto-Encoders and site-independent models McGee et al. 2021 , but a generative method subsequently developed for MSA-T has been reported to better reproduce higher-order statistics Sgarbossa et al. 2023 , though its parameters lack clear physical interpretation and the reasons for this result are unclear. GPSMs must distinguish the causes of covariation De Juan et al. 2013 . In the case of the Potts model an identity filtering procedure is critical. The Potts model is trained to reproduce the site-statistics of a training MSA from the protein family, assuming each sequence is an independent and identically distributed ( i.i.d. ) sample. To be i.i.d , the sequences can be envisioned to have evolved from a distant ancestral sequence under a common fitness function and mutational process encoded in P(S\mid\theta_{F}) , and enough time must have passed for any statistical correlation with each other to be effectively nil due to mutational saturation. Phylogenetic relationships violate the independence assumption, as clusters of orthologs and other closely evolutionarily related sequences are explicitly non- i.i.d. . In standard Potts methodology the related sequences are filtered to eliminate phylogenetic redundancy. In contrast, MSA-T accounts for phylogenetic relationships through a column-wise attention layer Lupo et al. 2022 ; Chen et al. 2024a , and is not explicitly trained on MSA statistics but rather on a masked prediction task insensitive to phylogenetic structure, in predicting character states in input MSAs from all protein families at once which were randomly masked. A method has been proposed to use this in a generative fashion for a single protein family by repeatedly masking and resampling an input MSA of that family to produce novel sequences Sgarbossa et al. 2023 . The MSA-T probability distribution for this generative method is P(S\mid\theta,M_{F},p) for generating sequence S , and depends the model’s pre-trained parameters \theta , an input MSA M_{F} for a protein family F , and a masking frequency parameter p . One goal of this study is to control for how phylogenetic clusters

Read the complete text