What Makes a Representation Good for Single-Cell Perturbation Prediction?
Wenkang Jiang, Yuhang Liu, Yichao Cai, Erdun Gao, Jiayi Dong, Ehsan Abbasnejad, Lina Yao, Javen Qinfeng Shi · 2026
Read What Makes a Representation Good for Single-Cell Perturbation Prediction? by Wenkang Jiang, Yuhang Liu, Yichao Cai, Erdun Gao, Jiayi Dong, Ehsan Abbasnejad, Lina Yao, Javen Qinfeng Shi (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-19343.txt.
Introduction
Jiang and colleagues tackle a blind spot in predicting how single cells react to genetic changes. Gene readings are swamped by stable background signals, while change-specific signals are thin, so models mix the two or drop the useful part. Their PerturbedVAE framework pulls the change-specific signal apart from the background and rebuilds causal patterns for prediction. Benchmark tests beat prior methods, most clearly on new combined changes.
Catalog record
| CREATOR | Wenkang Jiang, Yuhang Liu, Yichao Cai, Erdun Gao, Jiayi Dong, Ehsan Abbasnejad, Lina Yao, Javen Qinfeng Shi |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 127028 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2605.19343
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
What Makes a Representation Good for Single-Cell Perturbation Prediction? Wenkang Jiang, Yuhang Liu, Yichao Cai, Erdun Gao, Jiayi Dong, Ehsan Abbasnejad, Lina Yao, Javen Qinfeng Shi Abstract Single-cell perturbation modeling is fundamental for understanding and predicting cellular responses to genetic perturbations. However, existing approaches, from causal representation learning to foundation models, often struggle with an overlooked challenge: gene expression is dominated by perturbation-invariant information, while perturbation-specific signals are intrinsically sparse. As a result, learned representations either entangle invariant and perturbation-specific information, leading to spurious and non-generalizable predictors, or suppress perturbation-specific signals altogether, rendering them ineffective for prediction. To address this, we propose PerturbedVAE , a general framework designed to resolve this signal imbalance. The framework explicitly separates perturbation-specific information from dominant invariant structure and recovers causal representations to effectively utilize such information for prediction. We further provide an identifiability analysis that characterizes the conditions under which sparse perturbation effects can be reliably recovered, thereby clarifying how the framework can be concretely specified under such conditions. Empirically, PerturbedVAE achieves state-of-the-art performance on a widely used benchmark across multiple evaluation settings, yielding significant gains on out-of-distribution combinatorial predictions and uncovering interpretable perturbation-response programs. 1 Introduction The goal of single-cell perturbation modeling is to predict how a cell’s gene expression changes in response to genetic perturbations ( 51 ) , such as CRISPR-based perturbations ( 31 ; 21 ; 16 ; 54 ) . Unlike standard predictive tasks, this problem involves perturbations that actively modify the underlying biological system. As a result, it presents two core challenges: generalization and interpretability. The first concerns generalization to unseen perturbations, e.g., combinatorial ones ( 16 ; 54 ) . The second concerns understanding how perturbations reshape underlying cellular programs ( 25 ; 47 ) . Existing work on single-cell perturbation modeling has primarily focused on two directions: (i) learning causal representations, and (ii) learning general representations using foundation models (FMs) (See Sec. A for additional discussion). The first direction is motivated by leveraging causal mechanisms to support both generalization and interpretability ( 36 ; 76 ; 46 ; 15 ) . These approaches typically employ latent causal generative models to capture high-level causal variables that govern gene expression, and aim to recover such variables from data, also known as causal representation learning ( 61 ; 45 ) . The second direction is inspired by the success of FMs in various domains, and emphasizes data and model scale to learn powerful, transferable representations ( 14 ; 23 ; 74 ; 58 ; 65 ) . The Perturbation Suppression Hypothesis (Sec. 2.1 ). Across both directions above, an empirical property of single-cell perturbation data is often underemphasized: perturbation-invariant information, such as background cellular programs, typically dominates gene expression, while perturbation-specific information is sparse. This gives rise to what we term the Perturbation Suppression Hypothesis : when invariant variation dominates the data, approaches that do not explicitly separate invariant and perturbation-specific information tend to fail systematically. In particular, such approaches either entangle invariant information into perturbation-related representations, undermining their perturbation-specific semantics and leading to poor generalization, as in existing causal representation learning methods, or suppress perturbation-induced signals that are critical for accurate perturbation prediction, as in FM-based approaches. Perturbation-Aware Representations (Sec. 2.2 ). Motivated by the perturbation suppression hypothesis, we advocate for learning perturbation-aware representations , which explicitly aims to: 1) separate perturbation-specific information from dominant, perturbation-invariant one ( i.e., Extraction ), 2) organize the extracted perturbation-specific information in a manner that supports generalization, and crucially, interpretability ( i.e., Utilization ): Contributions. Beyond introducing the perturbation suppression hypothesis and perturbation-aware representations (Sec. 2 ), we make the following contributions. First, guided by learning perturbation-aware representations, we propose a general framework, termed Perturbation-aware Variational Autoencoders (PerturbedVAE), which combines an alignment-based component to isolate perturbation-specific information from dominant, unperturbed cellular programs, with a latent causal model that organizes the isolated information, enabling principled generalization to unseen perturbations (Sec. 3 ). Second, we provide an identifiability analysis that characterizes the conditions under which perturbation-specific information can be reliably recovered. Importantly, this analysis clarifies how the proposed framework admits theoretically grounded specifications (Sec. 4 ). Third, we empirically demonstrate substantial improvements over strong baselines across multiple single-cell perturbation benchmarks, with particularly strong gains in generalization to unseen combinatorial perturbations, while yielding interpretable representations of perturbation effects (Sec. 5 ). 2 Perturbation Suppression and Beyond We begin by introducing the perturbation suppression hypothesis, motivated by a fundamental yet often overlooked property of single-cell perturbation data: perturbation-invariant information typically dominates gene expression. From this perspective, we show that representations that do not explicitly distinguish perturbation-invariant from perturbation-specific information may fail to effectively utilize or preserve perturbation-induced signals. Finally, we introduce perturbation-aware representation learning to address this hypothesis. 2.1 Perturbation Suppression Hypothesis Largely because of experimental, environmental, and cost constraints, perturbation-specific signals typically occupy only a small portion of the gene expression space, while unperturbed background programs account for the majority, even in large-scale perturbation datasets. For example, in widely used Perturb-seq benchmark datasets, the number of distinct perturbations is relatively limited (e.g., on the order of \sim 284 in the dataset of 50 ), whereas gene expression profiles span tens of thousands of measured genes (e.g., \sim 19,264 genes ( 2 ) ). In this setting, methods that are not explicitly designed to separate perturbation-invariant and perturbation-specific information tend to exhibit the following failure modes: either the learned perturbation-related representations become entangled with invariant information, or perturbation-induced signals are suppressed, as below. Implications for Learning Causal Representations. Although conceptually appealing, the effectiveness of learning causal representations is not closely supported by theoretical guarantees that are typically grounded in identifiability results, which ensure that the underlying causal variables can be recovered up to simple transformations. A key assumption underlying most existing identifiability results is access to sufficiently rich interventional data, i.e., that all latent causal variables are perturbed across environments ( 36 ; 76 ; 46 ; 15 ) . In practice, however, single-cell perturbation data is often characterized by partial intervention: only a small subset of genes is perturbed, while large portions of genes remain invariant, as mentioned above. This mismatch between theoretical assumptions and available perturbation data in practice encourages invariant background information to be absorbed into perturbation-related (i.e., causal) representations. This may constitute perturbation suppression, whereby invariant information may be mixed into perturbation-related representations when such separation is not explicitly enforced. As a result, constructing causal models on such mixed representations may be inaccurate, leading to limited generalization ability. Implications for FMs. FMs for single-cell transcriptomics are typically trained to learn generic representations by emphasizing fitting the overall data distribution. Given the dominance of perturbation-invariant information, such objectives naturally prioritize encoding shared, perturbation-invariant structure. As a consequence, perturbation-specific signals, which occupy only a small fraction of the expression space, may become less accessible in the learned representations. To empirically assess this effect, we conduct a linear probing experiment on frozen representations learned by several FMs. Specifically, we treat perturbation conditions as surrogate labels (i.e., variable \mathbf{u} defined in Sec. 3.1 ) and evaluate whether they are linearly decodable from the learned representations, a standard diagnostic for assessing what information is encoded in an accessible form. We compare representations from UCE ( 58 )