Predicting gene essentiality and drug response from preclinical perturbation screens with layered ensemble of autoencoders and predictors Authors: Barbara Bodinier, Gaetan Dissez, Lucile Ter-Minassian, Linus Bleistein, Roberta Codato, John Klein, Eric Durand, Antonin Dauvin Published: 2026 ABSTRACT High-throughput preclinical perturbation screens, where the effects of genetic, chemical, or environmental perturbations are systematically tested on disease models, hold significant promise for machine learning-enhanced drug discovery due to their scale and causal nature. Predictive models trained on such datasets can be used to (i) infer perturbation response for previously untested disease models, and (ii) characterise the biological context that affects perturbation response. Existing predictive models suffer from limited reproducibility, generalisability and interpretability. To address these issues, we introduce a framework of Layered Ensemble of Autoencoders and Predictors (LEAP), a general and flexible ensemble strategy to aggregate predictions from multiple regressors trained using diverse gene expression representation models. LEAP consistently improves prediction performances in unscreened cell lines across modelling strategies (increase in Spearman’s correlation ranging from 1.4% to 4.4%). In particular, LEAP applied to perturbation-specific LASSO regressors (PS-LASSO) provides a favorable balance between near state-of-the-art performance (Spearman’s correlation of 0.321 for gene essentiality prediction) and low computation time. We also propose an interpretability approach combining model distillation and stability selection to identify important biological pathways for perturbation response prediction in LEAP. Our models have the potential to accelerate the drug discovery pipeline by guiding the prioritisation of preclinical experiments and providing insights into the biological mechanisms involved in perturbation response. The code and datasets used in this work are publicly available.Supplementary InformationThe online version contains supplementary material available at 10.1038/s41598-026-56381-0. Introduction Gene and drug perturbation screens have transformed preclinical studies by enabling the systematic evaluation of a vast number of perturbations across diverse biological contexts (e.g. different cancer cell lines), helping the identification and positioning of novel therapeutic targets and compounds1–4. These datasets are particularly valuable as the perturbations—whether genetic or chemical—are applied in a controlled experimental setting, allowing for direct observation of cause-and-effect relationships. In oncology, significant initiatives have been undertaken to create and publicly share datasets of unprecedented scale. The Cancer Dependency Map (DepMap) project5, for instance, contains data on the essentiality of over 17,000 genes in more than 1,000 cancer cell lines obtained from high-throughput CRISPR-Cas9 viability screens. Gene essentiality captures the change in cell viability following targeted CRISPR-Cas9 knockout. Essential genes are critical for cell survival and proliferation in specific molecular contexts. Similarly, studies like GDSC6, CCLE7, CTRP8, and PRISM9 have measured the impact of various compounds on cancer cell line viability. Drug response experiments involve exposing cancer cell lines to varying concentrations of compounds and quantifying cell viability. These studies also gather deep molecular characterization of the cell lines, including RNA sequencing to define the biological context. This enables the investigation of how molecular context influences cancer vulnerability and paves the way for more personalized medicine. In this paper, we train machine learning models to predict viability responses to perturbations from pre-perturbation molecular profiles of the disease models in which they were tested. These approaches can be used to predict the impact of a perturbation in untested disease models10,11, thus providing insights into the relevance of targets or compounds in different indications or disease models. This capability can aid in prioritizing targets, compounds, or preclinical experiments in the early stages of the drug discovery pipeline. Previous studies have proposed machine learning approaches for the prediction of gene essentiality10–12 or drug response with promising performances13–18. Existing approaches can be classified into two categories: perturbation-specific (PS) and pan-perturbation (PP) models. Perturbation-specific models11,13–15 aim to predict the response of different biological systems to a single perturbation, using molecular data as input. When data includes multiple perturbations, separate models are trained for each one. In contrast, pan-perturbation models10,12,16–18 are trained on data from multiple perturbations simultaneously. In this paper, a pan-perturbation model is a single-label regressor10,12,16–18 trained on all available pairs of disease models and perturbations. A PP model leverages features from both the disease model (molecular data) and the perturbations, which may be represented by pathway-based fingerprints for a genetic perturbation or by the molecular structure for a drug. Note that a multi-label regressor predicting the response to multiple perturbations from the molecular profiles of the disease models may also be considered as a pan-perturbation model, but such models are omitted from the present paper as they have not been studied as extensively. Pan-perturbation models have the potential to uncover both mechanisms that are specific to a perturbation and those that are shared across perturbations. Unlike perturbation-specific models, pan-perturbation models leveraging perturbation descriptors can intrinsically predict responses to untested perturbations, provided these can be appropriately captured by their embeddings. However, in this paper, we focus on predictions of responses to seen perturbations in unscreened disease models, such as new cell lines, tissues, or more complex disease models like patient-derived xenografts (PDX). This ability to infer perturbation responses in diverse disease models remains a key need in drug discovery. Existing perturbation-specific models have been trained using a variety of algorithms including both linear (e.g. Ridge, LASSO, Elastic Net) and non-linear (e.g. neural networks, XGBoost, Random Forests) regressors19. Pan-perturbation methods such as DeepDep10 for gene essentiality prediction, or tDNN18 and DrugCell17 for drug response prediction combine representations of disease models and perturbations. Previous work from our team, OmicsRPZ12, improved gene essentiality predictions by first generating these two separate representations and then training a Light Gradient Boosting Model (LGBM) on these combined features. We leverage our previous work on representation learning, but focus here on the choice of a prediction model and the development of a novel ensemble framework. The present work aims to improve the robustness, interpretability and prediction performances of models predicting perturbation responses in unscreened cell lines. We propose Layered Ensemble of Autoencoders and Predictors (LEAP), a novel and general ensemble approach that can be applied to both perturbation-specific or pan-perturbation models to predict perturbation response using gene expression data. LEAP uses an ensemble of regressors trained in randomized subsets of cell lines embedded with different representation models. We apply the LEAP framework using representations obtained from Masked Autoencoders (MAE) that are randomly initialized with different seeds, as they were previously identified as the best gene expression representation models for such tasks in previous work12. We pre-train the autoencoders using unlabelled data on various disease models (cell lines and PDX) to capture common patterns across biological contexts and reduce computational time. The novelty in LEAP lies in the integration of two established techniques: (i) recent advances in deep representation learning, which have achieved state-of-the-art results in various domains, and (ii) model ensembling, which enhances prediction precision and robustness by utilizing model randomized diversity20–22. To facilitate the biological interpretation of LEAP, we also introduce a pathway-based explanation framework that overcomes the challenges posed by the high-dimensionality and strong correlations patterns that are inherent to gene expression data. For this, we create Reactome23 pathway-level features constructed from linear combinations of genes that belong to the same pathway24. We then distill LEAP by supervising the training of LASSO regressors using these pathway-level features as input11. The sharp stability selection method25, designed specifically for molecular feature selection in high dimensional settings, enables the identification of stable sets of biological pathways that drive LEAP predictions. We evaluate the performance of LEAP for gene essentiality and drug response prediction in unscreened cell lines, tissues or disease models by adapting the setup previously used in the DeepDep10 and OmicsRPZ12 studies. We use more recent releases of the datasets, and we standardize the training and evaluation framework between gene essentiality and drug response tasks. We quantify the increase in prediction performance by using LEAP with multiple modeling strategies, including perturbation-specific and pan-perturbation approaches with both linear and non-linear regressors. In line with previous publications15,26–29, we consider that drug discovery operates within a transductive framework, which implies that unlabelled test data (e.g. RNAseq profiles) are available during training. This can be leveraged by pretraining the representation model on both labelled and unlabelled data with limited impact on the prediction performances, as demonstrated in our ablation studies. In this paper, we pre-train MAE representation models on all available RNAseq data and compare prediction performances obtained using different regression approaches on the same data representation. To evaluate the ability of our interpretability strategy to recover important biological pathways for prediction, we compare prediction performances obtained with LEAP and with a LASSO regressor using selected pathway-level features only. This study aims to (i) introduce a novel ensembling framework to improve robustness in predicting perturbation response in unscreened cell lines, (ii) benchmark different regression approaches on these tasks, and (iii) identify biological pathways driving these predictions. We show that LEAP improves performances in predicting perturbation response in unlabeled biological contexts regardless of the modeling strategy. Notably, we show in an ablation study that the ensemble of different data representations in LEAP outperforms methods that ensemble prediction models alone. We identify that LEAP applied to perturbation-specific LASSO regressors offers a good balance between near state-of-the-art performance and low computational burden. Human evaluation for the top predicted compounds confirms plausible links between some of the pathways driving predictions in LEAP and known mechanisms of actions for these drugs. Results Outline We propose LEAP, a novel approach to predict perturbation response (gene essentiality or drug response) in a disease model based on gene expression before perturbation (Fig. 1). We use publicly available data from perturbation experiments conducted in cell lines in the DepMap5 and PharmacoDB30 studies, and in Patient-Derived Xenografts (PDX) in the PDX Encyclopedia31(Supplementary Table 1, Supplementary Fig. 1). We consider three challenges, where we aim to predict perturbation response in (i) unscreened cell lines, (ii) cell lines from an unscreened tissue of origin, or (iii) unscreened PDX (Table 1). Fig. 1Illustration of LEAP. The unlabelled RNAseq data is first transformed using R MAE models that have been trained with different random seeds (Step 1). We use -fold cross validation on each of the five representations of the data and ensemble the K regressors trained on the different cross validation folds with the best performing hyper-parameters (Step 2). In Step 3, predictions from the K x R regression models are averaged to get predictions from the final LEAP model. Throughout the paper, we use R = 5 and K = 5. Illustration of LEAP. The unlabelled RNAseq data is first transformed using R MAE models that have been trained with different random seeds (Step 1). We use -fold cross validation on each of the five representations of the data and ensemble the K regressors trained on the different cross validation folds with the best performing hyper-parameters (Step 2). In Step 3, predictions from the K x R regression models are averaged to get predictions from the final LEAP model. Throughout the paper, we use R = 5 and K = 5. Table 1Task description. For each of our six tasks, we report the type of label to predict (gene dependency or drug response), dataset version, number of cell lines, number of perturbations and number of data points (pairs of cell lines and perturbations). Task 1’ is a similar task to task 1, using an older version of the data and a different set of perturbations in order to compare our approach with previous work.ChallengeTask idLabelDataNumber of cell linesNumber of perturbationsNumber of (cell line, perturbation) pairsi - Prediction in unscreened cell lines1Gene dependencyDepMap (version 23Q4)1,0191,5391,568,2221’Gene dependencyDepMap (OmicsRPZ12)8931,2231,092,1392Drug response (AAC)PharmacoDB1,399649551,024ii - Prediction in unscreened tissue of origin3Gene dependencyDepMap (version 23Q4)1,0191,5391,568,2224Drug response (AAC)PharmacoDB1,399580528,463iii - Prediction in unscreened disease model5Drug response (AAC)PharmacoDB, PDX Encyclopedia1,390(140 PDX)56,129(227 PDX) Table 1Task description. For each of our six tasks, we report the type of label to predict (gene dependency or drug response), dataset version, number of cell lines, number of perturbations and number of data points (pairs of cell lines and perturbations). Task 1’ is a similar task to task 1, using an older version of the data and a different set of perturbations in order to compare our approach with previous work.ChallengeTask idLabelDataNumber of cell linesNumber of perturbationsNumber of (cell line, perturbation) pairsi - Prediction in unscreened cell lines1Gene dependencyDepMap (version 23Q4)1,0191,5391,568,2221’Gene dependencyDepMap (OmicsRPZ12)8931,2231,092,1392Drug response (AAC)PharmacoDB1,399649551,024ii - Prediction in unscreened tissue of origin3Gene dependencyDepMap (version 23Q4)1,0191,5391,568,2224Drug response (AAC)PharmacoDB1,399580528,463iii - Prediction in unscreened disease model5Drug response (AAC)PharmacoDB, PDX Encyclopedia1,390(140 PDX)56,129(227 PDX) Task description. For each of our six tasks, we report the type of label to predict (gene dependency or drug response), dataset version, number of cell lines, number of perturbations and number of data points (pairs of cell lines and perturbations). Task 1’ is a similar task to task 1, using an older version of the data and a different set of perturbations in order to compare our approach with previous work. 1,390 (140 PDX) 6,129 (227 PDX) We propose a modelling strategy that builds upon our previous pan-perturbation approach OmicsRPZ12, which proposed SOTA models to predict gene essentiality alongside a benchmark strategy. We use the Masked Auto-Encoder (MAE) identified in this previous work as one of the best approaches across tasks. We apply the LEAP framework to different prediction algorithms: three pan-perturbation (PP) models, including PP-LGBM, which is an update of the best model from OmicsRPZ using our MAE model, PP-tDNN18 and PP-MLP, and four perturbation-specific (PS) models, including the baseline PS-KNN, PS-MLP, PS-LGBM and PS-LASSO (Supplementary Table 2). In LEAP, we design a two-layer ensembling strategy aimed at enhancing the robustness of both the data representation and the prediction model (Fig. 1). We train five MAE models with different random initializations on the same large dataset combining RNAseq data from all available cell lines and PDX samples. We then train regression models on the five different data representations generated by the MAEs. We tune the hyper-parameters of each regressor by maximizing Spearman’s correlation using a 5-fold cross-validation within the training set. For the first layer of ensembling, we average the predictions from the five regression models that are trained for hyper-parameter tuning with the best performing hyperparameters. For the second layer of ensembling, we average the predictions of the five sets of regressors obtained with different data representations. In total, LEAP ensembles 25 regression models. In our approach, the randomization of the initial seeds of the autoencoder generates embeddings that capture complementary signals from RNAseq data. The second layer of ensembling leverages these complementary signals and yields better prediction performance. We quantify the change in prediction performance when applying LEAP to each of these prediction algorithms compared to base algorithms with the same hyper-parameter tuning and only the first layer of ensembling. We further compare our approach with published models, including DeepDep10 and all models trained in the OmicsRPZ12 study on the gene dependency prediction in unlabeled cell lines (challenge i, task 1’). Finally, we extract the most important biological pathways to predict perturbation response in LEAP. For this, we create an expression dataset of lower dimensionality with 1,832 Reactome pathway features, each defined as the first principal component of a PCA applied on all genes in the pathway. We then train LASSO regressors using pathway-level features to predict labels inferred by LEAP and use stability selection to identify important pathways. LEAP consistently improves perturbation response predictions compared to the base model Applying LEAP consistently yields an increase in prediction performance compared to the base algorithm for all modeling approaches in tasks 1 and 2 (increase ranging from 1.4% for PS-KNN in task 2 to 4.4% for PP-tDNN in task 1, Fig. 2A-B, Supplementary Table 3). For all modelling approaches, LEAP outperforms the corresponding base model for all 10 training/test splits. In particular, LEAP applied to PS-LASSO (Spearman’s correlations of 0.321 in task 1 and 0.350 in task 2), PS-MLP (0.307 in task 1 and 0.346 in task 2) and PP-LGBM (0.309 in task 1 and 0.367 in task 2) regressors stand out as the best performing approaches. LEAP (PS-LASSO) provides the best balance between prediction performance and computation time (approximately 30x and 150x faster than PS-MLP and PP-LGBM respectively on task 1, 45x and 340x faster on task 2, Supplementary Fig. 2). Fig. 2Performances in predicting perturbation response in unscreened cell lines. We evaluate the performances in predicting A gene dependency in unscreened cell lines, B drug response in unscreened cell lines, and C drug response in unscreened cell lines from an external validation study (PRISM). Performances are evaluated over 10 repeated holdout splits (A, B) or over the entire external validation dataset for models trained on the 10 same training sets (C). We calculate the average per-perturbation Spearman’s correlation over all perturbations, and report the average (data point) and standard deviation (bar) of this metric over the 10 test sets. We consider four perturbation-specific (PS) approaches with different regression algorithms, including KNN, MLP, LGBM and LASSO regression, and three pan-perturbation (PP) approaches using tDNN, MLP, and LGBM. We compare the performances of our novel approach LEAP applied to these different models (LEAP, filled data points) with base models (empty data points). Base models only include the first layer of ensembling, where predictions obtained from regressors trained in each the 5 CV folds are averaged. Performances in predicting perturbation response in unscreened cell lines. We evaluate the performances in predicting A gene dependency in unscreened cell lines, B drug response in unscreened cell lines, and C drug response in unscreened cell lines from an external validation study (PRISM). Performances are evaluated over 10 repeated holdout splits (A, B) or over the entire external validation dataset for models trained on the 10 same training sets (C). We calculate the average per-perturbation Spearman’s correlation over all perturbations, and report the average (data point) and standard deviation (bar) of this metric over the 10 test sets. We consider four perturbation-specific (PS) approaches with different regression algorithms, including KNN, MLP, LGBM and LASSO regression, and three pan-perturbation (PP) approaches using tDNN, MLP, and LGBM. We compare the performances of our novel approach LEAP applied to these different models (LEAP, filled data points) with base models (empty data points). Base models only include the first layer of ensembling, where predictions obtained from regressors trained in each the 5 CV folds are averaged. LEAP (PS-LASSO) generates better performances than a baseline approach using target expression only in task 1, which confirms that this model can capture more complex predictive signals from multiple genes (Supplementary Fig. 3). LEAP (PS-LASSO) also outperforms all other models in the external validation study (Fig. 2C). The prediction performances obtained with LEAP (PS-LASSO) are close to the drug response measurement reproducibility across studies (Supplementary Fig. 4). The variability in prediction performance across perturbations is correlated with the standard deviation of the labels in both tasks 1 and 2 (Supplementary Fig. 5). LEAP (PS-LASSO) and LEAP (PS-MLP) also generalise better to unscreened tissues in tasks 3 and 4 (Supplementary Fig. 6). We observe an increase in prediction performance when using LEAP on top of PS-LASSO approaches for all tissues (Supplementary Fig. 7). These results demonstrate that our LEAP framework improves performances regardless of the choice of prediction heads. LEAP also yields an increase in performance over PS-LASSO models for 3 of the 5 drugs that are available in our task 5 in PDX (Supplementary Fig. 8, Supplementary Table 4). Although these results are encouraging, more validation data is needed to be able to conclude on the ability of such models to generalise to disease models that are more representative of patients. The ensembling framework improves performances in LEAP We evaluate the impact of each of our methodological choices on the prediction performance in an ablation study (Supplementary Fig. 9, Supplementary Table 5). We observe a small drop in prediction performance when pre-training the MAE representation model in all available DepMap cell lines instead of cell lines used in the training set only. Pre-training the MAE representation models on (i) DepMap only, (ii) DepMap and GDSC, or (iii) DepMap, GDSC and PDXE only yields limited changes in prediction performances. We distinctly evaluate the gain in performance due to the first and second layers of our ensemble learning strategy in this ablation study (Supplementary Table 5, Supplementary Fig. 9). The ensembling of the five regressors trained on the splits of the 5-fold cross validation (first layer ensembling) yields an increase in performance of 1% (0.306 to 0.309 and 0.335 to 0.339 respectively in tasks 1 and 2) compared to the use of a single regressor trained on the full training set. Further ensembling the 25 regressors trained on 5 different data representations (second layer ensembling) yields a larger 3 to 4% (0.309 to 0.321 and 0.339 to 0.350 respectively in tasks 1 and 2) increase in performance (Supplementary Figs. 9 and 10). To further assess the relevance of the second layer ensembling, we compare the performance of LEAP with an ensemble of 25 LASSO regressions trained on different subsets of the same deep representation (Supplementary Table 6). Increasing the number of regressors that are fitted on the same data representation from 5 to 25 yields an increase in prediction performance of 0.005 in task 1 and 0.004 in task 2. The larger gain in performance when using LEAP (0.012 in task 1 and 0.011 in task 2) is then due to the use of multiple data representations in the second layer ensembling, and not only to the larger number of regressors that are ensembled. LEAP (PS-LASSO) outperforms state-of-the-art approaches in unscreened cell lines We compare LEAP (PS-LASSO) to previous approaches for the prediction of gene essentiality in cell lines using the same data and evaluation framework as in OmicsRPZ12. LEAP (PS-LASSO) outperforms all approaches proposed in OmicsRPZ with different representation models and yields an increase in average per-perturbation Spearman’s correlation of 16% (0.256 to 0.297) compared to the best OmicsRPZ model (Fig. 3, Supplementary Table 7). LEAP’s predictions are driven by stable, biologically relevant pathway features To enhance the biological interpretability of LEAP, we develop a pathway-based explanation framework that distills LEAP into stable pathway signatures. For each dependency or drug, we identify a subset of Reactome pathways predictive of LEAP outcomes (PS-LASSO) using a stability selection25 approach applied to pathway-level expression profiles. This results in perturbation-specific lists of biological pathways. Our distillation method generates stable results, with stability scores around 0.81 for task 1 and 0.82 for task 2 (Supplementary Fig. 11, Supplementary Tables 8 and 9). We observe strong correlations between the prediction performances obtained using (i) selected pathways only as input in a LASSO regressor, or (ii) LEAP (PS-LASSO) for both tasks 1 and 2 (Fig. 4). This result validates the faithfulness of the distilled model to the teacher model. In addition, we report an average decrease of the performance in predicting true labels of only 0.039 in task 1 and 0.030 in task 2 when using selected pathways only compared to LEAP (PS-LASSO). This small decrease in prediction performance compared to the teacher model suggests that our interpretability strategy successfully pinpoints the most important biological pathways to predict perturbation response. Fig. 3Comparison of the performances in predicting gene dependency in unscreened cell lines using LEAP (PS-LASSO) or existing approaches (task 1’). We compare the performance of our approach LEAP (PS-LASSO) with previously reported performances obtained in DeepDEP10and using different representation models in OmicsRPZ12. Models are evaluated for the 1,223 gene perturbations included in the OmicsRPZ study over 10 repeated holdout test sets of unscreened cell lines. (A) Boxplots for different models where each point indicates the average Spearman’s correlation coefficient over the 1,223 gene perturbations obtained in each of the 10 test sets. (B) Per-perturbation performances obtained using the best model from OmicsRPZ (x-axis) or using our model (y-axis) are compared. We calculate the average per-perturbation Spearman’s correlation over all perturbations, and report the average (data point) and standard deviation (bar) of this metric over the 10 test sets. Note that the performances of OmicsRPZ model are not recomputed and were obtained with different training-test splits of the same dataset. Comparison of the performances in predicting gene dependency in unscreened cell lines using LEAP (PS-LASSO) or existing approaches (task 1’). We compare the performance of our approach LEAP (PS-LASSO) with previously reported performances obtained in DeepDEP10and using different representation models in OmicsRPZ12. Models are evaluated for the 1,223 gene perturbations included in the OmicsRPZ study over 10 repeated holdout test sets of unscreened cell lines. (A) Boxplots for different models where each point indicates the average Spearman’s correlation coefficient over the 1,223 gene perturbations obtained in each of the 10 test sets. (B) Per-perturbation performances obtained using the best model from OmicsRPZ (x-axis) or using our model (y-axis) are compared. We calculate the average per-perturbation Spearman’s correlation over all perturbations, and report the average (data point) and standard deviation (bar) of this metric over the 10 test sets. Note that the performances of OmicsRPZ model are not recomputed and were obtained with different training-test splits of the same dataset. To further evaluate this, we analyse the main Reactome pathways selected by our model for some of the top predicted compounds (Supplementary Table 10). Overall, our model successfully identified pathways with a clear and plausible link to the drug’s known mechanism of action. These plausible links include pathways directly involved in the known mechanism of action of the drug or expected downstream cellular responses, based on the biological function of the target. For example, the pathway “Signaling by MAPK mutants” was successfully selected by our model for refametinib and SCH772984, which are both known to target key kinases of the MAPK signaling pathway. Similarly, analysis of the pathways selected for the PLK1 inhibitor GSK-461,364 highlights a significant involvement in the ‘breakdown of the nuclear lamina’. This is highly consistent with PLK1’s regulatory role in mitotic cell division, a process intrinsically linked to the precise and controlled disassembly of the nuclear lamina. In contrast, the identified pathways for triazolothiadiazine were plausible but less direct. This could reflect a wider spectrum of mechanisms for this compound, which is known to interact with various biological targets and activities. Overall, these results indicate that LEAP predictions for drug response are grounded in biologically plausible mechanisms. Fig. 4Evaluation of distillation performance. We compare the Spearman’s correlation between true labels and predicted values obtained from LEAP (PS-LASSO) (y-axis) or PS-LASSO regressors fitted using stably selected biological pathways only as input (x-axis). Each data point represents one gene (task 1, A) or drug (task 2, B). The Spearman’s correlations are computed in the 10 test sets available in tasks 1 and 2, respectively. For perturbations that are present in multiple test sets, we report the average Spearman’s correlation between true and predicted labels over the available test sets. Evaluation of distillation performance. We compare the Spearman’s correlation between true labels and predicted values obtained from LEAP (PS-LASSO) (y-axis) or PS-LASSO regressors fitted using stably selected biological pathways only as input (x-axis). Each data point represents one gene (task 1, A) or drug (task 2, B). The Spearman’s correlations are computed in the 10 test sets available in tasks 1 and 2, respectively. For perturbations that are present in multiple test sets, we report the average Spearman’s correlation between true and predicted labels over the available test sets. Discussion Identifying disease models or patients who are likely to respond to a given treatment is a critical need in drug discovery. In this paper, we measure prediction performance with per-perturbation Spearman’s correlation to evaluate the ability of the model to rank disease models by their response to a given perturbation. Our model, LEAP (PS-LASSO) outperforms previous approaches in unlabeled cell lines, including DeepDep (72% increase in average Spearman’s correlation in task 1’, although DeepDep’s predicted values were obtained from a previous publication and present a risk of leakage which may favor DeepDep due to potential overlap between the training set used in the previous study and the test set used in the current study). Although pan-perturbation models take additional information as input (perturbation fingerprints), they do not always outperform their perturbation-specific counterparts. We hypothesise that this could be due to (i) the use of a global loss discriminating more between perturbations than between samples, and (ii) a more challenging tuning. In particular, one pan-perturbation model requires much more parameters than one per-perturbation model, and the optimal hyper-parameter set must fit all perturbations and cannot be optimized for each perturbation. In practice, perturbation-specific models are also less computationally expensive and more scalable than pan-perturbation models. For instance, the LEAP (PS-LASSO) approach takes approximately 25 min to train for a single perturbation (including 25 min on ml.g4dn.12xlarge to pre-train the 5 MAE, which can be leveraged across multiple perturbations, and 5 s on ml.m6i.32xlarge to fit and tune the 25 perturbation-specific LASSO regressors on 10 training sets). The prediction performance obtained with LEAP (PS-LASSO) in an external validation study (average Spearman’s correlation of 0.199 in PRISM) is close to the label measurement reproducibility (average Spearman’s correlation of 0.220 between drug responses measured in our training studies and in PRISM), which suggests that our model captures most of the predictive signal. The observed Spearman’s correlation in drug response between our training studies (GDSC, CTRP and CCLE) and PRISM remains low, despite the corrections for technical differences between studies implemented in the R package PharmacoGx. Previous publications32,33 also commented on this low measurement reproducibility and reported Pearson’s correlations between measured responses in different studies varying between 0 and 0.8 depending on the drug and studies. We demonstrate the use of LEAP with five MAE models, each trained on all available unlabelled samples to transform the data before training the regression models. Pre-training the representation models is attractive for two reasons. First, the use of pre-trained representation models instead of training the representation model on each training set reduces the computational time. Second, the representation model can be trained on a larger and more comprehensive RNAseq dataset. For this, we leverage a larger publicly available RNAseq data from cell lines and PDX models that did not necessarily have perturbation responses available. In our evaluation framework, the label (perturbation response) from the test set is unseen, but the features (RNAseq) are seen during representation model training. Given new RNAseq data where perturbation responses are unknown, our unsupervised MAE models could be retrained on all available RNAseq data, including both labelled and unlabelled samples15. We demonstrated in our ablation studies that pretraining the representation model on both labelled and unlabelled data (transductive framework) does not inflate prediction performances compared to a purely inductive framework with representation models fitted on training examples only. This pre-training may actually slightly deteriorate prediction performances (Spearman’s correlations of 0.310 with pre-training compared to 0.306 without in Task 1). We hypothesize that this small decrease in performance is due to the fact that a MAE trained on all samples learns features that may be underrepresented in the training set and cannot be exploited by the regressors. In this paper, we chose to pre-train MAE representation models on all available RNAseq data to reduce computational burden, but prediction performances may be further increased by fitting representation models on training examples only. Our sensitivity analyses show that the ensembling of regression models trained on data representations obtained with multiple MAE models in LEAP generates an increase in performance compared to the ensembling of the same number of regressors trained on a single representation. This suggests that the stochasticity of deep learning representation models can provide robustness and capture complementary patterns. Our representation learning ensemble framework may be beneficial in other machine learning tasks, including for instance predictions from histology slides. We identify important Reactome pathways in LEAP using a novel interpretability framework based on stability selection for model distillation11,25. Our selected pathways are stable and yield prediction performances that are close to that of the teacher model. Human review of these selected pathways suggests plausible implications based on known mechanisms of actions of the drugs. Beyond biological validation, our interpretability framework could enable a better understanding of the biological mechanisms involved in perturbation response, including the identification of alternative pathways potentially involved in resistance mechanisms34. Several research directions can be explored to improve on our work. First, the hyper-parameter tuning is based on grid search using restrictive grids for all approaches and some of the hyper-parameter choices could be further explored (e.g. dimensionality in MAE). This could be improved in future work by expanding the hyper-parameter search space and using a better optimized parameter sampling algorithm, provided that the computational budget is increased. The use of limited grids of hyper-parameters may favour perturbation-specific models, which allow for different sets of hyper-parameters per perturbation. Similarly, parameters that relate to the processing and representation of gene expression data could be further tuned for the specific task of perturbation response: representation dimension, normalisation, scaling, gene selection. For those parameters, we adhere with parameters selected in previous work12. Second, we use a simple ensembling strategy by calculating the unweighted average of predictions from multiple regressors in LEAP. Performances could potentially be improved by using a weighted average based on the performances of individual regressors, more complex ensembling techniques (e.g. routing or boosting), or a different number of ensembled models. Third, we use established perturbation fingerprints for the pan-perturbation models. Perfecting these representations may improve the performances of pan-perturbation models. Fourth, even if the hyper-parameter selection of pan-perturbation models is dictated by per-perturbation metrics, their loss and consequently their optimization rely on a global loss over all perturbations and preclinical models, which may be detrimental to their final performance. Fifth, all models presented in the present paper only use gene expression profiles as input. Although previous work suggested that gene expression is the most informative modality to predict perturbation response10, additional data modalities, including mutations and protein expression, are available and could be leveraged to better characterize the cell lines. Sixth, there are known intrinsic differences between the molecular profiles of cell lines and PDXs. In particular, previous work suggests that the gene expression in PDX is closer to the one in tumors than the gene expression in cell lines, as cell lines suffer from the lack of cell diversity, tumor micro-environment, and sometimes high passage numbers35. Although we aim at exploring the generalization capabilities of LEAP by evaluating models trained on cell lines data on PDX, those known differences may partially explain the lower performances on this task. Finally, while the biological pathways that we have identified as predictive of perturbation response are supported by correlation patterns and existing literature, our interpretability pipeline may benefit from experimental validation. Future work could also focus on applying LEAP to transfer learning approaches to improve prediction performances in out-of-domain data, including in held-out tissues, disease models or even patients. In particular, the LEAP framework could be applied to other models that are designed to generalize beyond the training context, including (i) TCRP11, which uses a model-agnostic meta-learning strategy to pre-train neural networks that can be quickly adapted to new tasks with limited data (few shots), or (ii) CodeAE15, which employs an autoencoder to align disease-model molecular data with patient data while handling potential confounding factors. Our method could also be adapted to predict binary labels15. This may improve robustness to noise in the labels and allow to learn common patterns across perturbation responses with different levels of granularity in cell lines, PDX and patients (e.g. from AAC to RECIST scores). If successful, we anticipate that such prediction models trained in preclinical data and adapted to a clinical setting could be instrumental in better prioritising patients and indications to include in early phases of clinical trials when patient response data is still scarce. Such extensions could ultimately improve the likelihood of clinical success by targeting patients who are more likely to respond well to the treatment. Conclusion We propose LEAP, a new framework ensembling regressors trained over multiple deep representations of expression profiles. Our framework consistently improves performances for all prediction algorithms and tasks that we consider. Our layer ensembling strategy yields an increase in prediction performance beyond ensembling the same number of regressors fitted on the same representation of the data. Our benchmark of multiple regression methods in both pan-perturbation and perturbation-specific settings reveals limited added value of pan-perturbation models for predictions of responses to seen perturbations in unscreened cell lines. In particular, the use of perturbation-specific LASSO (PS-LASSO) strikes an attractive balance between computation time and prediction performance. LEAP (PS-LASSO) outperforms state-of-the-art models for gene essentiality and drug response prediction in unscreened cell lines. Its lower computational burden and minimal tuning requirements facilitate broader and more effective applications in the prediction of perturbation response. We demonstrate that LEAP is interpretable and captures plausible biological signals using stability selection for model distillation. By generating more accurate, robust and biologically interpretable predictions of perturbation effects in cell lines, we believe that our approach could provide valuable insights in preclinical development of anti-cancer therapies. For instance, LEAP (PS-LASSO) could readily be used to prioritize wet lab validation experiments by predicting perturbation response in unscreened cell lines, potentially from other indications, and identifying the key biological pathways driving the predictions. Future work should focus on improving and validating the clinical relevance of such prediction models. We make our code and evaluation pipeline available for future research. Materials and methods Datasets This study is based on publicly available data from DepMap, PharmacoDB and PDX Encyclopedia (Supplementary Table 1). DepMap CRISPR knockout screens on human cancer cell lines and related omics datasets are downloaded from DepMap 23Q4. Perturbation screens’ outcomes are obtained from the project Achilles36, and processed by DepMap5 through a pipeline37,38 using Chronos39 on each screen library to output gene effect scores. For every studied gene representing a dependency of interest, gene effect scores are then aggregated and corrected per cell line to generate gene dependency scores. Both scores are available on the DepMap platform. Our study focuses on the gene dependency score as it measures the probability that a gene is essential for the survival of a particular cell line. Gene expression data is obtained from CCLE40 via the DepMap portal. We use the publicly available transcripts per million (TPM) normalized counts as cell lines descriptor inputs for our models. For task 1’ and comparison with OmicsRPZ12, gene dependency data and TPM gene expression data are downloaded from DepMap 22Q441. Perturbation outcomes and omics originate from the same sources and pipelines, without corrections added in 23Q4, but do not cover as many cell lines as in later versions. PharmacoDB Drug perturbation screens on human cancer cell lines are downloaded from PharmacoDB30 using the R package PharmacoGX (version 3.4.0)42. From the 15 studies that are accessible through PharmacoGX, we retain CCLE-2015, GDSC-2020 (v1-8.2 and v2-8.2), CTRPv2-2015 and PRISM-2020 which are respectively referred to as CCLE, GDSC (v1 and v2), CTRP and PRISM. The tested drugs (Supplementary Table 1) cover a wide range of anti-cancer drugs including cytotoxic agents (paclitaxel, bendamustine) and targeted therapies (erlotinib, everolimus, ruxolitinib), and range from experimental molecules to clinically-approved drugs. For the different learning tasks, we use the area above the dose-response curve (AAC) which is measured on a drug-specific dose range. A high AAC shows a high average efficacy. Gene expression data before perturbation is downloaded with PharmacoGX when available, which is only the case for some of the cell lines studied in GDSC (v1 and v2) amongst the retained studies. For other studies or cell lines without gene expression data in GDSC (v1 and v2), we use the gene expression data from DepMap 23Q4. PDX encyclopedia Drug response screens in patient-derived xenografts (PDX) are obtained from PDX Encyclopedia31. Of the 63 available perturbations, we restrict our analyses to the five mono-therapy drugs that were studied previously11: Paclitaxel, Cetuximab, Erlotinib, Tamoxifen and Trametinib. Drug response is provided as the minimal percentage of tumor volume change observed over 10 days of treatment. We download available gene expression raw counts. Perturbation fingerprints In the pan-perturbation model approach, perturbations’ encodings are required as a unique model learns and predicts the outcome of diverse perturbations. In practice, such models use gene and drug biological or molecular descriptors (also called fingerprints) to better capture similarities between perturbations43. Gene fingerprints We use gene fingerprints derived from the Molecular Signatures Database (MSigDBv6.244) that describe the affiliation of each gene (dependency of interest) in gene sets that are up or down-regulated in response to a perturbation experiment. Most of those signatures are curated from the literature. Gene perturbations with similar fingerprints tend to be activated or repressed together in perturbation experiments included in the Molecular Signatures Database. We use the same 3115 chemical and genetic perturbations (CGP) curated gene sets as in DeepDEP10, which were obtained after filtering out the 318 sets that do not involve any of the 1298 genes studied in their paper. Drug fingerprints Drug fingerprints are solely based on their molecular composition. Every drug perturbation name obtained from PharmacoDB30 or PDX Encyclopedia31 is mapped to its corresponding Compound Identifier (CID) on PubChem via the PubChem API45. Canonical Simplified Molecular Input Line Entry System (SMILES) are then downloaded from the compound card and further processed with the github repository CDDD46 and the RDKit47 package to remove salts and stereochemistry from the SMILES sequence. Additionally, we keep existing filters in the CDDD package that filter out molecules based on their molecular weight, number of heavy atoms or partition coefficient. Morgan fingerprints are then generated from SMILES with RDKit as vectors, using a radius of 2 and a size of 2048. As the SMILES are downloaded from PubChem, perturbations that are combinations of compounds or drugs that do not match to any CID (lack of matching terms, inexistence of a compound card for compounds in early development stages) cannot be encoded with fingerprints and are excluded from tasks that compare the performances of pan-perturbation versus perturbation-specific model approaches. Data preparation Sample and perturbation selection Overall, the data curation of DepMap5, PharmacoDB30 and PDX Encyclopedia31 is done in two steps where (i) we keep samples with available RNAseq data, information on the tissue of origin and exclude outliers, (ii) we select the perturbations that will be investigated (Supplementary Fig. 1). For PharmacoDB, we have an additional final step to remove duplicated sample-perturbation pairs arising when combining data from GDSC, CCLE and CTRP. DepMap For our main tasks, we use the version of DepMap released in Q4 of 2023 (version 23Q4). We drop 79 DepMap cell lines with no available RNAseq data (Supplementary Fig. 1A). Two outliers are then identified in the DepMap data by visually inspecting the first principal components obtained from a Principal Component Analysis (PCA) of the RNAseq data. After excluding these outliers, we define a new list of dependencies of interest with standard deviation of the gene dependency above 0.210, resulting in 1,539 dependencies. For comparison with previous studies, we also reproduce our analyses using a former version of the DepMap data in task 1’ using 893 cell lines and 1,223 dependencies of interest that were used in OmicsRPZ12. PharmacoDB We first prepare the data from each of the five retained studies individually (Supplementary Fig. 1B). We only keep the drugs with available fingerprints to allow for the comparison of different modeling approaches (see below). We then remove duplicated responses measured for the same drug in the same cell line across the five studies. More precisely, within each single study, when the same drug is tested multiple times on the same cell line, we aggregate the observations by taking their median response. This operation helps reduce uncertainty and noise in experiments performed in the same study. Later, if the same drug is tested on the same cell line in different studies, observed outcomes are not aggregated across those studies as experimental setups such as concentration ranges may vary. Therefore, we keep only one of the studies’ outcomes by taking the first appearance of the experiment based on an arbitrary order of the studies: GDSC v2, GDSC v1, CCLE, CTRP and PRISM. Unless a precise study is explicitly mentioned, we refer to the aggregation of the 4 datasets GDSC v1, GDSC v2, CCLE and CTRP as PharmacoDB. For all learning tasks, we only retain drugs that are administered to at least 75 cell lines. We use PRISM as an external validation set and retain drugs that are administered to at least 15 cell lines in this dataset. PDX Encyclopedia We keep the 140 PDX with available RNAseq and drug response data (Supplementary Fig. 1C). Of the 63 treatments tested in the study (including 25 combinations of drugs), we only keep the 5 drugs that were also administered to cell lines in the PharmacoDB dataset11. RNAseq data preprocessing We download TPM counts from DepMap5 and transform raw counts into TPM normalized counts using an in-house software package for GDSC6 and PDX Encyclopedia31. TPM counts are then log2-transformed with the addition of a pseudo count of 1. We keep the 16,926 genes available in all three studies and concatenate the samples from the three RNAseq datasets. To correct for study effects while preserving biological differences due to the tissue of origin, we apply combat48 as implemented in InMoose49 to log2-transformed TPM counts. We then filter the 5,000 most variant genes. We standardize the TPM counts, so that each gene expression has a mean of zero and a standard deviation of one. Tasks We define three types of challenges, where we aim to predict perturbation response in (i) held-out cell lines, (ii) an held-out tissue, or (iii) an held-out disease model (Table 1)11. Challenge I: prediction in unscreened cell lines The first challenge encompasses two tasks that focus on predicting the response of a new cell line to a perturbation (a gene knockout or a drug) already performed on other cell lines. From a clinical perspective, this challenge mimics a key approach in precision oncology: to determine if a new patient would respond to a therapy that was already tested in other patients. The first task consists in predicting gene dependency in cell lines using the DepMap5 dataset. For this, we train and evaluate models to predict the dependency of each of the 1,539 genes defined above using the 23Q4 release of the DepMap dataset that was curated as described above (task 1). For an exact comparison with OmicsRPZ12, we also implement an alternative version of this task using the same version of the DepMap dataset that was used in OmicsRPZ (task 1’) where we aim to predict the dependency of 1,223 genes. The second task focuses on the prediction of drug response (AAC) in cell lines using the PharmacoDB30 dataset (task 2). Following recent recommendations50, we focus on AAC instead of other measures like IC50. The models are trained and evaluated in GDSC v2 and v1, CTRP and CCLE, and we keep PRISM as an external test set. We predict the response to the 648 drugs available in the training set (GDSC, CTRP and CCLE) that are administered to at least 75 cell lines (Supplementary Fig. 1). The evaluation of performances in PRISM is restricted to the 254 drugs that are also included in the training data. To evaluate the prediction performance of the different modeling approaches, we use a repeated holdout cross-validation framework12. The data is split into a training set with 80% of the cell lines and a test set with the 20% remaining cell lines. This procedure is repeated 10 times to increase the robustness of the evaluation. Challenge II: prediction in an unscreened tissue The second challenge comprises two tasks that aim at inferring the response to perturbation (a gene knockout or a drug) in cell lines originating from a tissue that is not observed in the training set (Table 1). This challenge could be seen as a drug repurposing task, where we aim to evaluate if an existing drug might be effective for a new indication or a different subpopulation of patients. It is also valuable for target identification and validation, especially when genes or drugs have been screened across various cell lines, but not specifically in the tissue of interest. For instance, in DepMap5, only 24 cell lines are available as a proxy for mesothelioma, and only 3 of sarcomatoid type. These out-of-domain tasks are more arduous as models aim to predict in cell lines with tissue-related molecular specificities that are not seen at training time. For the gene dependency task 3, the out-of-domain capabilities of the different models are assessed on each of the 18 tissues with more than 25 cell lines available in the latest DepMap data, so that there is always a minimum of 10 cell lines available for few-shot learning, and 15 cell lines for computing a performance metric. For drug response, we first keep the 10 tissues with the largest sample size in the PharmacoDB30 data (task 4). The tissue selection slightly differs from task 3 as most drugs are not tested in all cell lines, which requires us to set a threshold on both the number of tissues and the number of cell lines tested for each drug within a tissue to select test cases. As a result, we use the 525 drugs that are tested in at least 25 cell lines for each of these 10 tissues (see supplemental material). We evaluate prediction performance in cell lines from the tissue of origin that was not used in the training set. To evaluate variability, we create 100 test sets by repeatedly removing 10 cell lines from the target tissue at random. We repeat this procedure such that each tissue is in turn considered as the target tissue. We report the tissue-specific performances (average of the per-perturbation performance on a given tissue over all perturbations and test sets) as well as the average performance over all held-out tissues. Challenge III: prediction in an unscreened disease model Challenge 3 aims to evaluate the ability of the models trained in cell lines to generalize to other disease models (Table 1). For this, we first train the models to predict drug response using all available cell lines in the full PharmacoDB30 dataset (task 5). These models are then used to predict the negative minimum change in tumor volume compared to the pre-treatment baseline over a 10-day post-treatment period measured in PDX treated with the same drug. To assess variability and be as comparable as possible to published results11, we evaluate prediction performance over 100 randomly sampled subsets of the PDX data obtained by removing 10 samples from the PDX data (the ones used in training in a few-shot setting) and keeping the remaining ones as a test set. Of note, the number of test samples depends on the perturbation, as all treatments are not measured in all PDX. Machine learning framework We aim to predict perturbation response (gene dependency or drug response) based on gene expression data. Some of our models additionally use perturbation-specific fingerprints as input. Overall modeling approach To predict perturbation response, we use a regression algorithm which takes a lower dimensional representation of the data as input. We compare two types of regression approaches: a perturbation-specific approach based on multiple models (PS) where independent regressors are trained to predict the response for each perturbation, and a pan-perturbation model approach (PP) where we predict the response to any (perturbation, disease model) pair based on the gene expression of the disease model and the fingerprints of the perturbation. Our approach can be decomposed into three steps where we (i) train a Masked Auto Encoder (MAE) to represent the RNAseq data, (ii) extract the first 500 components from a Principal Component Analysis (PCA) of the fingerprint data (for the pan-perturbation approach only), and (iii) train a regression algorithm using the data embeddings as input12. Under the perturbation-specific (PS) paradigm, we consider baseline K-nearest neighbors (PS-KNN), LASSO regressors (PS-LASSO), and LightGBM models (PS-LGBM), each trained on a single perturbation’s data. We also include three pan-perturbation (PP) methods: PP-LGBM and tDNN, with the latter further adapted into PP-MLP. For a fair comparison of the regression approaches, all methods are integrated with our MAE in place of their original autoencoders. To improve the prediction performance of all models (except for the baseline PS-KNN), we use ensembling by aggregating the predictions obtained with multiple regressors trained on different subsets of the data. In LEAP, we further ensemble multiple regressors trained on different data representations obtained with different initializations (Fig. 1). Gene expression representation algorithm We train representation learning models on RNAseq data to generate embeddings for the disease models (cell lines or PDX). These representation models are agnostic to the perturbation as they only depend on the molecular profiles of the disease models. The OmicsRPZ12 study demonstrated superior performance in downstream prediction tasks by employing a Masked Auto-Encoder (MAE). We use the same MAE with hyper-parameters recommended by the OmicsRPZ study for RNAseq representation. Briefly, we start from the method developed in VIME51 and we adapt it to (i) remove the mask-predict pretext-task, which didn’t improve the prediction performance of the downstream task but also significantly increased training time (results not shown), and (ii) optimize the mask allocation code. For each training batch, we randomly mask 30% of the input entries in the batch data matrix and use the VIME corruption method to permute these masked entries with values coming from other samples. We fix the representation dimensionality and several other parameters of the MAE based on the settings used in the OmicsRPZ paper. The representation dimension is fixed at 256, with 512 units in the first hidden layer, and training capped at 1000 epochs with early stopping (patience = 20, delta = 10**-5). Since we are in a transductive task setup, and to reduce computational burden, we do not train distinct MAE models on the different training sets used for each of the 5 tasks. Instead, we train the MAE models once using all available unlabelled samples with RNAseq measurements across the DepMap5, PharmacoDB30 and PDX Encyclopedia31 datasets, totalling 1920 cell lines and 191 PDX. The five MAE models trained on the same data but with different initializations are then used for all tasks (Fig. 1). In LEAP, we propose to use an ensemble of 5 MAE models obtained with different random seeds to leverage variability in this procedure and eventually increase robustness and prediction performance20. The change in random seed affects the selection of the early stopping holdout set, masking, and augmentation operations, and the initialization of the different layers. The 5 MAE models are trained on the same dataset including all unlabelled samples. Regression algorithms We consider seven modeling approaches. For a fair comparison across the prediction models approach, we use them jointly with our MAE and consider limited grids of up to 10 combinations of parameters (Supplementary Table 2). In methods that incorporate already an AE, we replace their AE with our MAE. The perturbation-specific K nearest neighbor52 (PS-KNN) constitutes a baseline in our study. In PS-KNN, the response to a given perturbation is predicted as the average response observed for the K nearest cell lines based on their gene expression profile which have available response to the perturbation of interest. We consider the K = 5 nearest neighbors. In PS-LASSO, we train one LASSO53 model for each perturbation. Each of the regressors in PS-LASSO takes molecular features (RNAseq) of the cell lines as input. The hyper-parameters of each of the regressors in PS-LASSO are tuned independently by maximizing prediction performance for the corresponding perturbation. We consider a small grid of 10 automatically defined values of the L1-regularization parameter using the sklearn library. Similarly, we train one LGBM for each perturbation in PS-LGBM. This LGBM has 400 estimators with a maximum depth of 10 and 31 leaves. We tune the learning rate and the proportion of features to keep in each subsample. We consider a limited grid of two values for the learning rate (0.005 and 0.01) and five values for the feature proportion (0.05, 0.1, 0.15, 0.2 or 0.25). We also train one MLP for each perturbation in PS-MLP. This MLP is trained with a surrogate loss for optimizing the Pearson’s correlation between predicted and observed perturbation responses. We use the Adam optimizer with a batch size of 64, a dropout rate of 0.2, a maximum of 200 epochs with early stopping and a learning rate scheduler. We tune (i) the initialisation of the learning rate using a grid of five values (0.0005, 0.001, 0.002, 0.005, 0.01), and (ii) the number of hidden layers, considering either one or two hidden layers of size 20. In PP-LGBM, we use a pan-perturbation regression model to predict the response to any perturbation in a given cell line based on both sample-level features (RNAseq) and perturbation-level features (fingerprints)10,12. As in the OmicsRPZ study12, we use an LGBM with 500 estimators, a maximum depth of 20 and 4,000 leaves, and we tune the learning rate and the L1-regularization parameter. We consider a limited grid of five values for the learning rate (0.01, 0.02, 0.03, 0.04 and 0.05) and two values for the regularization parameter (0 or 1). We also incorporate PP-tDNN18, a pan-perturbation deep neural network model designed to integrate gene expression and drug fingerprints through two subnetworks. Each subnetwork has three dense hidden layers, whose outputs are concatenated and passed through four additional hidden layers with consecutive halvings of nodes. Dropout layers follow all but the last hidden layer. Initial parameters and architecture are as defined in previous work18. We use a grid of five values for the dropout rate (0, 0.1, 0.25, 0.45, 0.7). Finally, we introduce PP-MLP, a version of tDNN tuned to enhance performance in our study. Unlike the original tDNN, PP-MLP employs tailored hyperparameter settings and additional regularization to improve predictive accuracy across perturbations. In particular, we use hidden layers of sizes (512, 256, 128, 64, 32, 16). As in PS-MLP, we use the Adam optimizer with a batch size of 64, a dropout rate of 0.2, a maximum of 200 epochs with early stopping and a learning rate scheduler. We further use batches, and tune the number of batches to be either 2048 or 8192. We also tune the initialisation of the learning rate considering the same grid of five values (0.0005, 0.001, 0.002, 0.005, 0.01) and use the same loss as in PS-MLP. Hyper-parameter tuning procedure. We use grid search to tune the hyper-parameters of all regression models, except the baseline PS-KNN. For each combination of hyper-parameters, we perform a 5-fold cross-validation within the training data, maximizing the per-perturbation Spearman’s correlation. For PS approaches, we tune the hyper-parameters of each perturbation-specific regression model separately. For PP approaches, we tune the hyper-parameters of the pan-perturbation regression model by maximizing the average per-perturbation Spearman’s correlation. In all cases, we use 5 folds made of non-overlapping sets of cell lines. Layered ensembling To improve prediction performances in LEAP, we use an ensemble model where the final prediction is defined as the average of the predictions obtained from multiple regressors. For this, we first use the 5 MAE models to generate 5 different representations of the training data. The regressors trained in each iteration of the cross-validation conducted on each of the 5 representations of the data are used for ensembling. A total of 25 regressors are ensembled, coming from the data embeddings of 5 MAE models and for each, 5 regressors trained in a 5-fold CV. Performance metrics Prediction performance is measured by comparing the true and predicted perturbation response in the test set of unscreened cell lines (challenge I), cell lines from an unscreened tissue (challenge II), or unscreened PDX (challenge III). As in previous work, we use the Spearman’s and Pearson’s correlations as performance metrics12. We report both (i) the average of per-perturbation correlations measured by comparing the true and predicted responses to each of the perturbations, and (ii) the overall correlation measured by comparing the true and predicted response for all sample-perturbation pairs in the test set. Although overall metrics are not appropriate in our setting as they primarily capture differences between the perturbations rather than differences between samples54, we report them in Supplementary Materials for comparison with previous studies. As small variations in ranking may not always be meaningful, we also report the per-perturbation Mean Squared Error (MSE) as a complementary metric. Biological interpretability Interpretability framework To generate biologically interpretable explanations for LEAP predictions, we transform gene expression profiles into pathway-level representations. Specifically, for each Reactome pathway, we perform principal component analysis (PCA) on the gene expression matrix restricted to that pathway’s genes, and use the first principal component (PC1) as a continuous pathway expression feature24. We then distill LEAP by fitting perturbation-specific LASSO models regressing predicted values from LEAP as a function of pathway-level features. To assess the robustness and consistency of pathway selection, we apply stability selection25,55,56. Briefly, LASSO regressors are fitted on 100 subsamples of the training data to compute selection proportion for each of the pathway features. Pathways that are consistently selected across subsamples are considered stable predictors. The LASSO regularisation parameter and threshold in selection proportion are calibrated by maximising the sharp score measuring model stability55. We report a normalised sharp score, where a value of 1 would indicate that the same features are selected in all regressors fitted on 100 different subsamples of the training data. Distillation validation Finally, to validate the explanatory power of the selected pathways, we train reduced LASSO models using only the most stable pathways as input. We compare their predictive performance to LEAP models trained for the same perturbation. High correlations between reduced and full model performance would indicate that selected pathways capture a substantial portion of the predictive signal driving LEAP’s predictions. Supplementary Information Below is the link to the electronic supplementary material. Supplementary Material 1 Supplementary Material 1