Characterizing the replicability of cell types defined by single cell RNA-sequencing data using MetaNeighbor.

Crow M, Paul A, Ballouz S, Huang ZJ, Gillis J. · 2018

Read Characterizing the replicability of cell types defined by single cell RNA-sequencing data using MetaNeighbor. by Crow M, Paul A, Ballouz S, Huang ZJ, Gillis J. (2018) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/europmc-pmc5830442.txt.

Introduction

Crow and co-authors present MetaNeighbor, a test of whether cell types replicate. Single-cell RNA sequencing keeps naming new cell types. Lab quirks and loose naming make those claims shaky. A shared yardstick was missing.

MetaNeighbor scores how well a cluster in one dataset matches another. The team first checks neuron identity across eight datasets. Best practices for such checks emerge from that test. Applied to new interneuron subtypes, 24 of 45 replicate. Those sturdy types yield candidate marker genes. Large, varied gene sets prove best for matching. Readers get a route to trusting single-cell type claims.

Catalog record

CREATORCrow M, Paul A, Ballouz S, Huang ZJ, Gillis J.
YEAR2018
COLLECTIONPAPERS
FORMATTXT
FILE SIZE63 KB
LICENSECC BY 4.0
RIGHTSCC BY
SOURCEEurope PMC

Source: https://europepmc.org/articles/PMC5830442

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Characterizing the replicability of cell types defined by single cell RNA-sequencing data using MetaNeighbor By Megan Crow, Anirban Paul, Sara Ballouz, Z. Josh Huang, Jesse Gillis Published in: Nature Communications DOI: 10.1038/s41467-018-03282-0 ABSTRACT Single-cell RNA-sequencing (scRNA-seq) technology provides a new avenue to discover and characterize cell types; however, the experiment-specific technical biases and analytic variability inherent to current pipelines may undermine its replicability. Meta-analysis is further hampered by the use of ad hoc naming conventions. Here we demonstrate our replication framework, MetaNeighbor, that quantifies the degree to which cell types replicate across datasets, and enables rapid identification of clusters with high similarity. We first measure the replicability of neuronal identity, comparing results across eight technically and biologically diverse datasets to define best practices for more complex assessments. We then apply this to novel interneuron subtypes, finding that 24/45 subtypes have evidence of replication, which enables the identification of robust candidate marker genes. Across tasks we find that large sets of variably expressed genes can identify replicable cell types with high accuracy, suggesting a general route forward for large-scale evaluation of scRNA-seq data. INTRODUCTION Single-cell RNA-sequencing (scRNA-seq) has emerged as an important new technology enabling the dissection of heterogeneous biological systems into ever more refined cellular components. One popular application of the technology has been to try to define novel cell subtypes within a tissue or within an already refined cell class, as in the lung1, pancreas2–5, retina6,7, or others8–10. Because they aim to discover completely new cell subtypes, the majority of this work relies on unsupervised clustering, with most studies using customized pipelines with many unconstrained parameters, particularly in their inclusion criteria and statistical models7,8,11,12. While there has been steady refinement of these techniques as the field has come to appreciate the biases inherent to current scRNA-seq methods, including prominent batch effects13, expression drop-outs14,15, and the complexities of normalization-given differences in cell size or cell state16,17, the question remains: how well do novel transcriptomic cell subtypes replicate across studies? In order to answer this, we turned to the issue of cell diversity in the brain, a prime target of scRNA-seq as deriving a taxonomy of cell types has been a long-standing goal in neuroscience18. Already more than 50 single-cell RNA-seq experiments have been performed using mouse nervous tissue (e.g., ref. 19) and remarkable strides have been made to address fundamental questions about the diversity of cells in the nervous system, including efforts to describe the cellular composition of the cortex and hippocampus11,20, to exhaustively discover the subtypes of bipolar neurons in the retina6, and to characterize similarities between human and mouse midbrain development21. This wealth of data has inspired attempts to compare data6,12,20 and more generally there has been a growing interest in using batch correction and related approaches to fuse scRNA-seq data across replicate samples or across experiments6,22,23. Historically, data fusion has been a necessary step when individual experiments are underpowered or results do not replicate without correction24–26, although even sophisticated approaches to merge data come with their own perils27. The technical biases of scRNA-seq have motivated interest in correction as a seemingly necessary fix, yet evaluation of whether results replicate remains largely unexamined, and no systematic or formal method has been developed for accomplishing this task. To address this gap in the field, we propose a simple, supervised framework, MetaNeighbor (meta-analysis via neighbor voting), to assess how well cell-type-specific transcriptional profiles replicate across datasets. Our basic rationale is that if a cell type has a biological identity rooted in the transcriptome, then knowing its expression features in one dataset will allow us to find cells of the same type in another dataset. We make use of the cell-type labels supplied by data providers, and assess the correspondence of cell types across datasets by taking the following approach (see schematic, Fig. 1):We calculate correlations between all pairs of cells that we aim to compare across datasets based on the expression of a set of genes. This generates a network where each cell is a node and the edges are the strength of the correlations between them.Next, we do cross-dataset validation: we hide all cell-type labels (“identity”) for one dataset at a time. This dataset will be used as our test set. Cells from all other datasets remain labeled, and are used as the training set.Finally, we predict the cell-type labels of the test set: we use a neighbor-voting algorithm to predict the identity of the held-out cells based on their similarity to the training data.Fig. 1MetaNeighbor quantifies cell-type identity across experiments. a Schematic representation of gene set co-expression across individual cells. Cell types are indicated by their color. b Similarity between cells is measured by taking the correlation of gene set expression between individual cells. On the top left of the panel, gene set expression between two cells, A and B, is plotted. There is a weak correlation between these cells. On the bottom left of the panel we see the correlation between cells A and C, which are strongly correlated. By taking the correlations between all pairs of cells we can build a cell network (right), where every node is a cell and the edges represent how similar each cell is to each other cell. c The cell network that was generated in b can be extended to include data from multiple experiments (multiple datasets). The generation of this multi-dataset network is the first step of MetaNeighbor. d The cross-validation and scoring scheme of MetaNeighbor is demonstrated in this panel. To assess cell-type identity across experiments we use neighbor voting in cross-validation, systematically hiding the labels from one dataset at a time for testing. Cells within the test set are predicted as similar to the cell types from other training sets using a neighbor-voting formalism. Whether these scores prioritize cells as the correct type within the dataset determines the performance, expressed as the AUROC. In other words, comparative assessment of cells occurs only within a dataset, but is based only on training information from outside that dataset. This is then repeated for all gene sets of interest Conceptually, this resembles approaches for the validation of sample clustering28,29, which have primarily been applied to compare microarray results with respect to tumor subtyping30,31. Our method builds on these ideas, adapting and applying them for the first time to the question of cell identity in scRNA-seq, and specifically exploiting the patterns of co-expression believed to drive results32. Because our implementation is extremely fast, this approach readily permits carefully defined control experiments to investigate the data features that drive high performance, such as the dependence on expression variability, gene set size, rarity of cell types, or subtlety of transcriptional identity. We evaluate the replicability of cell-type transcriptional identity by taking sequential steps according to the basic taxonomy of brain cells: first classifying neurons vs. non-neuronal cells across eight scRNA-seq studies, then classifying cortical inhibitory neurons vs. excitatory neurons, and for our final step, we align interneuron subtypes across three studies. With detailed control experiments and empirical modeling, we validate the use of HVGs for cross-dataset cell identification, a common approach for feature selection within individual experiments4,33–35. Testing hundreds of gene sets, we find strong replication of neuronal identity when compared to non-neurons,

Read the complete text