Assessing Performance of Orthology Detection Strategies Applied to Eukaryotic Genomes
Feng Chen, Aaron J. Mackey, Jeroen K. Vermunt et al. · 2007
Read Assessing Performance of Orthology Detection Strategies Applied to Eukaryotic Genomes by Feng Chen, Aaron J. Mackey, Jeroen K. Vermunt et al. (2007) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/plos-pone-0000383.txt.
Introduction
Feng Chen and colleagues compared computer methods for spotting orthologs, in a 2007 PLOS ONE paper. Orthologs are genes in different species that descend from one gene in a shared ancestor. Finding them correctly is a basic step in comparing genomes and tracing how genes evolved.
Several detection strategies exist, and they do not always agree. The team assessed how well the strategies perform when applied to eukaryotic genomes, the DNA of organisms with cell nuclei, from yeast to humans. Benchmarks like this help researchers choose tools they can trust, and they show where methods still mistake look-alike genes for true counterparts.
Catalog record
| CREATOR | Feng Chen, Aaron J. Mackey, Jeroen K. Vermunt et al. |
|---|---|
| YEAR | 2007 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 67 KB |
| LICENSE | Creative Commons Attribution License |
| RIGHTS | CC BY |
| SOURCE | PLOS ONE |
Source: https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0000383
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Assessing Performance of Orthology Detection Strategies Applied to Eukaryotic Genomes Authors: Feng Chen, Aaron J. Mackey, Jeroen K. Vermunt, David S. Roos Published: 2007, PLOS ONE DOI: 10.1371/journal.pone.0000383 ABSTRACT Orthology detection is critically important for accurate functional annotation, and has been widely used to facilitate studies on comparative and evolutionary genomics. Although various methods are now available, there has been no comprehensive analysis of performance, due to the lack of a genomic-scale ‘gold standard’ orthology dataset. Even in the absence of such datasets, the comparison of results from alternative methodologies contains useful information, as agreement enhances confidence and disagreement indicates possible errors. Latent Class Analysis (LCA) is a statistical technique that can exploit this information to reasonably infer sensitivities and specificities, and is applied here to evaluate the performance of various orthology detection methods on a eukaryotic dataset. Overall, we observe a trade-off between sensitivity and specificity in orthology detection, with BLAST-based methods characterized by high sensitivity, and tree-based methods by high specificity. Two algorithms exhibit the best overall balance, with both sensitivity and specificity>80%: INPARANOID identifies orthologs across two species while OrthoMCL clusters orthologs from multiple species. Among methods that permit clustering of ortholog groups spanning multiple genomes, the (automated) OrthoMCL algorithm exhibits better within-group consistency with respect to protein function and domain architecture than the (manually curated) KOG database, and the homolog clustering algorithm TribeMCL as well. By way of using LCA, we are also able to comprehensively assess similarities and statistical dependence between various strategies, and evaluate the effects of parameter settings on performance. In summary, we present a comprehensive evaluation of orthology detection on a divergent set of eukaryotic genomes, thus providing insights and guides for method selection, tuning and development for different applications. Many biological questions have been addressed by multiple tests yielding binary (yes/no) outcomes but no clear definition of truth, making LCA an attractive approach for computational biology. INTRODUCTION The rapid growth in the availability of genome sequence data, from an ever-increasing range of relatively obscure species, places a premium on the automated identification of orthologs to facilitate functional annotation, and studies on comparative and evolutionary genomics. Homologous proteins share a common ancestry, and may be characterized as either orthologs (which evolve by speciation only) or paralogs (which arise by gene duplication) [1] , [2] . Orthologs typically retain similar domain architecture and occupy the same functional niche following speciation, while (functionally redundant) paralogs are likely to diverge with new functions through point mutations and domain recombinations [3] , [4] . The concepts of orthology and paralogy are well-established in classical and molecular systematics [1] , and have been extended to describe more complicated situations associated with extensive gene duplications commonly observed in eukaryotic species [4] – [6] . In- and out-paralogs are analogous to the phylogenetic concepts in- and out-groups, denoting genes duplicated subsequent or prior to speciation, respectively. Recent duplications yield in-paralogs that may exhibit a many-to-one or many-to-many ortholog relationship with genes in the other species (termed co-orthologs). Several strategies have been employed to distinguish probable (co-)orthologs from paralogs, as summarized in Table 1 : phylogeny-based methods include RIO (Resampled Inference of Orthology) [7] and Orthostrapper/HOPS (Hierarchical grouping of Orthologous and Paralogous Sequences) [8] , [9] ; methods based on evolutionary distance metrics include RSD (Reciprocal Smallest Distance) [10] , [11] ; BLAST-based methods include Reciprocal Best Hit (RBH), COG (Cluster of Orthologous Groups) [12] – [15] /KOG (euKaryotic Orthologous Groups) [15] , and Inparanoid [5] , [16] . The problem of orthology detection is particularly acute for eukaryotic genomes, because of their large size, the difficulty of defining accurate gene models, the complexity of protein domain architectures, and rampant gene duplications [3] , [17] . To address these difficulties, we previously developed the OrthoMCL algorithm [18] , which improves on RBH by ( i ) recognizing co-ortholog relationships ( Figure 1 ), ( ii ) using a normalization step to correct for systematic biases when comparing specific pairs of genomes, and ( iii ) using a Markov graph clustering (MCL) algorithm [19] to define ortholog groups. OrthoMCL and Inparanoid exhibit similar performance when comparing two species, but the former is extensible to cluster orthologs across multiple species. Analysis of independently assigned EC number annotations suggests a high degree of reliability [18] , and orthology predictions for 55 genomes are available at OrthoMCL-DB [20] . Solid lines connecting A1 and B1 represent putative ortholog relationships identified by the ‘reciprocal best hit’ (RBH) rule. Dotted lines (e.g. those connecting A1 with A2 and A3, or B1 with B2) represent putative in-paralog relationships within each species, identified using the ‘reciprocal better hit’ rule. Putative co-ortholog relationships, indicated by dashed gray lines, connect in-paralogs across species boundaries (e.g. A3 and B2). Alternative orthology detection strategies (including phylogeny, distance or BLASTP-based), or homology detection methods. Parameters analyzed using the LCA benchmarking framework to assess their effect on orthology detection performance ( Figure 4 ). The fraction of positively predicted protein pairs (using default parameter settings) within the entire sampling dataset of 567,255 cross-species homologous protein pairs (defined by Pfam domains). The average fraction of positively predicted protein pairs (using default parameter settings) from 100 sampling replicates (of the average total of 1590.15 pairs). Despite the many ortholog identification methods now available, no comprehensive statistical comparison has yet been reported, in part because the lack of a genomic-scale error-free ‘gold standard’ dataset makes it difficult to analyze performance. Functional genomics data are often used as a surrogate for true orthology, both for ortholog assignment (i.e. functional orthologs) [21] and performance assessment [18] , and have been used to benchmark a small selection of orthology detection methods, and transfer of functional annotations [22] . Such data are likely to result in many errors, however, especially when applied across large evolutionary distances [4] . Even in the absence of a reliable gold-standard, the comparison of results from alternative methodologies contains useful information, as agreement enhances confidence (provided that the methods employed are independent), and disagreement indicates possible errors (either false positives or false negatives). Latent Class Analysis (LCA) is a statistical technique that can exploit this information, and has been widely applied to multivariate categorical data in research of medical diagnostics, marketing, sociology, etc [23] , [24] . For example, when no single, reliable diagnostic test is available for determining the status (latent class) of individuals with respect to a certain disease, LCA can be used to estimate the accuracy (sensitivity and specificity) of multiple diagnostics. We have applied LCA to the evaluation and optimization of a comprehensive set of orthology detection methods, providing a guide for selecting methods and appropriate parameters. This study also provides an analysis of similarities and statistical dependence between these methodologies. Two widely used ortholog grouping methods – the manually curated KOG database and the automated OrthoMCL algorithm – are further compared with respect to the consistency of clustering, protein function, and protein domain architecture. To illustrate the relationships between orthology and homology detection methods, some other methods BLASTP [25] , SBH (Single-way or One-way Best Hit) and TribeMCL [26] were also included in the analysis. TABLE: Various orthology/homology detection methods under study Methods Strategy a Apply to Proteins Grouping Capability Parameters