AMAS: a fast tool for alignment manipulation and computing of summary statistics.
Borowiec ML. · 2016
Read AMAS: a fast tool for alignment manipulation and computing of summary statistics. by Borowiec ML. (2016) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/europmc-pmc4734057.txt.
Introduction
Marek L. Borowiec wrote AMAS, a fast tool for handling sequence alignments, in PeerJ in 2016, publishing solo in the article's own voice. Evolutionary studies now juggle hundreds or thousands of gene alignments at once, and simple chores, converting formats, joining alignments, splitting them by gene, computing basic statistics, become the slow step. AMAS does all of these from the command line or inside Python programs, for both DNA and protein data.
Its statistics cover taxa counts, alignment length, missing data share, base composition, and the variable and informative site counts that phylogenetics runs on. In tests it concatenated large alignments faster than rival tools, using parallel processing and nothing beyond Python's standard library. The program is free under an open license. For researchers drowning in alignment files, AMAS turns hours of file handling into minutes, with summary numbers that help filter bad loci early.
Catalog record
| CREATOR | Borowiec ML. |
|---|---|
| YEAR | 2016 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 21 KB |
| LICENSE | CC BY 4.0 |
| RIGHTS | CC BY |
| SOURCE | Europe PMC |
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
AMAS: a fast tool for alignment manipulation and computing of summary statistics By Marek L. Borowiec, Jingyi Jessica Li Published in: PeerJ DOI: 10.7717/peerj.1660 ABSTRACT The amount of data used in phylogenetics has grown explosively in the recent years and many phylogenies are inferred with hundreds or even thousands of loci and many taxa. These modern phylogenomic studies often entail separate analyses of each of the loci in addition to multiple analyses of subsets of genes or concatenated sequences. Computationally efficient tools for handling and computing properties of thousands of single-locus or large concatenated alignments are needed. Here I present AMAS (Alignment Manipulation And Summary), a tool that can be used either as a stand-alone command-line utility or as a Python package. AMAS works on amino acid and nucleotide alignments and combines capabilities of sequence manipulation with a function that calculates basic statistics. The manipulation functions include conversions among popular formats, concatenation, extracting sites and splitting according to a pre-defined partitioning scheme, creation of replicate data sets, and removal of taxa. The statistics calculated include the number of taxa, alignment length, total count of matrix cells, overall number of undetermined characters, percent of missing data, AT and GC contents (for DNA alignments), count and proportion of variable sites, count and proportion of parsimony informative sites, and counts of all characters relevant for a nucleotide or amino acid alphabet. AMAS is particularly suitable for very large alignments with hundreds of taxa and thousands of loci. It is computationally efficient, utilizes parallel processing, and performs better at concatenation than other popular tools. AMAS is a Python 3 program that relies solely on Python’s core modules and needs no additional dependencies. AMAS source code and manual can be downloaded from http://github.com/marekborowiec/AMAS/ under GNU General Public License. INTRODUCTION The amount of data used in modern phylogenetics has increased dramatically since the advent of next-generation sequencing (McCormack et al., 2013). Data sets composed of hundreds or thousands of loci are becoming commonplace. Efficient concatenation of alignments is important in modern phylogenetics where multiple concatenation procedures are often carried out to explore how phylgoenetic signal is structured across the data. For example, two recent phylogenomic studies (Sharma et al., 2014; Borowiec et al., 2015) required concatenation and independent inference on dozens of data sets varying e.g., in rate of evolution or the amount of missing data. Alignment summary statistics are also needed for identification and filtering out “gappy” or fast-evolving data from downstream analyses such as the ones carried out in the studies cited above. Because the size of alignments used in phylogenetics is growing rapidly, there is a need for a fast and easy to use tool that can supplement existing phylogenomic pipelines. Available tools used for concatenation of alignments are either unable to correctly parse all alignment formats, are not flexible in the output format, or are slow to work on very large data sets (see Performance section below). Modern phylogenetic analysis often requires a form of bioinformatics pipeline where output of one procedure is being redirected as input for another tool. Although a number of freely available tools for manipulating alignments and computing their basic statistics exist, some of the most popular ones are based on graphical user interfaces (e.g., Mesquite: Maddison & Maddison, 2015) and not appropriate for command-line or scripted pipeline analyses. Examples of command-line tools with functionality partially overlapping with AMAS are FASconCAT-G Kück & Longo (2014) and Phyutility Smith & Dunn (2008). Phyutility and FASconCAT-G both allow for concatenation and the latter is additionally capable of computing various alignment statistics. All three programs have a broad range of functions and direct comparisons are difficult (see Table 1 for an overview of functions). More specifically, in addition to conversion, concatenation, and producing alignment summaries, FASconCAT-G allows the user to, among other functions, write MrBayes blocks in NEXUS files or create consensus sequences. Phyutility also allows for interactions with the NCBI databases and a number of manipulations on phylogenetic trees. Because of the many non-overlapping functions of these programs, the comparisons here focus on computing time required for concatenation. It should also be emphasized that the tools compared here should be viewed as complementary and AMAS is not intended to replace them. As demonstrated below, however, AMAS outperforms both FASconCAT-G and Phyutility at concatenation on a single core of a desktop computer. AMAS also supports parallel processing for even faster computation using multiple cores. It is easy to install and use, requires only a standard distribution of Python 3 or newer, and is provided with a detailed instructions manual. [Table: Overview of functions available in AMAS, FASconCAT-G, and Phyutility.] METHODS To assess and compare the performance of the concatenation fuctions of AMAS, FASconCAT-G, and Phyutility, I used four recently published phylogenomic data sets: filtered DNA alignments of 8,295 exons of 52 vertebrates and 3,679 UCE loci of 49 amniote taxa Jarvis et al. (2014) (available at gigadb.org/dataset/101041), DNA and amino acid alignments of 1,478 loci from 144 arthropod taxa from Misof et al. (2014) (http://datadryad.org/resource/doi:10.5061/dryad.3c0f1), and amino acid alignments of 5,214 exons from 19 hymenopteran taxa from Johnson et al. (2013) (datadryad.org/resource/doi:10.5061/dryad.jt440). FASconCAT-G v1.02 (Kück & Longo, 2014) was downloaded from https://www.zfmk.de/en/research/research-centres-and-groups/fasconcat-g and used according to the user manual provided along with the software download. Concatenation runs with FASconCAT-G were done in two modes: with the -i option that prevents the program from simultaneous calculation of alignment statistics for faster computing times, and with simultaneous writing of the statistics. The former is comparable with the AMAS concatenation option. Phyutility (Smith & Dunn, 2008) is available at https://code.google.com/p/phyutility/. It was used according to the manual. For a complete list of verbatim commands used for benchmarking, see Table S1. The performance was assessed on a desktop computer equipped with 12 Intel(R) Core(TM) i7-4930K CPUs at 3.40 GHz, 32 GB of DDR3 RAM at 1,600 Hz, a generic 4TB 7,200 rpm hard drive, and a Ubuntu 12.04LTS operating system with Python 3.4.3 installed. I used the Unix command time -p to evaluate execution times. Each command was run three times with minimal background system load and average time recorded. AMAS ALGORITHMS When used from the command line, AMAS parses input files along with associated metadata into taxon names and sequences stored as basic Python object types such as strings, lists, dictionaries, and tuples. Summary calculations or manipulations such as concatenation or taxon removal are then performed on those objects. Following any action requested by the user, AMAS composes strings in appropriate, requested format and writes output into files. A visual explanation of several AMAS functions is provided in Fig. 1. Within the AMAS source code, most of the functionality is wrapped into the custom MetaAlignment class that allows the user to interact with the data using the program as a Python module, in addition to the command line. Using the MetaAlignment methods the user can interact directly with basic alignment meta data and components such as file names, taxa, and constituent sequences using tools for correct parsing of input and output formatting. [Figure: AMAS functionality.(A) Concatenation of two FASTA files. (B) File format conversion from FASTA to PHYLIP. (C) Splitting of a concatenated alignment according to pre-defined partitions. (D) Removal of sequences by name.