iotbx.cif: a comprehensive CIF toolbox.
Gildea RJ, Bourhis LJ, Dolomanov OV, Grosse-Kunstleve RW, Puschmann H, Adams PD, Howard JA. · 2011
Read iotbx.cif: a comprehensive CIF toolbox. by Gildea RJ, Bourhis LJ, Dolomanov OV, Grosse-Kunstleve RW, Puschmann H, Adams PD, Howard JA. (2011) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/europmc-pmc3236671.txt.
Introduction
Richard Gildea and colleagues presented iotbx.cif, a software toolbox for working with crystallographic data files, in this 2011 paper in the Journal of Applied Crystallography. The CIF format is the standard way to deposit and archive small-molecule crystal structures, and it also underpins the chemical dictionaries that macromolecular crystallography relies on. Programs that read and write the format correctly need solid foundations.
The toolbox gives developers a comprehensive set of tools for building applications that use CIF, handling its syntax and its official dictionaries. The authors place it in context: the format grew from small-molecule work into a backbone for resources such as the Protein Data Bank's chemical component dictionary and standard monomer libraries. Their aim is to make reliable CIF support easy to add to crystallographic software.
Catalog record
| CREATOR | Gildea RJ, Bourhis LJ, Dolomanov OV, Grosse-Kunstleve RW, Puschmann H, Adams PD, Howard JA. |
|---|---|
| YEAR | 2011 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 24 KB |
| LICENSE | CC BY 4.0 |
| RIGHTS | CC BY |
| SOURCE | Europe PMC |
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
iotbx.cif: a comprehensive CIF toolbox By Richard J. Gildea, Luc J. Bourhis, Oleg V. Dolomanov, Ralf W. Grosse-Kunstleve, Horst Puschmann, Paul D. Adams, Judith A. K. Howard Published in: Journal of Applied Crystallography DOI: 10.1107/S0021889811041161 ABSTRACT iotbx.cif is a comprehensive toolbox for the development of applications that make use of the CIF format. INTRODUCTION The CIF (Crystallographic Information File) syntax (Hall et al., 1991 ▶) has become firmly established (Brown & McMahon, 2002 ▶; Spek, 2009 ▶) as the file format for deposition and archiving of small-molecule crystal structures and increasingly their structure factors. Whilst the Protein Data Bank (PDB) format (http://www.wwpdb.org) is still the prevailing file format for deposition of macromolecular crystal structures, the CIF format is nonetheless important to macromolecular software through their extensive use of the PDB chemical components (http://www.wwpdb.org/ccd.html) and CCP4 monomer libraries (Vagin et al., 2004 ▶). In addition to the core CIF dictionary (Hall, Allen & Brown, 2005 ▶), the International Union of Crystallography (IUCr) maintains CIF dictionaries for describing the results of macromolecular (Fitzgerald et al., 2005 ▶), powder diffraction (Toby, 2005 ▶) and electron density studies (Mallinson & Brown, 2005 ▶), and for describing incommensurately modulated crystal structures (Madariaga, 2005 ▶). The Crystallography Binary Format (CBF) and image-supporting Crystallographic Information File (imgCIF) (Bernstein & Hammersley, 2005 ▶) are extensions to the CIF format to support inclusion of binary data in the CIF, in particular raw experimental data from area detectors. Furthermore, CIF is probably one of the most well known file formats within the field of chemistry, since it is predominantly the form in which chemists receive the results of a crystal structure analysis carried out on their behalf. The CIF format is intrinsically involved in a wide variety of crystallographic applications from data collection to publication and archiving of the outcomes of crystallographic studies. In addition there is a wealth of crystal structure coordinates and reflection data freely available in CIF format through the Crystallographic Open Database (COD; Gražulis et al., 2009 ▶) and the large quantity of data available as supplementary material for papers published in IUCr journals, for which many possible uses can be imagined. As such it is vital for a crystallographic library such as the cctbx (Grosse-Kunstleve et al., 2002 ▶) to provide high-quality tools for reading, creation and manipulation of CIFs, and extraction of crystallographic information from them. Several CIF programming libraries have been developed for various languages and environments, including Fortran (Hall & Bernstein, 1996 ▶; Rodriguez-Carvajal & González-Platas, 2003 ▶), C (Ellis & Bernstein, 2001 ▶; Westbrook et al., 1997 ▶), Objective C (Chang & Bourne, 1998 ▶), .NET (Lin, 2010 ▶), Java (Day et al., 2011 ▶), Perl (Bluhm, 2000 ▶) and Python (Hester, 2006 ▶). Whilst there existed several partial CIF parsers within the cctbx, each hand-crafted to suit a specific task [separate tools for reading the PDB chemical components and CCP4 monomer libraries (Painter & Merritt, 2004 ▶); as part of the phenix.cif_as_mtz tool; for reading fcf reflection files as output by SHELXL (Sheldrick, 2008 ▶)], a comprehensive CIF parser that was tightly integrated with the rest of the library was conspicuously absent. During the development of the smtbx (small-molecule toolbox) and OLEX2 (Dolomanov et al., 2009 ▶), it became apparent that the CIF format would play a central part in presenting the results of the procedures developed. In addition, there was a need to provide an interface for managing the contents of the CIF within OLEX2. Therefore it was decided to implement a new CIF framework within the iotbx (input/output toolbox) module of the cctbx. Given the availability of a clearly defined formal grammar for the CIF syntax (Hall, Spadaccini et al., 2005 ▶), it was decided to use the ANTLR parser generator (Parr, 2007 ▶) for the automatic generation of a lexer and parser. ANTLR was chosen because of its support for multiple programming languages, in particular its support for Python and C/C++. In addition, the associated ANTLRWorks (http://www.antlr.org/works/index.html) GUI development environment features a number of tools that aid the development of grammars, including visualization of syntax diagrams and rule dependency graphs. This enabled the majority of the development to be focused on the design of the internal representation of the CIF model, whilst ensuring that the resulting parser closely follows the formal CIF grammar. The code is structured in such a way that the parser is quite distinct from the model, meaning that an alternative representation of the model could be used with the same parser, and conversely a different parser could be used to populate the existing iotbx.cif model. As is standard practice in the cctbx, all aspects of the code are rigorously tested by unit tests, guarding against future regression of the code base. In addition the unit tests provide usage examples for the programmer interface. VALIDATION OF CIFS AGAINST DATA DICTIONARIES Successful parsing without errors of a given CIF indicates only that it is syntactically correct. CIF dictionaries allow for a machine-readable formal description of allowed data items and for possible restrictions on the attributes of their associated values. A collection of application-specific dictionaries are maintained by the Committee for the Maintenance of the CIF Standard (COMCIFS), and can be used to validate the contents of a given CIF. The CIF data dictionaries abide by the CIF syntax, with two distinct dictionary definition languages [DDL1 (Hall & Cook, 2005 ▶) and DDL2 (Westbrook et al., 2005 ▶)] currently in use. In the context of iotbx.cif, a CIF can be validated as follows: The smart_load_dictionary function allows for a dictionary to be loaded from a variety of sources, including from a locally stored version, by downloading from an arbitrary URL or via lookup in a CIF dictionary register (e.g. ftp://ftp.iucr.org/pub/cifdics/cifdic.register), allowing use of the most up-to-date version of the dictionary. A list of potential errors and warnings found during the validation is output by the procedure. The error handling is designed such that it is possible for an application making use of iotbx.cif to override the default error handler with one specific to the needs of the application. COMMON CIF SYNTAX ERRORS AND ERROR RECOVERY As a result of comprehensive testing of the iotbx.cif parser a number of commonly encountered syntax errors were identified. Among the sources of CIFs used are the COD, a selection of CIFs obtained from the IUCr journals web site, the PDB chemical components library, the CCP4 monomer library and the Durham in-house database of crystal structures. Among the commonly found errors are the following: (1) Missing starting and closing quotes. (2) Missing starting and closing quotes for a string containing whitespace. (3) Some text prepended to the CIF but not using CIF comment format. (4) Mismatching semicolon delimiters for a multi-line text field. (5) More than one data value per tag. (6) Missing data value for a tag. (7) Incomplete CIF – e.g. missing data block heading. (8) Intended data block heading containing whitespace or illegal character(s). This can happen if a program uses the file name as the data block heading when creating a CIF but does not remove/replace whitespace or illegal characters. (9) Non-ASCII characters – data values have been copied from other sources; for example, this could be