THE ARCHIVING AND DISSEMINATION OF BIOLOGICAL STRUCTURE DATA

Abstract

The global Protein Data Bank (PDB) was the first open-access digital archive in biology. The history and evolution of the PDB are described, together with the ways in which molecular structural biology data and information are collected, curated, validated, archived, and disseminated by the members of the Worldwide Protein Data Bank organization (wwPDB; http://wwpdb.org). Particular emphasis is placed on the role of community in establishing the standards and policies by which the PDB archive is managed day-to-day.

Historical Background

Structural biology is a relatively young science that can trace its roots to the first X-ray diffraction studies of pepsin in 1935 by Dorothy Crowfoot (Hodgkin), who at the time was a student of J.D. Bernal [1]. Twenty years later, Kendrew determined the structure of myoglobin [2,3]; shortly thereafter, Perutz determined the structure of hemoglobin [4,5]. Both won Nobel prizes for their achievements. Not long after these structures were published, the crystallographic community began discussions as to how to best archive these data and make them available. During this period, there were numerous grassroots meetings, one of which resulted in a petition, and many exchanges of handwritten documents. In 1971, the Cold Spring Harbor Laboratory hosted a symposium on protein crystallography, during which leaders in the field presented their seminal work [6]. Walter Hamilton, an attendee, offered to provide the first home for what is now known as the Protein Data Bank (PDB) [7]. The PDB was launched at Brookhaven National Laboratory, based on the Protein Structure Library created by Edgar Meyer [8]. The initial PDB archive contained fewer than ten structures, all of which were determined by X-ray crystallography. In the 1980s, structures determined using NMR methods began to be deposited, and in 1990 the first structure determined by electron microscopy was deposited. In 1982 the PDB reached 100 entries, in 1993 1,000 entries, in 1999 10,000, and in 2014 100,000 entries. At the time of writing, the PDB archive contains over 117,000 structures of proteins, nucleic acids, and their complexes with one another and with small molecule ligands.

The PDB as a Community Data Resource

From its inception, the PDB has been a community effort that has evolved with changes in scientific culture. For example, when the PDB was first created, data submission was voluntary. However, in the 1980s, members of the community became outspoken about the need to enforce mandatory data deposition. Various committees were set up to define what data should be required and when to disseminate the data. These guidelines were published in 1989, and over time, adopted by virtually all of the scientific journals that now require PDB deposition(s) as a prerequisite for publication of structural studies [9]. In 2008, further shifts in community sentiment led to mandatory deposition of experimental data together with atomic coordinates. In the current decade, the importance of reproducibility has been highlighted. The PDB convened method-specific Validation Task Forces and Workshops [10–13] to define what data should be collected and how best to validate the structural models, the experimental data, and the fit of the models to the data. Now every structure in the PDB comes with a publicly available validation report, and authors are strongly encouraged to include these reports with their manuscript submissions to journals.

The importance of global participation in data archiving was understood early in the creation of the PDB. Indeed, the announcement of the PDB in 1971 described the collaboration with the Cambridge Crystallographic Database Centre [7]. In 2003, a Memorandum of Understanding (MOU) among partners in the US (RCSB Protein Data Bank; http://www.rcsb.org), Japan (Protein Data Bank Japan or PDBj; http://www.pdbj.org), and Europe (Protein Data Bank in Europe or PDBe; http://pdbe.org) established the Worldwide Protein Data Bank (wwPDB) partnership, which is responsible for formalizing the procedures involved in collecting, standardizing, annotating and disseminating the data [14]. Subsequently, a global NMR specialist data repository BioMagResBank, composed of deposition sites in the US (BMRB; http://www.bmrb.wisc.edu) and Japan (PDBj-BMRB; http://bmrbdep.pdbj.org), joined the wwPDB.

The X-ray crystallography community has led the biological sciences in the area of data sharing. While the sociological/anthropological underpinnings of this leadership role have not been fully explored, much of what has transpired in the creation and evolution of the PDB can be traced to J.D. Bernal, who, in addition to being a brilliant scientific innovator, was a prominent social activist, whose beliefs were consistent with the conduct of the PDB [15].

Content of the PDB Archive

The PDB archive contains information about structural models that have been derived from experimental methods, including X-ray/neutron/electron crystallography, NMR spectroscopy, and 3D electron microscopy (3DEM). In addition to the 3D coordinates, the details of the chemistry of the polymers and small molecules are archived, as are metadata describing the experimental conditions, data-processing statistics and structural features such as the secondary and quaternary structure. The structure-factor amplitudes (or intensities) used to determine X-ray structures, and chemical shifts and restraints used in determining NMR structures are also archived. The electron density maps used to derive 3DEM models are archived in EMDB [16], and the experimental data underpinning them can be archived in EMPIAR [17]. In collaboration with community experts, pertinent data items are defined for each experimental field, with requirements evolving over time. The PDB data dictionary, originally developed to describe macromolecular crystallography, contains more than 4,400 data items. The dictionary combines data items common to all methods as well as those that are method specific. For example, the current dictionary contains 250 NMR- and 1200 3DEM-specific data definitions.

Over time, the holdings of the PDB have increased dramatically as has the complexity of the structures being archived (Figure 1).

Figure 1.

Growth of the PDB archive. A) Number of entries deposited annually (dark grey) and available at the end of each year (light grey); B) number of X-ray crystal structures, C) NMR structures, and D) 3DEM structures available each year.

A workshop held in 2005 led to the policy that purely in silico models should not be part of the PDB [18], and, instead, a modeling portal should be created for these models. The Protein Modeling Portal was established in 2007 [19].

Representation of PDB Data

The first data format used by the PDB was established in the early 1970s and was based on the 80-column Hollerith format used for punched cards. The atom records included atom name, residue name and sequence number. A “header record” contained some metadata. This format was readily accepted because it was simple and both human- and machine-readable. However, it had many serious drawbacks in that the size of the structural models was limited to 99,999 atoms and that relationships among the data items were implicit. These inherent weaknesses meant that significant domain knowledge was necessary in order to write software using this format.

In the 1990s, the IUCr chartered a committee to create a more formal data model. This committee proposed the Macromolecular Crystallographic Information File (mmCIF) [20]. mmCIF is a self-defining format in which every data item has attributes describing its features including relationships to other data items. Most importantly, mmCIF has no limitations with respect to the size of the archived structural model. The dictionary and the data files are completely machine-readable, and no domain knowledge is required to read the files. The first dictionary contained over 3,000 data items relevant to X-ray crystallography. Over time, terms specific to NMR and 3DEM were added, and the dictionary was renamed PDBx/mmCIF. In 2007, it was decided that PDBx would be the Master Format for data collected by the PDB. In 2011, major X-ray structure determination software developers agreed to adopt this data model so that all output from their programs would be in PDBx. In 2015, large structures archived in the PDB that had formerly been split into multiple entries were combined into single entries and mmCIF formatted files. Other structural biology communities are in the process of building on the PDBx/mmCIF framework to establish their own controlled vocabulary and specialist data items [19,21].

PDBML, an XML format based on PDBx/mmCIF [22], and its RDF (Resource Description Framework) conversion were developed to facilitate the integration of structure data with other life sciences data resources could be facilitated [23].

The Data Pipeline

Every data resource has a set of procedures for deposition, curation, validation, archiving and dissemination of data. The pipeline currently used by the wwPDB to populate the PDB archive is illustrated schematically in Figure 2.

Figure 2.

wwPDB Deposition, Annotation, and Validation pipeline. Each box represents a modular component of the data processing workflow.

In the very early days of the PDB, structures were deposited to BNL on magnetic tapes containing atomic coordinates with paper forms listing other data items, all sent first by mail and then via e-mail. A web-based system, called AutoDep, was created in the 1990s [24]. This system was later modified and used by PDBe [25] until very recently. The RCSB PDB and PDBj collected data using a system based on mmCIF called ADIT [26], and the BMRB in the US and its affiliate in Japan adopted a similar system called ADIT-NMR [27]. Although these systems were distinct, since 2003, the wwPDB partners have determined jointly what data should be collected and which procedures and algorithms should be used for data processing. In 2007, it was agreed within the wwPDB to create a single deposition,

Structures are made available to the public either immediately after they have been fully curated or -in most cases- when they are published in a journal. Usually, either the author or the journal informs wwPDB that the paper describing the structure is about to be published. PDB data are released in a two-stage process. Every Saturday at 03:00 UTC the polymer sequences, ligand SMILES strings, and crystallization pH for new structures designated for release are made public (http://wwpdb.org/download/downloads) as a courtesy to the protein structure modeling and computational chemistry communities to enable weekly blinded prediction challenge efforts (e.g., CAMEO [19], D3R CELPP [28]). Every Wednesday at 00:00 UTC, all new structures designated for release are made publicly available through the wwPDB FTP sites. On average about 200 structures are released every week. As evidence for the importance of this archive, in 2015, more than 500 million sets of atomic coordinates were downloaded from the wwPDB FTP sites.

Value-Added Resources

The wwPDB FTP sites provide the core data for many databases, services, and websites, including those run by the individual wwPDB partners. In the original wwPDB MOU, it was agreed that to best serve science, wwPDB partner websites would compete with one another and would offer many different kinds of services and features. The RCSB PDB has extensive search and reporting capabilities as well as an education portal called PDB-101 [26,29]. PDBe has multiple search and browse facilities as well as analysis and bioinformatics tools [30,31]. PDBj provides a variety of services and viewers and supports browsing in multiple Asian languages [23,32]. BMRB has many capabilities designed to serve the NMR community [33].

CATH [34] and SCOP [35,36] use the data in the PDB to classify the structural domains of proteins with an attempt to relate them to function. More recently, these two databases have agreed to work together and with other resources in the UK to provide predicted structural features under a unified system called Genome3D [37].

Additional specialty databases provide information on particular classes of macromolecules such as nucleic acids [38].

The Protein Structure Initiative (PSI) Structural Biology Knowledgebase (SBKB)[39] was an ambitious effort to unify information about protein sequence, structure and function. Unfortunately, the decision to discontinue funding the PSI means that this resource will cease to exist.

Challenges Going Forward

A review of the holdings of the PDB shows a steady growth (~10,000 new structures annually). More significantly, the complexity of the structural models continues to increase with more and more large heterogeneous assemblies entering the archive. Fortunately, there are no longer technical restrictions to receiving, annotating, validating, and disseminating these very large structures.

Historically, most structures were determined exclusively with the aid of a single experimental method: X-ray crystallography, NMR or 3DEM. In recent years, these traditional techniques are being combined with other methods to yield improved models. For example, it is now common practice to add data from small-angle scattering measurements to NMR-derived restraints to determine solution structures [40,41]. Similarly, NMR or X-ray data can be combined with cryoEM data in integrative modeling approaches [42]. Such integrative methods make it possible to combine data from different biophysical techniques with computational methods to create models of very large macromolecular machines [43]. However, hybrid approaches also present a variety of challenges including how to validate these structures and then how to archive them. As in the past, with the help and advice of an expert Task Force [44], this integrative challenge will be met by the wwPDB partners.

Highlights.

• The PDB is the first, community-driven, open-access digital archive in biology.

• The data in the PDB are highly curated.

• The wwPDB is an international partnership that manages the archive.

• The wwPDB agrees on all standardization, processing, and dissemination policies.

• Storing data and methods in a standardized way supports data reproducibility.