QA-RecombineIt: a server for quality assessment and recombination of protein models

Abstract

QA-RecombineIt provides a web interface to assess the quality of protein 3D structure models and to improve the accuracy of models by merging fragments of multiple input models. QA-RecombineIt has been developed for protein modelers who are working on difficult problems, have a set of different homology models and/or de novo models (from methods such as I-TASSER or ROSETTA) and would like to obtain one consensus model that incorporates the best parts into one structure that is internally coherent. An advanced mode is also available, in which one can modify the operation of the fragment recombination algorithm by manually identifying individual fragments or entire models to recombine. Our method produces up to 100 models that are expected to be on the average more accurate than the starting models. Therefore, our server may be useful for crystallographic protein structure determination, where protein models are used for Molecular Replacement to solve the phase problem. To address the latter possibility, a special feature was added to the QA-RecombineIt server. The QA-RecombineIt server can be freely accessed at http://iimcb.genesilico.pl/qarecombineit/.

INTRODUCTION

The availability of experimentally determined high resolution protein structures has significantly improved the understanding of biological mechanisms and has greatly facilitated rational drug design (1–4). Although the rate at which protein structures are being determined by experimental methods (mainly X-ray crystallography and nuclear magnetic resonance spectroscopy) has increased dramatically during the past years, this progress has not been efficient enough to match the amount of data on predicted protein sequences generated by genome sequencing efforts. At the time of writing, there are ∼83 000 protein structures in the PDB (5,6), but ∼23 million protein sequences in the non-redundant databases (non-redundant GenBank CDS translations, PDB, SwissProt, PIR and PRF) (7). As a complement to experimental methods, many computational methods have been developed to address the sequence-structure gap. However, to generate a 3D structural model of the target protein, typical users of structure modeling methods (e.g. biologists willing to predict a structure of their favorite protein) use only a few fully automated servers, usually the ones that have performed well in the most recent Critical Assessment of Techniques for Protein Structure Prediction (CASP) experiment (8). The recommended procedure is then to choose the model ranked the highest according to a model quality assessment program (MQAP), a method that predicts the accuracy of the model with respect to the true structure (without knowing the true structure) (9). Even though the model with the best overall score is selected, it does not mean that its local conformation over the entire length of the target sequence is always the closest to the native structure when compared with the conformations of the remaining models. Thus, even if more accurately modeled regions exist in the remaining models, they cannot be easily used to improve the main model.

To address this challenging problem, we developed the QA-RecombineIt web server. Our method operates in two stages (Figure 1). In the first stage (QA-mode), our server predicts the global quality of input models and provides estimates of local quality as the deviation between C-α atoms in the models and corresponding atoms in the unknown native structure. Together with the input models, these predictions subsequently become the input for the second stage (RecombineIt-mode), in which fragments predicted to be better than others are judiciously combined to generate hybrid (consensus) models. Finally, hybrid models are scored by the MQAPs implemented in the QA-mode and then presented to the user.

Figure 1.

Flowchart describing the main functionalities of the QA-RecombineIt server. Protein sequence in FASTA format and 3D structure models of the target protein must be provided to execute the server. QA-RecombineIt implements two modules, including QA-mode for assessment of protein models and RecombineIt-mode for merging the best quality fragments derived from the input models. By default, these modules operate in a fully automatic way. However, more advanced users can modify the operation of the fragment recombination algorithm (RecombineIt-mode) by selecting the method according to which best fragments and/or models will be picked (box A), and/or manually identifying the models (box B) and/or fragments(s) (box C) on the base of which the hybrid model(s) will be created.

Although other freely available servers exist that either predict protein structure from the amino acids sequence (10–14) or assess the quality of protein models (15–19), QA-RecombineIt is unique in providing an integrated underlying methodology for both scoring and improving models. QA-RecombineIt can be run for a set of models generated by a variety of protein structure methods, including fold recognition/de novo modeling, template-based modeling and loop refinement, among others. Essentially, our server can be used at the last step of any protein structure prediction procedure.

MATERIALS AND METHODS

Model quality assessment

QA-RecombineIt relies on five different MQAP procedures. The MQAPs can be divided into two categories: (i) single-model MQAPs, i.e. methods that assess the quality of a single model and (ii) clustering MQAPs, i.e. methods that operate by structural comparisons between a number of alternative models generated for the target sequence. It was shown that in cases where many models based on varied prediction methods are available, clustering approaches significantly outperform single-model MQAPs (18,20–22). However, in cases where only one or few alternative models are available, the gap between the clustering MQAPs and single-model MQAPs is marginal, with single-model MQAPs outperforming clustering MQAPs in some cases (18). Table 1 gives a brief overview of the MQAPs integrated by the QA-RecombineIt server. In the following sections, these methods are presented in detail.

Table 1.

MQAPs implemented in QA-RecombineIt

Name	MetaMQAP	ProQ2	DFIRE	GOAP	MQAPmulti
Type	LG/S	LG/S	G/S	G/S	LG/C

S, single-model MQAP; C, clustering MQAP; L, MQAP that predicts local accuracy of a model; G, MQAP that predicts the global accuracy of a model.

MetaMQAP

MetaMQAP is a meta-predictor that combines output from a number of original methods used for predicting global and/or local accuracy of protein structural models, including VERIFY3D (23), PROSA (24), BALA-SNAPP (25), ANOLEA (26), PROVE (27), TUNE (28), REFINER (29) and PROQRES (30). In addition, MetaMQAP analyzes the following features of individual residues: secondary structure, solvent accessibility and depth within the structure. Together with linear regression, these parameters are used to assess the deviations of C-α atoms for a given model. Then, based on these predictions, the global accuracy of the model is calculated and expressed according to the Global Distance Test (GDT_TS) (31). This indicator corresponds to the average value of fractions of C-α atoms in the model that are placed within the distances of 1, 2, 4 or 8 Å from corresponding C-α atoms in the experimentally determined structure.

ProQ2

ProQ2 evaluates the following features (16): atom–atom and residue–residue contacts, surface accessibility, predicted secondary structure, predicted surface exposure and evolutionary information calculated over a sequence window. Based on these features, a support vector machine algorithm predicts the value of S-score (32) as the measure of accuracy for each residue in a protein model. This score is defined as

where d_i is the distance for residue i between the experimentally determined structure and the model, and d₀ is a distance threshold. As the threshold is set to 3 Å, the S-score is sensitive to errors ranging from 0 to 3 Å. Once the local quality prediction is performed, the global quality prediction for a model is achieved by summing the local predictions and then dividing the sum by the length of the target sequence.

Statistical potentials: DFIRE and GOAP

In general, statistical or knowledge-based potential is a pseudo-energy function derived from known protein structures. The majority of statistical potentials use the inverse Boltzmann law to convert the ratio of observed and expected frequencies of interactions into a scoring function that assesses the likelihood that a given interaction is ‘good’ or ‘bad’ (33,34).

DFIRE is a distance-dependent, structure-derived, all-atom statistical potential based on a distance-scaled, finite, ideal-gas reference state of uniformly distributed non-interacting ideal gas points. The DFIRE potential was found to be efficient in the refinement of protein structures, in particular segments that contain secondary structure elements or loops (35,36). A detailed description of the potential was presented by Zhou and Zhou. (37).

GOAP was developed for atomic-resolution modeling and refinement. Zhou and Skolnick improved the description of pairwise atomic interactions, similar to that of DFIRE, by introducing the orientation-dependence of all individual heavy atoms. To achieve this, a plane-like object is introduced for each atom by using two of its bonded neighboring atoms and itself. Then, a potential value is calculated for each pair of interacting atoms. To do so, the method analyzes the distance between a given pair of atoms and the mutual orientation of the planes assigned to the atoms. The GOAP potential has been described in detail by Zhou and Skolnick (38).

MQAPmulti

MQAPmulti is the only method implemented in QA-RecombineIt that clusters the protein models being scored. This method takes both protein models and the amino acids sequence of the protein under analysis as input. MQAPmulti executes three separate modules to calculate intermediate scores and then combines them into one final score. First, MQAPmulti predicts protein structure features from the target sequence using third-party methods; these features include secondary sequence (39–41), solvent accessibility (42,43) and contact maps (44). The predictions obtained are compared with values of the corresponding features calculated directly from the 3D structural models under evaluation. These (dis)agreement terms, together with in-house implementation of the DFIRE statistical potential and the number of unsatisfied hydrogen bond donors/acceptors, are used to estimate a global accuracy of each of the input models. To do so, a linear regression is applied. It should be emphasized that at this stage, every model is scored independently of the others; thus, the predicted global quality is single-model based. In the remaining part of this manuscript, we will call this score ‘True-MQAPmulti score’. Second, models are clustered according to their structural similarity. Two measures of similarity between two models are applied: GDT (31) (see earlier in the text) and Q-score (45,46), which measures the structural similarity between two models by comparing their internal residue distances. Third, MQAPmulti combines the True-MQAPmulti scores of all models with pairwise similarity of these models to enhance the model quality assessment when only a few models are considered. It is based on the assumption that values of single-model scoring function, on average, decrease as models become more similar to the native structure. It was postulated and then proven (47) that the model closest to the native structure should provide the highest correlation coefficient of a score (provided by such a single-model MQAP) versus distance, when used as the reference in pairwise comparisons with the remaining models. For a set of models, MQAPmulti calculates the correlation coefficient for each model and uses the obtained correlation coefficient as a score of model correctness. MQAPmulti uses two sets of such Pearson’s correlation coefficients: between the score provided by the True-MQAPmulti score and either GDT_TS or Q-score. Finally, the predictions provided by the aforementioned three modules (eight different scores in total) are used, together with a support vector machine algorithm (48,49), to predict the GDT_TS score of the models concerned. The method was trained on CASP7 and CASP8 single-domain protein targets.

Tertiary structure prediction using fragment recombination

RecombineIt is an automatic fragment recombination algorithm that was inspired by both the Frankenstein’s monster approach to comparative modeling—a manual methodology developed previously in our laboratory (50,51)—and many years of experience of our laboratory members in protein structure prediction, in particular in building models for challenging cases that include combination of template-based and template-free modeling [e.g. (52)]. The RecombineIt algorithm comprises four steps. First (stage I), five models are picked according to the highest global score provided by MQAPmulti (or MetaMQAP score, if <50 models are submitted by a user); these models will be used later on as the ‘leading’ models. Simultaneously (stage II), based on the PSIPRED (39) secondary structure prediction, the target sequence is divided into partially overlapping blocks (10 overlapping residues), initially containing one secondary structure element and half of each loop region connected to this element. For long sequences, if needed, these initial blocks are merged (with neighboring blocks in the sequence) to reduce the total number of blocks to ≤15. Then, based on these block boundaries, the input models are cut into fragments. Next (stage III), all possible combinations of the fragments are ranked (without explicitly generating 3D models for each combination). To rank a given combination of fragments, the sum of local MQAPmulti scores (or MetaMQAP scores if <50 models are submitted) is calculated for all residues. In addition, the combination is penalized if its fragments are as follows: (i) derived from models that have different folds from each other (i.e. the TM-score between a pair of models is below 0.3); (ii) derived from models with folds that differ from the folds of the ‘leading’ models (TM-score < 0.3); and (iii) if the similarity of overlapping regions of two merged together fragments is measured to have TM-score smaller than 0.9 (53). If the number of possible combinations of fragments is higher than 10⁸, an in-house developed genetic algorithm is used to find the best combinations of the fragments. Finally (stage IV), 3D models are built for each of the 100 top-scored combinations of fragments by using MODELLER (54) (version 9v3) in a multi-template mode. In this last step, each fragment is considered to be a single template. As the fragments of a given combination only partially overlap in their sequences, it would be impossible for the MODELLER program to model the 3D structure without additional distance restraints between the fragments. Thus, for each fragment, a set of distances between its C-α atoms and the remaining C-α atoms from the model, from which this fragment was derived, is measured. Then, the data are provided for the MODELLER program. Notably, for a given combination of fragments that are derived from many models, there are always two variants of a pair for given residues X and Y. One variant corresponds to the situation in which X is contained in a fragment derived from model Z′, and Y is contained in the rest of the same model. However, for a different model (Z′′), the situation is reversed: Y is located in a fragment, whereas X is found within the rest of the model Z′′. Thus, for cases where the conformations of models Z′ and Z′′ are not identical to one another, the distances between C-α atoms of residues X and Y in model Z′ may not be equal to those observed in model Z′′. To avoid overwriting such conflicting distance constraints assigned to the same pair of residues, only the distances between C-α atoms of odd-numbered residues within a fragment and C-α atoms of even-numbered residues within the remaining part of the model are taken into account.

By default, the QA-RecombineIt server uses MQAPmulti in stages I and III (or MetaMQAP if <50 models are submitted), and MetaMQAP is used by default at stage IV. However, a user can select another set of MQAPs (those described in the previous subsection) to be executed and/or used by RecombineIt.

DESCRIPTION OF THE QA-RecombineIT WEBSERVER

The QA-RecombineIt web server operates through two stages. In the first stage, called the QA-mode, starting from protein sequence and computational models of the protein, our server predicts both global and local accuracy of these models. In the second stage, called RecombineIt-mode, the server runs an algorithm that performs a ‘recombination’ of the best ranked parts of the input models into new hybrid structures that are likely to be better than the input models themselves. Such an approach, when used manually by human predictors, has proven to generate, on average, more accurate models than corresponding input models (50,51). Finally, QA-RecombineIt, in the last step, predicts the model quality for both the original input models as well as the hybrid models resulting from recombination and then selects the highest ranked model(s).

INPUT AND OUTPUT

QA-mode

The only data required as input are a protein sequence of the target protein either as a one-letter sequence or in the FASTA format and 3D structure models for that protein sequence. The models can be produced by a variety of protein structure predictors. Although a model does not need to have all of its residues modeled, the only restriction is that the number of a residue has to be in agreement with its number in the target sequence. On sequence submission to the server, a unique URL for the output of model quality assessment (QA-mode) is generated. The user may bookmark this URL. If a user provides an email address, she or he will be sent a reminder with the link to an output page once the job has been completed (Figure 2A). The results page is split into five panels. The ‘Summary’ panel does not only summarize the output but also allows users to select an MQAP to rank models or to visualize their local correctness. The ‘Tools’ panel allows users to either download the selected models annotated according to the local model quality (B-factor values modified according to predictions provided by MetaMQAP or ProQ2 or MQAPmulti) or to use the TM-score program (53) to superimpose the selected models and then visualize this superposition by JMOL (55) (Supplementary Figure S1). The ‘RecombineIt submission’ panel allows users to execute the RecombineIt algorithm by pressing the ‘Submit recombination of models’ button. Advanced users may want to activate an option, by which they can define ‘leading’ models and/or fragments; this can be accomplished by pressing the ‘advanced mode’ button (Figure 2B). This will force RecombineIt to build models that are as close as possible to one of the selected ‘leading’ models and/or to select fragments that are similar to those selected as ‘leading’ fragments for recombination. Here, it is important to emphasize that manual selection of ‘leading models’ can have a significant impact on the results. Thus, this functionality is only recommended to advanced users with experience in template selection for a given type of targets. The best template does not necessarily have to exhibit the highest sequence similarity with the target. Other important factors that should be considered include the accuracy of the structure (e.g. resolution and R-free of a crystallographic structure or the number of restrains per residue in an nuclear magnetic resonance structure) and the similarity between the physiological conditions (e.g. solvent, pH, ligands, quaternary interactions) of the template and the physiological conditions in which the user wants the target to be modeled. For example, this advanced option can be useful to those wanting to model a ligand-bound protein structure; users can select a model created on the base of a structure with a ligand as the ‘leading’ model. Selecting ‘leading’ fragments helps to have control over the local conformation (e.g. active site) of models built and to avoid the inclusion of templates that may have higher sequence similarity or better resolution, but exhibit ‘wrong’ conformation owing to interactions with other molecules or the absence thereof. The next panel, ‘Sequence 1D information’, shows secondary structure, solvent accessibility and disorder regions predicted for the target protein (see ‘Materials and Methods’ section). Each line of the ‘Model quality assessment’ (Figure 2B) panel contains global quality assessment for a given model. Local model quality (MetaMQAP or ProQ2 or MQAPmulti) is also reported in the form of a heat map (a spectrum of colors from blue to red represents the spectrum of residues predicted to be correct or incorrect). The last panel ‘Detailed information’ shows (dis)agreements between secondary structure and solvent accessibility predicted (from sequence) versus observed (in the model). Also the predictions of global and local accuracies are reported. The latter is presented as interactive charts of residue deviations (in Ångströms) in the function of a residue number. Finally, for each model, the image of its 3D structure is shown and colored according to the predicted local quality.

Figure 2.

QA-RecombineIt outputs. The example of the output of QA-mode (A). (B)—global and local quality of models. Local quality of each model is presented as a heat map. Once the user activates the ‘advance mode’, they can modify the operation of the fragment recombination algorithm by manually identifying the fragments and/or entire models on the base of which the hybrid model(s) will be created. The last panel (C) summarizes the fragment composition of the hybrid models generated. Word balloons indicate and explain the most important features of the results page by which the user can interact with the page; LBM, left mouse button.

RecombineIt-mode

Once a user clicks ‘Submit recombination of models’ button, the QA-mode output becomes an input to the RecombineIt-mode. As in the QA-mode, a unique URL for the output of RecombineIt-mode is generated, which the user may bookmark. The output page of the RecombineIt-mode is similar to that of the QA-mode, except for three differences. First, in contrast to the QA-mode, the RecombineIt-mode has an additional panel named ‘Model fragments’ that summarizes the fragment composition of the generated hybrid models (Figure 2C). At the top of this panel, an interactive model cloud is presented, where the font sizes correspond to the importance of each model in the recombination process, i.e. font size represents the number of times that a fragment from a given model has been applied to generate a hybrid model. Below the model cloud, a linear combination of fragments is presented for each hybrid model. Second, additional fields, named ‘Fragment definition’, are added to the output of the RecombineIt-mode to show the boundaries of the fragments used by RecombineIt. The last difference between the aforementioned two outputs is the lack of a ‘RecombineIt options and submission’ panel.

Molecular replacement mode

In protein crystallography, the determination of a 3D structure of a protein of interest entails gathering the intensity and phase information for a crystal diffraction pattern [review (56)]. Unfortunately, in a typical macromolecular X-ray diffraction experiment, only the intensities of reflections are measured, and information about phases is lost. The Molecular Replacement (MR) technique is one solution to this problem. MR approximates the phase information using a 3D structure of a related protein or a theoretical model of the protein under investigation (called a ‘search model’) (57). Recently, we have shown that using comparative models only marginally increases (by 4.5%) the MR success ratio in comparison with the structures of templates. However, as we have demonstrated, the situation changes dramatically once the comparative models are used together with their local accuracy (58). One of the ways to improve the utility of theoretical models of protein structure in MR is to recalculate the B-factor of each atom on the base of its predicted accuracy and then use such a modified search model with MR programs that take into account B-factor values as the indicator of uncertainties of atomic positions. By doing that, we have shown that predicted local accuracy of a model increases the MR success ratio by 45% compared with corresponding templates (58). Inspired by these findings, we have added functionality to QA-RecombineIt. By using the “Tools” panel, users can download models with B-factor values modified according to the MQAPmulti local score. Such models can be used as search models by MR programs that take into account atoms’ B-factor values, e.g. AMoRe (59) or MOLREP (60).

VALIDATION

Both components of QA-RecombineIt were tested in the CASP9 experiment, a community-wide blind assessment of computational methods for protein structure prediction. MQAPmulti was ranked the 4th best MQAP in predicting local model quality (http://predictioncenter.org/casp9/doc/presentations/CASP9_QA.pdf) and among the 10 best performing MQAPs for global model quality assessment (http://predictioncenter.org/casp9/doc/presentations/CASP9_QA.pdf). For local quality assessment (expressed in Ångströms), the Pearson’s correlation between the correct and predicted accuracy was 0.58. For global model quality (GDT_TS), the correlation was 0.96.

In the case of tertiary structure prediction in the CASP9 experiment, all targets were assigned two expiration dates: first, for automatic server predictors that can meet a 3-day deadline, and, second, for all other predictors, including slower automated methods and human groups. Models submitted for the first deadline were made available for predictors targeting the second deadline. During the CASP9 experiment, RecombineIt operated in a fully automatic mode and used as input only the models of automated server predictors submitted for the first deadline (see Supplementary Figure S2 for more details). Our method was run in its basic mode, i.e. neither ‘leading’ models nor fragments were selected manually. For the targets with at least one template to predict protein structure, including 52 template-based modeling (TBM) targets and 3 TBM/free modeling (FM) targets, the ‘Zhang’ group performed the best among all predictors (server and regular ones), achieving the sum of GDT_TS Z-scores = 51.89. The RaptorX server was ranked as the best among servers and 16th among all predictors (sum of GDT_TS Z-scores = 44.77). RecombineIt was ranked 4th with the sum of GDT_TS Z-scores = 49.69. Hence, RecombineIt run in its ‘basic’ mode produced predictions that were on the average better than all fully automated servers that generated models, which served as the RecombineIt input. Our method also outperformed most of the human expert groups, whose performance surpassed that of all of the servers and whose models were not used by our method.

Although RecombineIt performs well for template-based models, its predictive power is limited in cases of template-free modeling (Supplementary Table S1). For the FM targets, where no template existed to predict the target protein structures, the top-scoring ‘Eloffson’ group achieved the sum of GDT_TS Z-scores = 27.14, whereas the highest ranked server, QUARK, scored 23.08. In this category, our method achieved the GDT_TS Z-score sum of 21.31, which was worse that the score of the QUARK method, whose models were used as an input. Thus, RecombineIt can be safely recommended as a ‘consensus predictor’ for cases of difficult TBM, where the protein to be modeled may exhibit a known fold that is likely to be present in at least some of the input models. However, in the case of structures with completely new folds, and if good models are absent in the starting data set, our procedure cannot guarantee any improvement over the best starting model, even though it is likely to propose a solution that is better than the average.

CONCLUSIONS

The QA-RecombineIt server provides a unified interface for quality assessment and recombination of protein structure models. The algorithms underlying the QA-RecombineIt server were independently tested in the recent CASP9 competition and were found to be competitive in several categories, including model quality assessment and protein structure prediction of TBM targets and TBM/FM targets. As our server can be run with an input consisting of models generated by a variety of protein structure methods (both template-based and template-free), QA-RecombineIt can be used as the final step in any protein structure prediction procedure.

SUPPLEMENTARY DATA

Supplementary Data are available at NAR Online: Supplementary Table 1 and Supplementary Figures 1–2.