HIGH-ACCURACY PROTEIN STRUCTURES BY COMBINING MACHINE-LEARNING WITH PHYSICS-BASED REFINEMENT

Abstract

Protein structure prediction has long been available as an alternative to experimental structure determination, especially via homology modeling based on templates from related sequences. Recently, models based on distance restraints from co-evolutionary analysis via machine learning have significantly expanded the ability to predict structures for sequences without templates. One such method, AlphaFold, also performs well on sequences where templates are available but without using such information directly. Here we show that combining machine-learning based models from AlphaFold with state-of-the-art physics-based refinement via molecular dynamics simulations further improves predictions to outperform any other prediction method tested during the latest round of CASP. The resulting models have highly accurate global and local structure, including high accuracy at functionally important interface residues, and they are highly suitable as initial models for crystal structure determination via molecular replacement.

INTRODUCTION

High-resolution protein structures are key for understanding mechanistic details of biological function and for guiding rational drug design. The main experimental source of such structures is X-ray crystallography, nuclear magnetic resonance spectroscopy, or cryo-electronmicroscopy. By now, there is fairly comprehensive coverage of the manifold of protein structures encountered in biology, but a complete set of structures for any specific organism other than human immunodeficiency virus remains a very distant goal due to experimental limitations^1,2.

The computational modeling of protein structures has long been an alternative^3,4. Structures can be predicted with reasonable accuracy by using templates based on sequence homology⁵. More recently, high-accuracy models have also been derived based on intramolecular distance restraints according to inferred co-evolutionary relationships^6–8, especially for sequences where structural templates cannot be identified. Co-evolution analysis is highly suitable for the application of deep learning methods that are trained to predict residue interactions from multiple sequence alignments^9–18. This has been demonstrated most convincingly by DeepMind’s AlphaFold program during the recent 13^th Critical Assessment of Techniques for Protein Structure Prediction (CASP13)¹⁹. AlphaFold outperformed all other methods for ‘free-modeling’ (FM) targets where templates were not available but was also competitive in the ‘template-based modeling’ (TBM) categories, although, presumably, without using templates directly²⁰.

Predicted protein structures often capture secondary structures and fold topology correctly, but retain modeling errors compared to experimental structures²¹. Reaching experimental accuracy is the goal of structure refinement techniques. Physics-based methods based on molecular dynamics (MD) simulations have been most successful in improving both global and local structure^22–28. Current methods can provide consistent refinement for most structures while it is becoming increasingly possible to approach experimental accuracy based on root mean square deviations (RMSDs) of 1 Å or better as demonstrated during CASP13²⁹. Interestingly, refinement was especially successful for two targets that came initially from AlphaFold predictions. This suggested that MD-based refinement could allow significant improvements of machine-learning based models due to complementarity between the data-driven and physics-based approaches. Indeed, we find that the application of MD-based structure refinement to AlphaFold’s machine learning models submitted during CASP13 leads to models that overall surpass the accuracy of other approaches, remarkably even for targets where traditional template-based modeling is possible. The combination of machine learning and physics-based refinement therefore results in the highest-resolution protein structure models to date.

METHODS

Refinement of AlphaFold Models

The latest version of the PREFMD protocol²⁹ was applied to refine AlphaFold (A7D)’s “model 1”s generated during CASP13 for protein tertiary structure predictions. The AlphaFold models were downloaded from the CASP web page (http://predictioncenter.org/download_area/CASP13/). The overall refinement method consisted of pre-sampling, sampling, and post-sampling stages. Details are described elsewhere²⁹ and only the key points are briefly outlined here. In the pre-sampling stage, local stereochemical errors such as atomic clashes and poor backbone dihedral angles were resolved by locPREFMD²⁴. At the sampling stage, molecular dynamics (MD) simulations were conducted to explore conformational space. One of two variants was chosen based on target size. In the full iterative protocol, three iterations of MD simulations were carried out, while a more conservative protocol consisted of only a single iteration. For every iteration in the iterative protocol, MD simulations were started from new initial structures chosen from the previous iteration. In the iterative protocol, flat-bottom harmonic restraints were used to allow significant structure changes from the initial model within a certain radius. This protocol was used for smaller targets with radii of gyration smaller than 17 Å. In the conservative protocol, a harmonic restraint function was used to achieve more consistent, but moderate refinement for larger targets where extensive sampling would have been a challenge. The latest version of CHARMM force field, c36m³⁰, and its inhouse modification²⁹ were used for both sampling protocols. After the sampling stage, generated snapshots were evaluated by using the Rosetta score³¹. Conformations with low Rosetta scores were then subjected to ensemble averaging and locPREFMD was applied again to the ensemble averaged structures. Finally, residuewise errors were estimated by using root-mean-square-fluctuation (RMSF) from short MD simulations. We note that in the original method used during CASP13 we attempted to identify putative ligands, but this was not done here in order to implement a fully automatic protocol.

CASP13 assessment scores

For template-based modeling (TBM) category targets, the weighted Z-score sum of Global Distance Test-High Accuracy (GDT-HA)³², Local Distance Difference Test (lDDT)³³, Contact Area Difference-all atom (CAD-aa)³⁴, SphereGrinder³⁵, and Accuracy Self Estimate (ASE) scores³⁶ were used.³⁷ This score considers global structure similarity (GDT-HA), local structure similarity (lDDT, CAD-aa, and SphereGrinder), as well as estimates of residue-wise model errors. Free modeling (FM) category targets were assessed separately in CASP13 using a different score that consists of the weighted Z-score sum of Global Distance Test-Tertiary Structure (GDT-TS)³² and Quality Control Score (QCS)³⁸, both of which focus only on global structure similarity.²⁰

Homologous Sequence Search

To investigate correlations between the number of homologous sequences for a target and the quality of models from AlphaFold and its refined model we searched for homologous sequences based on the Uniclust30 database (clustered sequences of UniProtKB at 30% pairwise sequence identity)³⁹. HHblits⁴⁰ was used for the sequence search with default options except for the number of iterations set to 3. Sequences with a very higher pairwise sequence identity (above 90%) were filtered out using hhfilter, a program from HHsuite^41,42. Results were compared based on the Uniclust30 databases released in August 2018 and September 2016, but no significant differences were observed. The results reported here are based on the August 2018 release.

Protein Models as Molecular Replacement Search Models

Crystallographic data sets were downloaded from the Protein Data Bank (PDB)⁴³ and prepared by using Phenix.⁴⁴ Molecular replacement (MR) was conducted with Phaser⁴⁵ to evaluate log-likelihood gains (LLG) of protein models. We did not perform full MR searches to reduce the computational costs with the large number of models that were considered. Instead, we carried out rigid-body refinement after superimposition of a model on its corresponding component in the crystal structure. If there were multiple different components in the crystal, the model was evaluated after the other components were placed as background structures, and the LLG were calculated with the difference made by the model. Predicted residuewise model errors e were utilized as B-factors calculated as 8π²/3 e². We report the best LLG between the MR results with and without the predicted errors for a given model.

RESULTS AND DISCUSSION

We applied the physics-based model refinement protocol established during CASP13 to “model 1” predictions from AlphaFold (group A7D during CASP13) for all targets. The refinement protocol is based on MD simulations and does not apply any knowledge of the native structure or other data (see Methods).

Model accuracy

The model qualities for the generated models were evaluated according to the CASP13 assessment procedures³⁷, i.e., TBM-score and FM-score for targets in the TBM and FM categories, respectively (see Methods). Using the same scores as during CASP13 allowed us to directly compare with results from other groups without having to reevaluate all of the predictions. Figure 1 summarizes the scores for the AlphaFold models before and after refinement in comparison with models from other groups generated with different methods. As reported during CASP13, AlphaFold models were very competitive, ranking first in the FM category and 4^th in the TBM category^20,37. MD refinement further improved AlphaFold predictions. TBM targets were refined most, resulting in an increased TBM-score of 61.7 (from 44.6) that surpassed the performance of the best template-based modeling protocols. The improvements after refinement were greater for TBM-easy targets (36.0 vs. 24.2) than for TBM-hard targets (25.6 vs. 20.3), where ‘easy’ and ‘hard’ relate to the availability and quality of suitable templates. For FM targets, where templates are not available, refinement improved AlphaFold models moderately with an increased FM-score from 67.2 to 69.0. Improvements due to refinement were consistent across different targets. 78 out of 104 (75%) of targets were refined, including 34 out of 40 (85%) TBM-easy targets.

Figure 1.

Assessment of refined AlphaFold models in comparison with CASP13 results. (A) Scores (sum of assessor’s formula) for TBM (top) and FM (bottom) targets. Results for AlphaFold (A7D) before and after refinement are shown in red and blue, respectively. Other top-performing groups are shown in grey. TBM-easy and FM-hard targets are shown in darker colors, while TBM-hard and FM/TBM targets are shown in lighter colors. (B) Head-to-head comparison between AlphaFold models before and after refinement based on assessor’s formula. Each point corresponds to a target. Targets for TBM-easy, TBM-hard, FM/TBM, and FM-hard are depicted in red circles, orange squares, green triangles, and blue Xs, respectively. The number of better targets in each subcategory is shown at the top (refined model is better) and bottom (AlphaFold model is better) in the same colors.

Changes in individual quality measures after refinement (Figure S1 and Table S1) indicate that improvements were not limited to specific structure features, but that there was an overall enhancement in global and local structural quality as well as better residue-wise error estimation. During CASP13, AlphaFold generated the best overall models of any group for 25 out of 104 (24%) targets. AlphaFold+refinement resulted in the best models for 41 targets (39%) (see Figure S2). Interestingly, even though AlphaFold or our refinement method did not use templates, it was possible to generate the best models for 14 TBM-easy and 8 TBM-hard targets, respectively, significantly improving upon the accuracy of the unrefined AlphaFold models and surpassing the average performance of any other group in all categories based on Z-scores (see Figure S3 and 4, Table S2 and S3). Only in the TBM-easy category and only with respect to GDT-HA scores were the refined AlphaFold models exceeded by models from the Zhang group.

AlphaFold’s deep-learning method focuses primarily on predicted inter-residue distance distributions and backbone dihedral angles and the accuracy in the resulting models is essentially limited to the residue level. Moreover, model errors likely reflect structural variations within in a given protein family since contacts are obtained from deep multiple sequence alignments. This leaves room for physics-based approaches to provide significant refinement at the atomistic level. As may be expected, the accuracy of AlphaFold models depends on how many homologous sequences were available (see Figure S5A) while MD-based refinement success is independent of available homologous sequences (Figure S5B), further highlighting the complementarity of both approaches.

Figure 2 illustrates two typical examples of successful refinement: The initial AlphaFold models have correct folds, but they slightly mis-predict some regions such as loops and display suboptimal packing between side-chains. These issues were resolved by the refinement. Loop structures are presumably more difficult to predict by AlphaFold because there are fewer inter-residue contacts compared to secondary structure elements. On the other hand, the exact packing of side chains due to physical laws may be difficult to obtain simply based on sequence-derived contacts.

Figure 2.

Examples of successful refined models with (A) T0964-D1 for TBM category and (B) T0990-D1 for FM category. AlphaFold models and their refined models are shown in blue and magenta, respectively. Experimental structures are overlayed in yellow after superposition. Regions that were most significantly improved after refinement are indicated by green arrows. Model quality metrics are shown below each model.

Accuracy of protein-protein interfaces

Protein-protein interactions are important in the function of many proteins^46–48. There were 27 hetero- and 57 homo-oligomeric targets allowing the separate evaluation of the accuracy of interface residues (defined as residues with a heavy atom distance closer than 10 Å to another protein). The interface-RMSD (iRMSD) based on the backbone atoms of interface residues was lower than 4 Å in either initial or refined AlphaFold models for 14 hetero- and 22 homo-oligomeric targets. Among these targets, refinement improved not just the global structure (in terms of GDT-HA), but also the interface (in terms of iRMSD) (Figure 3A). On average, iRMSD values were decreased by 0.15 Å (p=4.1%, n=36) with improvements for 65% of the targets. Figure 3B illustrates a successful example of a signficanctly refined interface even although structures were refined as monomers. Two targets (T1015s1-D1 and T1019s1-D1) became significantly worse after refinement in terms of GDT-HA and iRMSD. These targets have a typical Zinc finger moiety but a Zinc ion was not included in the refinement. If these targets are excluded, the improvement in iRMSD increases to 0.22 Å (p=0.14%, n=34).

Figure 3.

Refinement at protein-protein interfaces. (A) Structure quality comparisons between AlphaFold and refined models in terms of interface RMSD (top) and GDT-HA (bottom). Hetero- and homo-olligomers are shown as red Xs and blue circles, respectively. (B) An example of refinement at the interface of T0997-D1. Both AlphaFold (top) and refined models (bottom) are shown in red and blue, while the experimental structure is shown in yellow and pink. Structures are depicted in cartoon representations, and interface residues are also shown as sticks. Models were superimposed onto the experimental structures by aligning the interface residues. Significantly improved interface regions are indicated by green arrows. Model qualities measures are given below each model.

Use of models for molecular replacement

A practical value of predicted structures is their use as starting models for solving crystal structures via molecular replacement (MR).⁴⁹ We followed the procedure that was used in the CASP13 assessment³⁷ where MR success in Phaser⁴⁵ was evaluated in terms of log-likelihood-gains (LLG). There are 19 crystallographic data sets available so far for a total of 27 CASP13 target domains: 12 single-component data sets for 12 target domains and 7 multi-component data sets consisted of multiple domains or heterooligomers for 15 target domains. (Table S4) LLG scores increased for most targets after refinement. LLG scores of 120 or higher often indicate the possibility for successful MR.⁴⁵ Between initial and refined AlphaFold models, this was reached for 14 targets and out of those the refined models had higher LLG than the AlphaFold model for 10 targets making success with MR more likely. Typically, MR relies on homology models with high sequence identity templates or MR-Rosetta⁵⁰ for sequences with lower sequence identity templates. The results presented here suggest that refined machine learning-based models are becoming a good alternative for solving crystallographic data via MR.

CONCLUSIONS

Co-evolutionary analysis and machine learning have significantly improved prediction accuracies for sequences where templates are not available but has also been demonstrated to perform well for sequences where templates could have been used instead. Here we show that physics-based refinement can further improve the accuracy of machine-learning models to exceed the accuracy of any other available method based on the targets that were assessed in CASP13. This suggests that a protocol that combines machine-learning to obtain residue-residue distance information and torsional preferences with distance-based model generation (such as in AlphaFold) followed by MD-based refinement to improve structural details at the atomistic level is emerging as a universal protocol for obtaining the most accurate predictions irrespective of whether template structures are available for a specific sequence or not. Additional analysis shows that functionally important residues at protein-protein interfaces can be predicted at high accuracy and that refined AlphaFold models are highly useful as search models for molecular replacement during crystallographic structure determination.