The power and pitfalls of AlphaFold2 for structure prediction beyond rigid globular proteins

Abstract

Artificial intelligence driven advances in protein structure prediction in recent years have begged the question: has the protein structure prediction problem been solved? Here, with a focus on non-globular proteins, we highlight the many strengths and potential weaknesses of DeepMind’s AlphaFold2 (AF2) in the context of its biological and therapeutic applications. We summarize the subtleties associated with evaluation of AF2 model quality and reliability using predicted local distance difference test (pLDDT) and predicted aligned error (PAE) values. We highlight various classes of proteins that AF2 can be applied to, and the caveats involved. Concrete examples of how AF2 models can be integrated with experimental data in the form of SAXS, solution NMR, cryo-EM, and X-ray diffraction are discussed. Finally, we highlight the need to move beyond structure prediction of rigid, static structural snapshots towards conformational ensembles and alternate biologically relevant states. The overarching theme is that careful consideration is due when using AF2-generated models to generate testable hypotheses and structural models, rather than treating predicted models as de facto ground truth structures.

Main

DeepMind’s AlphaFold2 (AF2) has revolutionized structural biology with its deep-learning algorithm that enables accurate prediction of three-dimensional protein structures from only the target amino acid sequence, potentially solving the half a century old protein structure prediction problem: how to predict 3D structures from only sequence information^1–4. AF2 has opened the door to understanding protein folds, structures, interactions, and function at organismal levels through modeling of 98.5% of the human proteome⁵. One common critique of AF2 is that it requires significant computational resources to run the software locally (up to 3 terabytes of disk space and a modern NVIDIA GPUs with gigabytes of memory). Several efforts have alleviated these limitations, including the AlphaFold Protein Structure Database^6,7, which houses over 200 million pre-run AF2 predictions, as well as ColabFold⁸ and OpenFold⁹, which allow users to run a modified AF2 protocol on open access servers in minutes. These platforms have allowed the public, industry, and academics without computational resources to model and analyze structures of their favorite target using AF2 with just a few clicks of a button. Another critique of AF2 concerns whether it has truly solved the protein structure prediction problem. Several groups have proposed that AF2 has only learned how to estimate three-dimensional structures using patterns extracted from known folds in the Protein Data Bank (PDB) and coevolutionary information between residues rather than the underlying physical and chemical basis of protein folding^3,10–12. This is strictly true since current versions of AF2 do not use energy functions that seek to identify native-like protein conformations, unlike its competitor Rosetta¹³. Others suggest that AF2’s algorithm may have indirectly learned a similar function¹⁴. Finally, some critics question the accuracy of the standard implementation of AF2 against different types of non-globular molecular targets, which could limit its potential applications^15,16. Overwhelming evidence suggests that machine learning software like AF2, RoseTTAFold, ESMFold, and related approaches are the best and most accurate answer to the structure prediction problem to date^1,17–19.

AlphaFold2’s AI-driven revolution

The basic workflow of AF2 is outlined in Fig. 1a. Users input the primary amino acid sequence of the target protein as one letter code in FASTA format. When more than one input sequence is provided AlphaFold-Multimer or AF2Complex is utilized^20,21. Lower and upper limits for input sequence lengths are defined by difficulties in generating reliable multiple sequence alignments (MSAs) for short (<10 amino acids) sequences and graphic processing / memory issues for long (>3,000 amino acids) sequences, respectively. Protein sequences can be obtained from annotated public databases, such as UniProt. The full details of the AF2 workflow have been discussed previously¹, but are briefly outlined below. Using the input sequence(s), AF2 first queries several databases to construct a pair representation and an MSA representation of the target. The pair representation is a matrix of pairwise interactions between amino acids that are likely to be spatially related (i.e., close to each other in space). The MSA representation is a collection of sequences that are evolutionarily related to the target sequence and provides mutational covariance information utilized by AF2. The pair and MSA representations are then passed through the Evoformer, a neural network block that exchanges information within the MSA and pair representations to establish spatial and evolutionary relationships. Next, the Structural Module parses information from the Evoformer to convert the representations into a three-dimensional protein structure. The entire process undergoes several rounds of iterative recycling to produce the final refined models. For each output AF2 generates a per-residue confidence score stored in the B-factor column of the model coordinate file (.pdb, .mmCIF, or related formats), the predicted Local Distance Difference Test (pLDDT) score, which ranges from 0 to 100 with higher values assigned higher confidence in the model (Fig. 1b)^1,22. AF2 also generates a Predicted Aligned Error (PAE) matrix, which evaluates the relative orientation and position of different parts (i.e., domains) of the model⁶. Higher PAE values correspond to lower confidence for the relative position and orientation of two parts of the protein in the model. Users should be especially careful to not assign biological or structural relevance to regions with low pLDDT (< 70) or high PAE values (> 5 Å)^5,23. However, as discussed below, high pLDDT or low PAE metrics, indicating high confidence in the prediction, do not promise agreement with native protein conformations, but instead estimate a likelihood for local and global coordinate positions and/or orientations.

Fig. 1: Overview of AlphaFold2.

a, The general workflow for an AF2 prediction is shown (derived from Jumper et al¹). The input is the primary amino acid sequence. The AF2 model of Ganglioside GM2 activator protein (https://alphafold.ebi.ac.uk/entry/P17900) is shown. The model is colored based on pLDDT value. b, Left: AF2 prediction for OSBP1 (https://alphafold.ebi.ac.uk/entry/P22059) is shown. The AF2 model is colored based on pLDDT value. The domains of the protein are noted: plextrin homology (PH), phenylalanines in an acidic tract motif (FFAT), OSBP-related ligand-binding domain (ORD), and coiled-coil domain (CC). Right: The AF2 PAE metrics shows the predicted relative position error for each residue in the sequence with low confidence values in white and high confidence values in green. The domains of OSBP1 have been manually annotated on the PAE graph.

An example AF2 model of oxysterol-binding protein 1 (OSBP1), a lipid transfer protein, obtained from the AlphaFold Protein Structure Database is shown in Fig. 1b. The pLDDT values plotted onto the model highlight that the PH, CC, and ORD domain structure are assigned very high to high predicted confidence, while the FFAT domain is predicted with very low confidence. The PAE graph reveals that the model has low confidence with respective to relative placement of PH, CC, FFAT, and ORD domains with respect to each other.

A critical evaluation of AlphaFold2’s applications

There are several open areas of research concerning AF2 in the context of its biological and therapeutic applications. First, accuracy evaluations of AF2 models relative to different types of protein folds present in the PDB, especially for new structures as they’re released. Second, expanding the types of systems that AF2 can be applied to, either through benchmarking the default AF2 pipeline on new types of targets, or through modifications in the AF2 protocol. A recent structural biology community assessment reports that on average AF2 generates models with quality near experimental structures across diverse target folds and applications. These types of studies with good reason affirm AF2’s utility but may give the impression that AF2 is without limitations. Below we summarize several potential applications of AF2 and provide examples where the predicted model deviates from the experimental structure. Cases where AF2’s performance is compromised are especially important to help us understand its limitations and provide opportunities to refine its deep learning-based algorithm in future iterations. These deviations can be broadly categorized into cases with i) inaccurate secondary, tertiary, and/or quaternary structure in regions where AF2 predicts low to moderate confidence, ii) inaccurate structure in regions where AF2 predicts high model confidence, iii) correct backbone structure but incorrect fine details (i.e., side chain rotamer placements), and iv) correct backbone structure for individual domains but inaccurate placement of domains relative to each other (Fig. 2a–l). For cases i and iv, low confidence in pLDDT scores and PAE graphs alerts users to interpret structures with caution, which is not immediately clear in cases ii and iii.

Fig. 2: Example applications of AlphaFold2 predictions that deviate from the experimental structure.

A superposition of the AF2 model and experimental structure for several classes of peptides/proteins are shown. AF2 models are colored according to the pLDDT value with overlayed experimental structures colored in pink. The PDB IDs of the experimental structures used for comparison are noted. AF2 models were fetched from the AlphaFold Protein Structure Database or derived the literature: a, PRNP (https://alphafold.ebi.ac.uk/entry/P23907), b, Insulin (https://alphafold.ebi.ac.uk/entry/P01308), c, Polycystin 2 (https://alphafold.ebi.ac.uk/entry/Q13563), d, PqqL (https://alphafold.ebi.ac.uk/entry/P31828), e, Complement Component C6 (https://alphafold.ebi.ac.uk/entry/P13671), f, ChRmine, g, At2g23090 (https://alphafold.ebi.ac.uk/entry/O64818), h, SecA (https://alphafold.ebi.ac.uk/entry/P28366), i, L-PGDS K59A/C65A, j, NT-9,k, A12 nanobody/HIV C186 gp120 complex, l, Beta-endorphin amyloid fibril. Red arrows denote areas where the AF2 model deviates from the experimental structure. It is important to note that predicted structures, gleaned from literature or the AlphaFold structure database, have been generated using different versions of the AF2 software and/or with different input parameters and thus cannot be directly compared.

The success of AF2 in predicting protein structures begs the question as to whether it can also accurately predict peptide structures (or, in some cases, lack of a well-defined structure). Peptide structure prediction poses additional challenges given that the benchmark set used to train AF2 excluded peptides, the difficulty in generating robust MSAs for short sequences, and observations that many peptides exist in solution as conformational ensembles rather than a single static conformation^1,24,25. McDonald et al performed a benchmark of 588 peptides revealing that AF2 predicts many α-helical and β-hairpin peptide structures with surprising accuracy²⁵. However, AF2 was challenged by mixed secondary structure membrane and soluble peptides, such as the prion protein PRNP (Fig. 2a)²⁵. It was also shown that the best ranked AF2 models (selected on the basis of high pLDDT score) often did not exhibit the lowest C_α root-mean-square deviation (RMSD) relative to the experimental structure, suggesting the pLDDT metric used by AF2 to assess protein models is not optimal for classification of peptide conformations²⁵. In a separate study, Tsaban et al showed that AF2 can be adapted to accurately model peptide/protein complexes irrespective of peptide length, although the results seemed biased towards helical structures and peptides that do not undergo large structural rearrangements upon binding²⁴. New methods are fine-tuning AF2-based pipelines for specific types of peptide/protein complexes (i.e., peptide/MHC)²⁶. These studies provide compelling evidence that AF2 can be applied across peptides, proteins, and peptide/protein complexes albeit with several limitations and caveats. Refinement of AF2 derived models with nuclear magnetic resonance (NMR) derived restraints, such as chemical shift values, torsion angles, residual dipolar couplings (RDCs), and nuclear Overhauser effects (NOEs) data, could help improve accuracy of peptide modeling¹⁵.

To date, the most common types of protein folds benchmarked in AF2 assessments are globular and extended / repeat proteins^1,5,17. NMR structural ensembles offer a unique validation metric to assess the accuracy of predicted AlphaFold2 models since AlphaFold2 was trained on a subset of the PDB that excluded NMR data^1,27–30. While AF2 performs exceptionally well on these types of folds on average, Fowler et al revealed that NMR ensembles can be more accurate than static AF2 models for dynamic proteins²⁸. As an example, the AF2 model of insulin deviates significantly from its experimental NMR structure (Fig. 2b), potentially due to the inability of AF2 to orient cysteine pairs for disulfide bond formation³¹. Another consideration is that AF2 models of many globular proteins, especially enzymes and metalloproteins, lack functionally relevant co-factors, prosthetic groups, or ligands. The authors of AF2 note that since it is trained on both apo and holo structures from the PDB, models may still be consistent with the expected structure in the presence of ligands or cofactors despite their absence in the AF2 workflow^1,5. However, whether the modeled structure resembles the apo or holo form of the protein is not immediately clear from analysis of pLDDT scores or PAE graphs³². Furthermore, deviations of AF2 models from experimental structures also occur when co-factors, prosthetic groups, or ligands induce structural changes, either locally or allosterically. For example, the NMR structure of Ca²⁺ bound polycystin 2 deviates from the AF2 model, potentially due to conformational changes upon calcium binding (Fig. 2c). Likewise, the AF2 model of the zinc protease PqqL deviates from the open, highly extended conformation determined by X-ray crystallography (Fig. 2d). New algorithms, such as AlphaFill, are actively being developed that could improve AF2 structure prediction and refinement for co-factor, prosthetic group, or ligand bound proteins³³. These modifications will enable AF2 to identify new therapeutic candidates³⁴. AF2 may also exhibit difficulties in structure prediction for extended proteins or proteins with repeat elements³⁵. In the case of the extended Complement C6 protein, AF2 predicts the structure of individual domains well but deviates in placement of domains relative to each other (Fig. 2e). For large macromolecules, users may be able to estimate the likelihood that AF2 correctly placed domains relative to each other by visualization of confidence scores in the PAE graph. However, it is important to remember that the PAE values are only confidence estimates. Furthermore, the accuracy of PAE graphs for inter-domain prediction has not been as extensively benchmarked as for intra-domain contacts³⁶.

Evaluation of membrane protein structure is another important application of AF2^5,37. Benchmarking AF2 against membrane proteins represents a challenge since the membrane environment, which includes lipids and other proteins, is not directly considered by current versions of AlphaFold³⁸. Furthermore, membrane proteins represent less than 3% of total structures in the PDB¹⁵, meaning that the training set used by AF2 was highly biased towards soluble proteins³⁹. Hegedűs et al benchmarked several membrane proteins not included in the original AF2 training set and concluded that on average AF2 predicts transmembrane proteins as well as soluble proteins³⁸. However, the authors note two important limitations. First, AF2 models with trans-membrane region lengths corresponding to non-physiological membrane thickness values can exhibit very high pLDDT scores (high model confidence), suggesting pLDDT scores alone are not sufficient to select native membrane protein conformations. Second, AF2 performs poorly for targets embedded in membrane thickness outside the 15–35 Å range as well as targets with novel features not commonly present in the PDB. In agreement with these findings, Azzaz et al have shown the difficulty in AF2 in modeling membrane proteins due to “epigenetic” factors (i.e., lipid environment, co-receptor induced structural changes, post-translational modifications) that control protein structure beyond the amino acid sequence⁴⁰. As an example, while the AF2 model of the channelrhodopsin ChRmine captures its overall fold, the modeled N-terminal region and extracellular loops deviate from its experimental high-resolution cryogenic electron microscopy (cryo-EM) structure (Fig. 2f), likely due to ChRmine’s unique covalent Schiff base feature⁴¹. It will be imperative to evaluate AF2 against membrane protein structures as they become more readily available as the result of high-resolution cryo-EM and advances in NMR spectroscopy.

Another unknown is how AF2 performs on intrinsically disordered proteins (IDPs) and proteins with intrinsically disordered regions (IDRs)^27,42. IDPs represents a challenge for AF2 since it is difficult to identify evolutionary constraints from MSAs of IDPs and IDRs due to sequence hypervariability. In addition, like peptides, IDPs and IDRs are best thought of as sampling diverse conformational ensembles rather than a single static conformation⁴². Preliminary studies suggest that the majority of targets with very low-confidence score (pLDDT < 50) assigned by AF2 are likely to be IDPs / IDRs rather than well-folded structures that AF2 fails to predict^42–44. However, for many targets, AF2 models with predicted disorder may not be relevant for structure and function analysis other than for assigning the likelihood for conformational heterogeneity²⁷. As an example, the NMR structure of IDR containing protein At2g23090 deviates from the AF2 model despite the confident pLDDT score (Fig. 2g). A study by Ruff et al showed that radius of gyration values of IDPs / IDR containing proteins calculated using static AF2 models significantly deviates from those experimentally obtained by small-angle X-ray scattering (SAXS)⁴². Future benchmarks should continue to evaluate AF2 against panels of IDPs and IDR containing proteins using novel Critical Assessment of Protein Intrinsic Disorder (CAID) targets⁴⁴. Efforts are also underway to establish whether AF2 can be used to predict alternative conformations or conformational ensembles of folded proteins (discussed in detail below). Several groups have suggested that the default AF2 pipeline has difficulty in modeling alternative conformations⁴⁵. For example, AF2 fails to predict the “open” activated conformation of the ATPase SecA (Fig. 2h). Interestingly, several groups have shown that modifications of AF2 have the potential to generate models that significantly deviate from each other, allowing for sampling of conformational landscapes. A study by del Alamo et al modified the AF2 pipeline by reducing the number of recycles and restricting the depth of randomly subsampled MSAs to sample functionally relevant alternative conformations of transporters and G-protein-coupled receptors⁴⁶. Similarly, Wayment-Steele et al found that clustering MSAs by sequence similarity enables AF2 to sample known alternative states of KaiB, RfaH, and Mad2⁴⁷. Further benchmarking of modified AF2 protocols against IDPs, IDR containing proteins, and alternative conformation is required to establish protein prediction strengths and limitations for those of systems^43,48.

There are several other challenging structural modeling problems in biology and therapeutics that AF2 is tasked with. One of the most sought-after applications of AF2 is predicting the effect of mutations on protein structure and/or stability¹⁷. The AF2 authors note that “AlphaFold has not been trained or validated for predicting the effect of mutations.” In support of this, studies have reported an inability of AF2 to predict the effects of mutations on protein structure and stability^14,49,50, which may be due to a training bias on stable structures or an inability to extract signal from small mutations through MSAs. As an example, structural perturbations induced by the K59A/C65A double mutation in Prostaglandin D Synthase are not accurately captured by AF2 (Fig. 2i). A recent adaption of AF, AlphaMissense, does not explicitly determine the structural effects of a mutation on a protein but provides the probability of a missense variant being pathogenic⁵¹. Other groups have suggested that developing AF2 workflows that are less depending on MSAs could be beneficial¹⁴.

Another challenge is modeling of novel three-dimensional folds that are either completely absent or not commonly represented in the PDB, such as de novo designed proteins. In these cases, AF2 has not been fully trained on novel topologies, which are not commonly found in the PDB. Furthermore, extraction of coevolutionary information from MSAs using de novo designed targets may be difficult since the amino acid sequences of de novo designed proteins deviate from naturally observed sequences. For some, this is ultimate test of whether AlphaFold may have solved the protein structure prediction problem. Interestingly, Moffat et al showed the AF2 performs well on the de novo designed proteins Top7, Peak6, Foldit1, and Ferredog-Diesel⁵². Slight deviations in tertiary structure are noted, such as for the nuclear transport factor 2 derived de novo designed protein NT-9, but the overall structure is well described (Fig. 2j). For targets without known homologs, such as computationally designed proteins, increasing the number of recycling iterations can improve the quality of the prediction⁸. An inverted version of AlphaFold, called AlphaDesign, has been used for de novo protein design with some success⁵³.

Evaluation of protein/protein interactions, including oligomerization, is another major potential application of AF2 that continues to be explored¹⁷. Yin et al benchmarked 152 heterodimeric protein complexes revealing that AF2 and AF2-Multimer had a 51% success rate^54,55. The authors note that AF2 had difficulty modeling antibody/protein complexes, such as the A12 nanobody/HIV gp120 complex (Fig. 2k). A separate study by Bryant et al reported a 63% success rate for heterodimeric complexes⁵⁶. Both studies suggest that a robust MSA coevolutionary signal is required for accurate complex modeling. Preliminary reports also suggest that AF2 may also be able to predict oligomeric states of proteins and amyloids^17,57. However, care must be taken when interpreting predictions, as highlighted by the incorrect AF2 model of the beta-endorphin amyloid fiber relative to its experimental solid-state NMR structure⁵⁸ (Fig. 2l).

Evaluation metrics and model reliability

As stated above, AlphaFold provides error categorizations in the form of pLDDT scores and PAE values to estimate confidence of its predictions and to evaluate overall model quality/reliability. For the majority of globular proteins, AF2 provides accurate, reliable models with high pLDDT (> 70) or low PAE values (< 5 Å) highlighting confidence in the prediction of the position of the atomic coordinates, which match experimentally determined native structures^5,17,23,59. In other cases, if the “best” AF2 model exhibits many residues with low pLDDT (< 70) or high PAE values (> 5 Å), the likelihood that the backbone structure matches the native conformation is very low and the model cannot be reliably interpreted. Previous analysis suggests that AF2 predicts on average ~50% of residues across all proteins with high confidence^17,59. Users can attempt to increase model quality (better pLDDT and PAE values) by generating a series of predictions with different parameters (number of recycles, number of random seeds, number of ensemble)¹ or employ integration with experimental data⁶⁰. However, cases in which where the AF2 evaluation metrics are good but the model doesn’t match experimental structure (Fig. 2a,d,e,k) suggest that care must be taken in blind faith in pLDDT and PAE metrics. The most dramatic case is when AF2 provides excellent evaluation metrics despite complete disagreement of the model’s backbone structure with an experimental structure. Terwilliger et al estimate that ~10% of residues predicted by AF2 with high confidence deviate from the backbone by more than 2 Å from native conformations observed in experimental structures⁶¹. There are also cases in which AF2 generates models with high confidence whose backbone structure is correct, but fine details, such as side-chain rotamer placement, are lacking. Jumper et al note that a rotamer is generally classified as correct if the predicted torsion angle is within 40° of the experimental torsion angle, which is correlated with pLDDT scores > 90¹. However, as noted by several groups, high pLDDT at a specific residue does not always indicate the correct rotameter has been modeled²³. Cases can also exist where AF2 predicts the correct backbone structure for individual domains but misplaces domains relative to each other, which should be recognizable in the output PAE matrices. Several groups have reported cases of AF2 models with low PAE values (< 5 Å) that deviated from experimental data^62,63. While no precise mechanism exists to identify these cases, some groups have employed molecular dynamics simulations to further evaluate stability and quality of AF2 models^64,65. Some groups have used MD simulations to suggest that pLDDT and PAE metrics provide information on dynamics and disorder^42,65. However, other reports have compared pLDDT scores with crystallographic B-factors to suggest AF2 confidence metrics are unable to provide direct information on local flexibility⁶⁶. The determinants driving cases where AF2 models are associated with high confidence but deviate from experimental structure are currently unknown and should be thoroughly evaluated in future studies, especially in the context of non-globular proteins, towards quantitatively defining limits of AlphaFold’s evaluation and error categorization metrics. Updated and refine approaches for error categorization may provide better methods for model quality assessment relative to pLDDT and PAE metrics^14,67.

Integration of AF2 models with experimental data

In cases where no experimental data is available (i.e., in vitro recombinant protein production or in situ characterization is not possible), insights into the structure and function of proteins may be primarily guided by AF2 predictions supplemented with molecular dynamics simulations to further evaluate model stability^64,65. In cases where recombinant protein can be prepared in milligram quantities required for biophysical characterization, AF2 models can be integrated with experimental data, typically in the form of SAXS, NMR, X-ray crystallography, and cryo-EM (Figure 3a–d). Here, experimental results are directly compared and contrasted against a series of AF2 models to evaluate which prediction, if any, adequately fits the data. AF2 models are increasingly used as initial templates to fit experimental data. The models subsequently undergo further refinement in an iterative fashion to match data towards generation of data driven structural models. Another possibility is the use of implicit experimental data to guide and restrain AF2 predictions (i.e., AF2 models are refined to best fit experimental data)⁶⁰. The integration of AF2 models with experiments is especially useful for cases where template structures or homologous models are lacking. The use of AF models in structure determination protocols has shown to reduce the time and effort required relative to ab initio model building^68–70.

Fig. 3: Integration of AlphaFold models with experimental data.

a, Schematic of using AF2 models together with either small angle X ray scattering (SAXS) or small angle neutron scattering (SANS) data. In this example, AF2 models are compared against SAXS data in the form of the pair distribution function, P(r) and log (I_q) vs q graphs. The SAXS envelope is fit together with AF2 models in an iterative fashion and refined to generate the final structure. b, Schematic of using AF2 models together with X-ray diffraction data. In the absence of an experimental template structure, AF2 models are iteratively used during the molecular replacement / phasing stages to process and fit the diffraction data in an iterative fashion. When the proper solution is found, the model is refined to generate the final structure. c, Schematic of using AF2 models together with solution NMR data. In one pathway, AF2 models are used together with experimental distance restraints (in the form of either nuclear Overhauser effects – NOEs, residual dipolar couplings – RDCs, paramagnetic relaxation enhancements – PREs, and/or pseudocontact shifts – PCSs) towards automated NMR resonance assignment via the predicted structure (in this case a 2D ¹H-¹³C methyl HMQC spectra). In another pathway, predicted distances in the AF2 models are compared to those obtained experimentally. If the restraints match, the AF2 model is validated and refined. If the experimental restraints do not match, the AF2 model can be refined/recalculated using those restraints. d, Schematic of using AF2 models together with cryo-EM data. 2D class averages obtained from cryo-EM experiments are reconstructed into 3D density maps. AF2 models are iteratively fitted into the cryo-EM density map and refined to generate the final structure. For panels a-d, all graphs / maps shown are conceptual (i.e., not real data). For all theoretical examples shown human glycolipid transfer protein (PDB ID 1SWX) was used.

As one example, theoretical SAXS profiles for a series of AF2 models can be predicted from the 3D coordinates and directly compared with experimental SAXS data in form of P(r) vs r or log (I_q) vs q plots where χ² values provide a goodness-of-fit measure for AF2 models relative to the solution state structure, which is time and ensemble averaged in SAXS^71–73. The best matching AF2 model is fitted into the experimental SAXS-derived envelope using a variety of software for further refinement⁷³ (Fig. 3a). Preliminary comparison of SAXS-derived versus AF calculated P(r) curves revealed that for many cases a static AF2 model does not adequately describe solution state structures^42,72. Recent methods have shown that fitting of SAXS data significantly improves when an ensemble of AF predicted structures is used rather than a static AF model⁷¹, highlighting the importance of integrating AF models with experimental data. An important caveat is that one must be wary of overfitting AF2 models to SAXS envelopes, especially for lower resolution data⁷⁴. Typically, χ² values of less than one are indicative of overfitting, and additional strategies such as the combination of V_c, Qr, X²_free, and R_sas metrics have been proposed as more robust evaluation metrics⁷⁵.

AF2 models have also been increasingly used during molecular replacement and phasing of X-ray diffraction data obtained from protein crystals^76–79 (Fig. 3b). Standard molecular replacement strategies require 3D coordinates of a template/homologous structure and work best when the template is < 2 Å C_α RMSD from the target structure⁸⁰. Recently, an AF2 integrated iterative procedure for molecular replacement has been developed where AF models are utilized during the initial structure-solution cycle, followed by data guided cycles of AF structure prediction and model rebuilding^60,69. This iterative procedure works extremely well as demonstrated in a benchmark where 187 of 215 structures were solved by AF-guided molecular replacement; the success was shown to be dependent on high confidence scores associated with the AF prediction⁶⁹. The use of AF models in molecular replacement can be further enhanced by downweighting or removing low confidence regions²³.

Another burgeoning area where AF models are integrated with experimental data is solution-state NMR^{15,27,28,30,81–85}. A series of AF2 models can be compared with experimental NMR data in the form of distance and conformation sensitive structural restraints obtained from NOEs⁸¹, RDCs^29,86, paramagnetic relaxation enhancements (PREs)^87,88, and/or pseudocontact shifts (PCSs)⁸⁹ (Figure 3c). If NMR-derived restraints match the AF2 model, the structure can be refined. Otherwise, the NMR-derived restraints can be used to recalculate the structure using the AF2 model as a template²⁸. Moreover, in the absence of NMR resonance assignments, AF2 models can be used as structural templates towards automated assignment⁹⁰. This is especially helpful for large biological assemblies where methyl side-chain labeling affords an increase in signal and resolution^91,92. Here, NMR assignments for methyl side-chain groups can be obtained using only methyl-methyl NOEs obtained from 3D NMR experiments and the atomic coordinates of a structure (or AF2 predicted structure) as input with software such as MAUS, MAGIC, and methylFLYA^91,93,94.

AF models have also been used extensively together with single particle cryo-EM data^60,76,95,96 (Figure 3d). 2D class averages generated from tens of thousands of particle images are used as the input for 3D classification and reconstruction. A series of AF2 models are fitted into the 3D cryo-EM density maps and each is evaluated for goodness-of-fit⁹⁶ and can be refined to generate a final structure⁹⁷. Like for X-ray diffraction studies, implicit incorporation of AF2 models, which are iteratively rebuilt on the basis of cryo-EM data, enables swift and robust structure modeling relative to ab initio model building^60,98. Here, the resolution of the cryo-EM data is essential for accurate model building. Reports suggest AF2 models should not be fit into cryo-EM maps with resolution greater than 6Å^98,99.

Beyond static snapshots: ensembles and conformational landscapes

Native conformations of proteins are often described as time averaged ensembles of conformations with Boltzmann-type distributions, especially IDPs and proteins with IDRs^100,101. Apart from IDPs, well-folded globular proteins, such as G-protein coupled receptors and kinases, also sample a wide range of conformations to carry out their biological function^102,103. The standard implementation of AlphaFold performs well only in detecting a single structural snapshot (the “ground state” structure), likely due to the lack of a large set of redundant protein conformers in the training set^16,32. Thus, several groups have worked to extend AlphaFold to include predictions of structural ensembles and excited state structures^102,104. Initial efforts to enhance sampling of different conformations have involved altering the number of sequences used to generate shallower MSA representations, masking coevolutionary information provided by MSAs, and splitting conflicting coevolutionary signals by clustering MSAs^46,47,105. Another approach is to enable dropout layers in the neural network, which are usually commonly used only during neural network training^8,106,107. These approaches have shown great promise in increasing the ability of AF2 to predict alternative conformations, although benchmarking has been limited by the small number of structures in the PDB solved with multiple conformations. The use of experimental data, especially SAXS, NMR, and cryo-EM, has also been described to guide modeling of ensembles and alternative conformations^71,82,108.

Conclusions and outlook

AlphaFold and other machine learning based structure prediction software represents a giant leap forward in our understanding of protein function and structures. However, they are not yet “one-sized fits all” solutions to the protein structure prediction problem. Current implementations of AF2 can provide highly accurate working models for most rigid, well-folded globular proteins, but may have issues predicting other classes of proteins. However, as suggested by recent work, we expect incredible progress in other classes of proteins in the coming years^33,37,47,51. Machine learning approaches are also expected to be applied towards structure prediction of biomolecules, including nucleic acids¹⁰⁹, carbohydrates¹¹⁰, and lipids. The case studies highlighted here reveal why caution must be taken in naive interpretation of AF2 models, even for cases with reasonable pLDDT and PAE confidence metrics (Fig. 2a–l). We expect that future studies will enable further refinement of error categorizations by teasing out fine details of cases with good evaluation metrics that don’t match experimental results. We also expect to see increase integration of experimental data with AF2 predictions. Several of the studies mentioned here also show that simple modifications in the AF2 workflow can further extend its accuracy and applications into new horizons. On average >10,000 protein structures are released in the PDB per year (https://www.rcsb.org/stats/growth/growth-protein). AF2 will continue to be evaluated against new experimental structures to further identify areas for improvement. Even in the face of an impressive display of accuracy, AlphaFold2 is still best utilized to complement and extend interpretation of experimental data at both structural and functional levels.

Data Availability

PyMOL sessions containing comparisons of AlphaFold models (extracted from the literature or AlphaFold Database) compared with experimental structures together with python script used to color coding structures based on pLDDT values are freely available at https://github.com/mcshanlab/AlphaFold_Models_Agarwal_McShan.