Abstract
The ‘structure assessment’ web server is a one-stop shop for interactive evaluation and benchmarking of structural models of macromolecular complexes including proteins and nucleic acids. A user-friendly web dashboard links sequence with structure information and results from a variety of state-of-the-art tools, which facilitates the visual exploration and evaluation of structure models. The dashboard integrates stereochemistry information, secondary structure information, global and local model quality assessment of the tertiary structure of comparative protein models, as well as prediction of membrane location. In addition, a benchmarking mode is available where a model can be compared to a reference structure, providing easy access to scores that have been used in recent CASP experiments and CAMEO. The structure assessment web server is available at https://swissmodel.expasy.org/assess.
Graphical Abstract
Graphical Abstract.
Introduction
3D structures of macromolecules are essential to understand the mechanisms of cellular processes, including signal transduction, gene expression, and metabolism, thus paving the way for advancements in fields ranging from basic research in structural biology to medicine and biotechnology. Experimental methods such as X-ray crystallography and high-resolution cryo-electron microscopy are employed to generate atomic-level structure representations, which are deposited in the Protein Data Bank (PDB) (1). As experiments are laborious in nature, computational modeling approaches provide an attractive complement. Comparative modeling involves deriving coordinates from structurally similar homologues with known structures. Over the decades, user-friendly pipelines for this process have been developed (2,3). Efforts were not only made for modeling individual targets but also large scale modeling efforts resulting in databases of theoretical models such as the SWISS-MODEL Repository (4,5) and ModBase (6). More recently, deep-learning-based methods such as AlphaFold2 (7), ESMFold (8) and RoseTTAFold (9) were increasingly successful in modeling accurate protein structure models, even in the absence of a clear homologue as a template, which led to the creation of the AlphaFold Protein Structure Database (10) and the ESM Metagenomic Atlas (8).
After CASP14 (11) and the release of AlphaFold2, attention more and more shifted towards challenges beyond tertiary structure prediction. New prediction categories were added to CASP15 (12), with an increased focus on protein complexes (13–15), the introduction of assessment of RNA structure predictions (16) and protein–ligand complexes (17). Assessing macromolecular complexes comes with a set of new challenges ranging from the need for reliable, accurate and fast chain mapping (14) to the development of scores to comprehensively measure the accuracy of these new types of predictions.
In this abundance of structural data, careful assessment of data quality is critical. The wwPDB consortium initiated such efforts by convening task forces specific to the underlying experimental methods to define and recommend relevant quality criteria (18–21). In the case of theoretical models, the provided data on structural quality is even more heterogeneous and dependent on the methodologies used to make the prediction.
In this article, we present a web server dedicated to assessing the quality of structures by providing access to a diverse set of structure evaluation tools in a user-friendly manner. The web server readily accepts any structural model, including those from sources described above.
A user-friendly and fully interactive web view connects sequence with structure information and with results from the structure evaluation tools and allows users to inspect quality indicators of a structure model in an integrated manner. The view includes stereochemistry information (from MolProbity (22) and Ramachandran plots (23)), secondary structure information (from DSSP (24)), global and local model quality assessment of the tertiary structure of protein models with QMEANDisCo (25), and prediction of membrane location. In addition, a benchmarking mode is available where a model can be compared to a reference structure with a comprehensive set of scores that have been used in recent CASP experiments (14–17,26) and CAMEO (27). The scores include assessments of local and global accuracy at the fold level and at interfaces of any type of macromolecular complexes. Hovering and clicking on evaluation results is directly highlighted in the sequence and structure views, which greatly facilitates the visual exploration and evaluation of structure models. Multiple selections can be performed by holding the Control or Command key.
The structure assessment web server is integrated within SWISS-MODEL and has received regular updates since its initial release 5 years ago. In 2024, on average the tool assessed >600 structures every day, emphasizing its usefulness.
Implementation
Overview of the web server functionality
The structure assessment web server is a dashboard organized into ‘components’, which display analysis results, and are connected to each other and to a 3D viewer (Figure 1). The dashboard adapts automatically to any given screen resolution and additionally allows the user to rearrange their own display of results to facilitate the analysis. For instance, arranging the alignment of model to reference chain next to the MolProbity results and 3D viewer, facilitates the analysis of outliers in sequence and structure space. In order to keep the interface clean and readable, additional information is revealed in tooltips that appear upon hovering the results.
Figure 1.
The fully interactive structure assessment workspace. Results can be saved for future reference (A). The ‘Alignments’ component (B) and 3D viewer (C) display the uploaded structure. The structure viewer (D) buttons enable customizing the display. Stereochemistry assessment results are available through the ‘Ramachandran Plots’ (E) and ‘Molprobity Results’ (F) components. Hovering over individual results provides additional information (G) and immediately highlights the respective locations in the ‘Alignments’ component (B) as well as the interactive 3D view (C). ‘Membrane Prediction’ (H) suggests that the uploaded structure possesses the characteristics of a transmembrane protein which is also displayed in the 3D view. This workspace additionally enabled the ‘QMEANDisCo’ (I) component which displays predicted per-residue model qualities, and the ‘Comparison to Reference’ (J) component which automatically establishes the most likely chain-to-chain assignment with a user defined reference structure and displays metrics specific for per-residue and full structure similarities as well as oligomeric interface similarities.
The structure assessment allows directly uploading any coordinates of a macromolecule or macromolecular complex in legacy PDB format or in the preferred PDBx/mmCIF format (28). This mode offers the most flexibility, with the choice of assessment tools to run and the possibility to upload a reference structure. Uploaded models are validated on the fly upon upload and undergo basic pre-processing to ensure they can be handled properly by the different tools that are offered by the server. Atoms or residues may be removed if they do not follow the nomenclature from the Chemical Component Dictionary from the PDB (29). In addition, biological assemblies are extracted from uploads in the PDBx/mmCIF format, which might result in the renaming of chains. Both the uploaded and pre-processed structure files can be downloaded for further manual inspection.
In addition, assessments can be performed directly from the SWISS-MODEL interactive modeling workbench. SWISS-MODEL provides QMEANDisCo as quality assessment for homology models (25,30), and predicted lDDT (pLDDT) (7) for models based on templates from the AlphaFold Database (10). However, around 20% of users forward their models to the structure assessment server in order to gain additional annotation from MolProbity, the Ramachandran plot and simplified display of target-template alignment. Finally, users can also run the structure assessment directly from the SWISS-MODEL Repository (SMR). This is particularly useful to assess experimental structures from the PDB or models from AlphaFold Database and ModelArchive (https://modelarchive.org/) as SMR displays structures from these resources per UniProtKB identifier. Currently, around 90% of assessments are started directly from SWISS-MODEL and SMR.
Once the analysis is completed, the results can be archived either as a fully interactive package to be uncompressed and displayed in a web browser directly from the local file system, or as a simpler package containing the results in JSON format together with the input coordinates file(s). Both are accessible from the ‘Project Data’ button (Figure 1A). Results are kept on our servers for a duration of 2 weeks. They can be accessed and shared by keeping track of their URL. Optionally, users can supply an email address or register an account to easily keep track of their results across browsers and computers. This website is free and open to all users and there is no login requirement.
Sequence and 3D viewers
The structure assessment dashboard provides a direct view of the sequence (‘Alignments’ component, Figure 1B) and three dimensional structure (Figure 1C) of the submitted model. The default 3D viewer is NGL (31). The two components are tightly integrated: hovering a residue in the structure highlights it in the sequence viewer, while clicking on the sequence centers onto the residue. It is also possible to select a stretch of sequence to zoom the 3D viewer to it. This allows to quickly identify the relationship between the sequence and the structure. Coloring can be controlled from the sequence and 3D viewer settings button (‘cogwheel’ button, Figure 1D) and affects both views simultaneously. Several color schemes are available, from quality estimates (model confidence or B-factors) to chemical properties or secondary structure.
The contents of the ‘Alignments’ component varies depending on the source of the model. In case a reference structure was uploaded alongside the model, the ‘Alignments’ component displays the reference-model alignment. Extra controls in the 3D viewer settings (Figure 1D) allow tuning the opacity of the reference structure to ease the analysis. If the assessment was started from the SWISS-MODEL workbench, the target-template alignment is displayed. In all other cases, the sequence extracted from the structure file is displayed in place of an actual alignment.
Finally, structures can be interactively annotated which enable custom coloring of specific residues and motifs, adding custom labels or highlighting side-chains of specific residues.
Stereochemistry analysis
Two web server components facilitate the evaluation of physical viability: ‘Ramachandran Plots’ (Figure 1E) and ‘MolProbity Results’ (Figure 1F).
A Ramachandran plot (23) provides an assessment of the overall plausibility of protein conformation, achieved by comparing the backbone dihedral angles of a structure with data extracted from 12 521 non redundant experimental structures (pairwise sequence identity cutoff 30%, X-ray resolution cutoff 2.5Å) as culled from PISCES (32). The contour plots allow rapid identification of unlikely conformations similar to other tools such as PROCHECK (33) or MolProbity (22) but seamlessly integrate with other web server components. The plot is directly linked to the ‘Alignments’ component (Figure 1B) and the 3D viewer (Figure 1C), so that hovering a point highlights the residue in these components, enabling the quick localisation of outliers in the sequence and structures. In addition, a tooltip displays additional information and results about the residue (Figure 1G).
Conversely, MolProbity (22) enables a more fine-grained all-atom stereochemistry validation for both proteins and nucleic acids. The structure assessment web server runs MolProbity version 4.4 as available from https://github.com/rlabduke/MolProbity and displays a selection of the most important validation results, including an overall score which reflects the crystallographic resolution at which such a quality would be expected. Additional details about the meaning of each of the scores and their interpretation is available from the structure assessment help page. Just like the Ramachandran plot, hovering an item in the displayed results (Figure 1F) directly highlights the problematic residue(s) in the sequence and 3D viewers and displays a tooltip with detailed information.
Membrane prediction
The ‘Membrane prediction’ component (Figure 1H) computes the most likely membrane location based on structural information using the membrane finding algorithm of the QMEANBrane tool (34) which is based on the solvation model described for the Orientations of Proteins in Membranes database (35). The results serve as input for a classification based on energetic and geometric criteria which aims at minimizing false positives. A full benchmark and evaluation is available in Supplementary Data 1. If a transmembrane segment is likely, it gets displayed in the 3D viewer. The full parameters are available in the downloadable JSON file.
Quality estimates
Quality estimates of theoretical models are an indispensable ingredient of any modeling pipeline as the expected accuracy of a theoretical model ultimately determines its usefulness for the biological question at hand. Structure models often come with their own quality estimates. For instance, models from AlphaFold2 or ESMFold contain a per-residue pLDDT estimate which can be visualized by selecting one of the ‘Confidence’ color schemes (Figure 1D). Homology models from SWISS-MODEL or SMR display a similar per-residue confidence score, QMEANDisCo.
In addition, the ‘QMEANDisCo’ component (Figure 1I) can be enabled for any homology-based protein model by activating the corresponding checkbox upon upload. It provides quality estimates for global model quality as well as per-residue estimates using the QMEANDisCo method (25). Per-residue estimates are visualized in the ‘Alignment’ component with a color gradient which can also be mapped as ‘Confidence’ color scheme onto the model in the 3D viewer to identify problematic regions. QMEANDisCo strongly relies on template information and should be analyzed with care for de novo methods such as AlphaFold2.
AlphaFold2 has introduced a new type of quality estimate, the Predicted Aligned Error (PAE), which no longer estimates the accuracy of single residues, but measures the confidence in the relative position of two residues within the predicted structure. While the PAE provides valuable information about the prediction, we decided not to include it in our assessment as it is very specific to AlphaFold2 models and has been well handled elsewhere (36).
Comparison to reference
By selecting the ‘Compare to reference’ checkbox, users can activate the respective component (Figure 1J) to compare theoretical models of macromolecular complexes to experimentally determined references for benchmarking purposes. This enables access to a comprehensive set of scores that have been used in recent CASP experiments (14–17,26) and CAMEO (27). They focus on local per-residue similarity (lDDT), global structure similarity (lDDT, RMSD, TM-score), and interface-specific similarity (QS-score, DockQ). While most of the scores have been designed with proteins in mind, most are useful also for the analysis of RNA structures. The component is split into three parts for which we provide a short summary in this manuscript. More detailed information can be found in the structure assessment server help page (https://swissmodel.expasy.org/assess/help).
Chain mapping: A one-to-one correspondence between model and reference chains is a prerequisite for scoring. The underlying algorithm was used in CASP15 (14,15) and aims to provide the optimal mapping with respect to the superposition independent QS-score (37). The mapped chains are displayed in a table which is connected with the 3D viewer for visualization and interactive exploration of the mapping. Non-mapped chains, for example in case of different stoichiometries in model and reference, are displayed separately.
Comparison scores assess the similarity between the model and reference structures. The displayed lDDT, TM-score and RMSD report on different aspects of structural similarity.
The local distance difference test (lDDT) (38) is a full atomic score and assesses the differences in interatomic distances between model and reference structure. It is thus inherently superposition free. As opposed to the original publication, lDDT displayed here natively supports oligomers as well as RNA/DNA for which it was extensively tested (15,16,39). Interatomic distances across interfaces get treated as any other pairwise distance. The global lDDT for the full model is displayed in the result table. Per-residue lDDT values are available in the interactive sequence display, which also allows to map lDDT as a color gradient onto the model in the 3D viewer by clicking on the ‘Display options’ (‘cogwheel’) icon in the display settings at the bottom left corner of the viewer (Figure 1D). If stereochemical irregularities are present that affect lDDT, they are displayed right underneath the global lDDT score and can be explored interactively.
TM-score (40) as computed with USalign (41) assesses the topological similarity of macromolecular complexes including RNA. It relies on a global superposition and considers backbone coordinates (Cα for amino acids, C3’ for nucleotides).
RMSD uses backbone coordinates (Cα, C3’) from all mapped residues in the full complex to derive a Kabsch superposition (42) which inherently comes with the respective RMSD value.
Interface scores complement measures of general structural similarity with interface centric similarity measures. The displayed global QS-score, the QS-best variant and DockQ-wave are color coded to provide a visual guide for model similarity. Additionally, a table which is linked with the 3D view allows users to interactively explore each protein-protein interface with their respective interacting chains, numbers of residue contacts, and per-interface QS- and DockQ scores.
QS-score (37) quantifies the similarity between interfaces as a function of interface contacts. QS-score can be computed for individual interfaces but the main intention is to aggregate contributions from all interfaces to get a single score of the full complex. QS-score has been updated to incorporate contacts involving nucleotides which enables chain mapping also with RNA and DNA. QS-best only considers the part of the structure that is present in both the model and the reference. This can be useful if the model is a substructure of the reference, or if the reference is not fully resolved.
DockQ (43) combines the established protein-protein interface similarity metrics Fnat (conserved fraction of native contacts), iRMS (interface RMSD) and LRMS (ligand RMSD) from the Critical Assessment of PRedicted Interactions (CAPRI) community (44) into one continuous score. It strictly processes single protein–protein interfaces but is complemented by DockQ-wave (14) which scores full protein complexes as a weighted average of per-interface DockQ scores. Protein-nucleotide interfaces do not affect DockQ-wave and weights are the number of native inter-chain contacts of each interface in the reference structure.
All described metrics are computed with the ‘compare-structures’ action (https://openstructure.org/docs/actions/) of the OpenStructure computational structure biology framework (45). The action provides a full benchmarking suite and can be executed locally for large scale analysis.
Similar tools
Several web servers and desktop tools provide a similar or overlapping set of features, here we list some of the most relevant ones. MolProbity (22) is the gold standard tool for stereochemistry analysis. A web server is provided to avoid the need to install the software. While the MolProbity web server provides additional capabilities (such as basic structure manipulation tools like adding hydrogens, filling small loops and NQH flips) and reports more detailed information, it does not provide an interactive, direct integration of stereochemical check results to 3D and sequence viewers. The ProteinPlus web server (46) provides a collection of related structure analysis and modeling tools, with a focus on protein-ligand complexes. PDBsum (47) is a database of detailed structure analysis of entries from the PDB and the AlphaFold Database and allows upload of arbitrary models. Analysis includes non-interactive sequence analysis as well as a Ramachandran plot. Models from the AlphaFold Database are colored according to the attached pLDDT. In addition, several molecular graphics desktop applications such as Coot (48) and UCSF ChimeraX (49), as well as the Phenix package (50), enable somewhat similar analyses with Ramachandran plots and stereochemistry checks, together with a direct interactivity with the 3D viewer. It should be noted that the comparison to a reference is a unique feature of the structure assessment web server and the underlying OpenStructure tool.
Conclusions
The structure assessment web server is a one-stop shop that tightly integrates an array of state-of-the-art tools that facilitate the analysis of the quality and accuracy of structural models. Stereochemistry analysis with Ramachandran plots and MolProbity analysis report on the physical viability of the model. Quality estimates are either displayed or computed for homology models and inform on the estimated accuracy of the predictions. By uploading a reference structure, a benchmarking mode scores the model coordinates against a known structure. Sequence and 3D structure viewers, together with the prediction of membrane location, connect all these results in an interactive and user-friendly interface that allows the visual exploration and evaluation of structure models.
The structure assessment web server is highly extensible. We are working on new components to be added such as display and scoring of protein–ligand interactions as well as additional sequence annotation.
Supplementary Material
Acknowledgements
We would like to thank all SWISS-MODEL users and particularly the ones subscribed to the beta testing program for their continuous feedback, which motivates and inspires the continuous development of SWISS-MODEL. We thank the sciCORE center for scientific computing (https://scicore.unibas.ch) at the University of Basel for providing computational resources and system administration support. This work was supported by funding from the SIB Swiss Institute of Bioinformatics (https://www.sib.swiss/) and the Biozentrum of the University of Basel (https://www.biozentrum.unibas.ch/).
Contributor Information
Andrew M Waterhouse, Biozentrum, University of Basel, Switzerland; SIB Swiss Institute of Bioinformatics, Computational Structural Biology, Basel, Switzerland.
Gabriel Studer, Biozentrum, University of Basel, Switzerland; SIB Swiss Institute of Bioinformatics, Computational Structural Biology, Basel, Switzerland.
Xavier Robin, Biozentrum, University of Basel, Switzerland; SIB Swiss Institute of Bioinformatics, Computational Structural Biology, Basel, Switzerland.
Stefan Bienert, Biozentrum, University of Basel, Switzerland; SIB Swiss Institute of Bioinformatics, Computational Structural Biology, Basel, Switzerland.
Gerardo Tauriello, Biozentrum, University of Basel, Switzerland; SIB Swiss Institute of Bioinformatics, Computational Structural Biology, Basel, Switzerland.
Torsten Schwede, Biozentrum, University of Basel, Switzerland; SIB Swiss Institute of Bioinformatics, Computational Structural Biology, Basel, Switzerland.
Data availability
The Structure Assessment web server is available at https://swissmodel.expasy.org/assess. This website is free and open to all users and there is no login requirement.
Supplementary data
Supplementary Data are available at NAR Online.
Funding
SIB Swiss Institute of Bioinformatics; University of Basel. Funding for open access charge: SIB Swiss Institute of Bioinformatics.
Conflict of interest statement. None declared.
References
- 1. wwPDB consortium Protein Data Bank: the single global archive for 3D macromolecular structure data. Nucleic Acids Res. 2019; 47:D520–D528. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2. Lam S.D., Das S., Sillitoe I., Orengo C. An overview of comparative modelling and resources dedicated to large-scale modelling of genome sequences. Acta Crystallogr. D Struct. Biol. 2017; 73:628–640. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3. Studer G., Tauriello G., Bienert S., Waterhouse A.M., Bertoni M., Bordoli L., Schwede T., Lepore R. Modeling of Protein Tertiary and Quaternary Structures Based on Evolutionary Information. Methods Mol. Biol. 2019; 1851:301–316. [DOI] [PubMed] [Google Scholar]
- 4. Bienert S., Waterhouse A., de Beer T.A.P., Tauriello G., Studer G., Bordoli L., Schwede T. The SWISS-MODEL Repository-new features and functionality. Nucleic Acids Res. 2017; 45:D313–D319. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5. Robin X., Waterhouse A.M., Bienert S., Studer G., Alexander L.T., Tauriello G., Schwede T., Pereira J. The SWISS-MODEL repository of 3D protein structures and models. Methods Principles Med. Chem. 2024; 83:175–199. [Google Scholar]
- 6. Pieper U., Webb B.M., Dong G.Q., Schneidman-Duhovny D., Fan H., Kim S.J., Khuri N., Spill Y.G., Weinkam P., Hammel M. et al. ModBase, a database of annotated comparative protein structure models and associated resources. Nucleic Acids Res. 2014; 42:D336–D346. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7. Jumper J., Evans R., Pritzel A., Green T., Figurnov M., Ronneberger O., Tunyasuvunakool K., Bates R., Žídek A., Potapenko A. et al. Highly accurate protein structure prediction with AlphaFold. Nature. 2021; 596:583–589. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8. Lin Z., Akin H., Rao R., Hie B., Zhu Z., Lu W., Smetanin N., Verkuil R., Kabeli O., Shmueli Y. et al. Evolutionary-scale prediction of atomic-level protein structure with a language model. Science. 2023; 379:1123–1130. [DOI] [PubMed] [Google Scholar]
- 9. Baek M., DiMaio F., Anishchenko I., Dauparas J., Ovchinnikov S., Lee G.R., Wang J., Cong Q., Kinch L.N., Schaeffer R.D. et al. Accurate prediction of protein structures and interactions using a three-track neural network. Science. 2021; 373:871–876. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10. Varadi M., Bertoni D., Magana P., Paramval U., Pidruchna I., Radhakrishnan M., Tsenkov M., Nair S., Mirdita M., Yeo J. et al. AlphaFold Protein Structure Database in 2024: providing structure coverage for over 214 million protein sequences. Nucleic Acids Res. 2023; 5:D368–D375. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11. Kryshtafovych A., Schwede T., Topf M., Fidelis K., Moult J. Critical assessment of methods of protein structure prediction (CASP)—Round XIV. Proteins: Struct. Funct. Bioinf. 2021; 89:1607–1617. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12. Kryshtafovych A., Antczak M., Szachniuk M., Zok T., Kretsch R.C., Rangan R., Pham P., Das R., Robin X., Studer G. et al. New prediction categories in CASP15. Proteins: Struct. Funct. Bioinf. 2023; 91:1550–1557. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13. Lensink M.F., Brysbaert G., Raouraoua N., Bates P.A., Giulini M., Honorato R.V., van Noort C., Teixeira J.M.C., Bonvin A.M.J., Kong R. et al. Impact of AlphaFold on structure prediction of protein complexes: The CASP15-CAPRI experiment. Proteins: Struct. Funct. Bioinf. 2023; 91:1658–1683. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14. Studer G., Tauriello G., Schwede T. Assessment of the assessment-All about complexes. Proteins. 2023; 91:1850–1860. [DOI] [PubMed] [Google Scholar]
- 15. Ozden B., Kryshtafovych A., Karaca E. The impact of AI-based modeling on the accuracy of protein assembly prediction: insights from CASP15. Proteins. 2023; 91:1636–1657. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 16. Das R., Kretsch R.C., Simpkin A.J., Mulvaney T., Pham P., Rangan R., Bu F., Keegan R.M., Topf M., Rigden D.J. et al. Assessment of three-dimensional RNA structure prediction in CASP15. Proteins. 2023; 91:1747–1770. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 17. Robin X., Studer G., Durairaj J., Eberhardt J., Schwede T., Walters W.P. Assessment of protein-ligand complexes in CASP15. Proteins. 2023; 91:1811–1821. [DOI] [PubMed] [Google Scholar]
- 18. Read R.J., Adams P.D., Arendall W.B. 3rd, Brunger A.T., Emsley P., Joosten R.P., Kleywegt G.J., Krissinel E.B., Lütteke T., Otwinowski Z. et al. A new generation of crystallographic validation tools for the protein data bank. Structure. 2011; 19:1395–1412. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 19. Montelione G.T., Nilges M., Bax A., Güntert P., Herrmann T., Richardson J.S., Schwieters C.D., Vranken W.F., Vuister G.W., Wishart D.S. et al. Recommendations of the wwPDB NMR Validation Task Force. Structure. 2013; 21:1563–1570. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 20. Henderson R., Sali A., Baker M.L., Carragher B., Devkota B., Downing K.H., Egelman E.H., Feng Z., Frank J., Grigorieff N. et al. Outcome of the first electron microscopy validation task force meeting. Structure. 2012; 20:205–214. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 21. Gore S., Sanz García E., Hendrickx P.M.S., Gutmanas A., Westbrook J.D., Yang H., Feng Z., Baskaran K., Berrisford J.M., Hudson B.P. et al. Validation of structures in the Protein Data Bank. Structure. 2017; 25:1916–1927. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 22. Williams C.J., Headd J.J., Moriarty N.W., Prisant M.G., Videau L.L., Deis L.N., Verma V., Keedy D.A., Hintze B.J., Chen V.B. et al. MolProbity: More and better reference data for improved all-atom structure validation. Protein Sci. 2018; 27:293–315. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 23. Ramachandran G.N., Ramakrishnan C., Sasisekharan V. Stereochemistry of polypeptide chain configurations. J. Mol. Biol. 1963; 7:95–99. [DOI] [PubMed] [Google Scholar]
- 24. Kabsch W., Sander C. Dictionary of protein secondary structure: pattern recognition of hydrogen-bonded and geometrical features. Biopolymers. 1983; 22:2577–2637. [DOI] [PubMed] [Google Scholar]
- 25. Studer G., Rempfer C., Waterhouse A.M., Gumienny R., Haas J., Schwede T. QMEANDisCo—distance constraints applied on model quality estimation. Bioinformatics. 2019; 36:1765–1771. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 26. Simpkin A.J., Mesdaghi S., Sánchez Rodríguez F., Elliott L., Murphy D.L., Kryshtafovych A., Keegan R.M., Rigden D.J. Tertiary structure assessment at CASP15. Proteins. 2023; 91:1616–1635. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 27. Robin X., Haas J., Gumienny R., Smolinski A., Tauriello G., Schwede T. Continuous Automated Model EvaluatiOn (CAMEO)-Perspectives on the future of fully automated evaluation of structure prediction methods. Proteins. 2021; 89:1977–1986. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 28. Westbrook J.D., Young J.Y., Shao C., Feng Z., Guranovic V., Lawson C.L., Vallat B., Adams P.D., Berrisford J.M., Bricogne G. et al. PDBx/mmCIF Ecosystem: foundational Semantic Tools for Structural Biology. J. Mol. Biol. 2022; 434:167599. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 29. Westbrook J.D., Shao C., Feng Z., Zhuravleva M., Velankar S., Young J. The chemical component dictionary: complete descriptions of constituent molecules in experimentally determined 3D macromolecules in the Protein Data Bank. Bioinformatics. 2015; 31:1274–1278. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 30. Waterhouse A., Bertoni M., Bienert S., Studer G., Tauriello G., Gumienny R., Heer F.T., de Beer T.A.P., Rempfer C., Bordoli L. et al. SWISS-MODEL: homology modelling of protein structures and complexes. Nucleic Acids Res. 2018; 46:W296–W303. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 31. Rose A.S., Bradley A.R., Valasatava Y., Duarte J.M., Prlic A., Rose P.W. NGL viewer: web-based molecular graphics for large complexes. Bioinformatics. 2018; 34:3755–3758. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 32. Wang G., Dunbrack R.L. Jr PISCES: a protein sequence culling server. Bioinformatics. 2003; 19:1589–1591. [DOI] [PubMed] [Google Scholar]
- 33. Laskowski R.A., MacArthur M.W., Moss D.S., Thornton J.M. PROCHECK: a program to check the stereochemical quality of protein structures. J. Appl. Crystallogr. 1993; 26:283–291. [Google Scholar]
- 34. Studer G., Biasini M., Schwede T. Assessing the local structural quality of transmembrane protein models using statistical potentials (QMEANBrane). Bioinformatics. 2014; 30:i505–i511. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 35. Lomize A.L., Pogozheva I.D., Lomize M.A., Mosberg H.I. Positioning of proteins in membranes: a computational approach. Protein Sci. 2006; 15:1318–1333. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 36. Elfmann C., Stülke J. PAE viewer: A webserver for the interactive visualization of the predicted aligned error for multimer structure predictions and crosslinks. Nucleic Acids Res. 2023; 51:W404–W410. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 37. Bertoni M., Kiefer F., Biasini M., Bordoli L., Schwede T. Modeling protein quaternary structure of homo- and hetero-oligomers beyond binary interactions by homology. Sci. Rep. 2017; 7:10480. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 38. Mariani V., Biasini M., Barbato A., Schwede T. lDDT: A local superposition-free score for comparing protein structures and models using distance difference tests. Bioinformatics. 2013; 29:2722–2728. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 39. Bernard C., Ghannay S., Tahi F. RNAdvisor: A comprehensive benchmarking tool for the measure and prediction of RNA structural model quality. 2023; bioRxiv doi:16 November 2023, preprint: not peer reviewed 10.1101/2023.11.14.567018. [DOI] [PMC free article] [PubMed]
- 40. Zhang Y., Skolnick J. Scoring function for automated assessment of protein structure template quality. Proteins. 2004; 57:702–710. [DOI] [PubMed] [Google Scholar]
- 41. Zhang C., Shine M., Pyle A.M., Zhang Y. US-align: universal structure alignments of proteins, nucleic acids, and macromolecular complexes. Nat. Methods. 2022; 19:1109–1115. [DOI] [PubMed] [Google Scholar]
- 42. Kabsch W. A solution for the best rotation to relate two sets of vectors. Acta Crystallogr. A. 1976; 32:922–923. [Google Scholar]
- 43. Basu S., Wallner B. DockQ: a quality measure for protein-protein docking models. PLoS One. 2016; 11:e0161879. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 44. Lensink M.F., Wodak S.J. Docking, scoring, and affinity prediction in CAPRI. Proteins. 2013; 81:2082–2095. [DOI] [PubMed] [Google Scholar]
- 45. Biasini M., Schmidt T., Bienert S., Mariani V., Studer G., Haas J., Johner N., Schenk A.D., Philippsen A., Schwede T. OpenStructure: an integrated software framework for computational structural biology. Acta Crystallogr. D Biol. Crystallogr. 2013; 69:701–709. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 46. Schöning-Stierand K., Diedrich K., Ehrt C., Flachsenberg F., Graef J., Sieg J., Penner P., Poppinga M., Ungethüm A., Rarey M. ProteinsPlus: a comprehensive collection of web-based molecular modeling tools. Nucleic Acids Res. 2022; 50:W611–W615. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 47. Laskowski R.A., Thornton J.M. PDBsum extras: SARS-CoV-2 and AlphaFold models. Protein Sci. 2022; 31:283–289. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 48. Emsley P., Lohkamp B., Scott W.G., Cowtan K. Features and development of Coot. Acta Crystallogr. D Biol. Crystallogr. 2010; 66:486–501. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 49. Meng E.C., Goddard T.D., Pettersen E.F., Couch G.S., Pearson Z.J., Morris J.H., Ferrin T.E. UCSF ChimeraX: Tools for structure building and analysis. Protein Sci. 2023; 32:e4792. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 50. Liebschner D., Afonine P.V., Baker M.L., Bunkóczi G., Chen V.B., Croll T.I., Hintze B., Hung L.W., Jain S., McCoy A.J. et al. Macromolecular structure determination using X-rays, neutrons and electrons: recent developments in Phenix. Acta Crystallogr D Struct Biol. 2019; 75:861–877. [DOI] [PMC free article] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.
Supplementary Materials
Data Availability Statement
The Structure Assessment web server is available at https://swissmodel.expasy.org/assess. This website is free and open to all users and there is no login requirement.