PhiSiCal-Checkup: A Bayesian framework to validate amino acid conformations within experimental protein structures

Significance

Structural biology, biochemistry, evolutionary biology, medicine, and drug development all require high-quality protein structures. Reliable tools for assessing structures remain essential to ensure and maintain quality. We present a Bayesian method for analysis and validation of amino-acid conformations within protein structures: “PhiSiCal ($\varphi \psi \chi$al) Checkup.” This method overcomes major, long-standing shortcomings and provides significant improvements over the current state of the art. By introducing more-reliable information-theoretic measures of “favorability” of amino acid conformations, PhiSiCal-Checkup provides ways to analyze protein structures that were previously out of reach for experimentalists and structure biologists.

Abstract

As structural biology and drug discovery depend on high-quality protein structures, assessment tools are essential. We describe a new method for validating amino-acid conformations: “PhiSiCal ($\varphi \psi \chi$al) Checkup.” Twenty new joint probability distributions in the form of statistical mixture models explain the empirical distributions of dihedral angles $〈\omega ,\varphi ,\psi ,{\chi }_1,{\chi }_2,\dots 〉$ of canonical amino acids in experimental protein structures. Marginal and conditional probability distributions for subsets of dihedral angles are derived from these joint mixture models. Together, these distributions are employed to measure rapidly the information-theoretic “favorability” of any proposed experimental protein structure. The inferred statistical models and measures overcome several shortcomings and afford improvements over the current state of the art in amino-acid conformation verification. Experimental comparisons are made against current protein conformation verification software. In a number of examples, we pick up outliers that are invisible to current methods. We also calculate, as part of verification, the sensitivity of favorability to small changes in a proposed structure accounting for the precision of coordinates. In some cases a near neighbor of a proposed amino-acid conformation may be either less or more favorable. This raises the question, is the current reliance on fixed “thresholds” for validation a good thing? PhiSiCal-Checkup is freely available for online and offline (open-source) use from https://lcb.infotech.monash.edu.au/phisical/checkup.

Experimental methods for protein structure determination produce raw measurements from which atomic coordinates are modeled as solutions to protein three-dimensional structures. The Worldwide Protein Data Bank (wwPDB) (1) administers the public archive (PDB) of protein coordinates (2) and scrutinizes them across a number of validation criteria, based on the guidelines of its validation task force (VTF) (3).

Key among the validation criteria is the knowledge-based assessment of amino-acid conformations. The current practice is to quantify the “favorability” of each amino acid’s main chain conformation (described by $〈\varphi ,\psi 〉$ dihedral angles) and, separately, side chain conformation (described by $〈{\chi }_1,{\chi }_2,\dots 〉$ dihedral angles) using their observed distributions within carefully curated datasets of protein structures (4). This quantification allows defining and detecting “outliers” from the distributions of conformational features. (Note that although outliers deserve further examination, they are not necessarily errors. Indeed, outliers that are correct often point to features of structural or functional interest. Conversely, non-outliers are not necessarily correct.) Popular programs to validate $〈\varphi ,\psi 〉$ and $〈{\chi }_1,{\chi }_2,\dots 〉$ dihedral angles developed over the past three decades include PROCHECK (5), WHAT_CHECK (6), O (7), and MolProbity (8).

MolProbity represents the current state of the art, recommended by VTF (9). However, it has several limitations, some of which are acknowledged by its authors:

• “What we do expect will improve in the future is a redefinition of conformational validation...Rather than doing separate Ramachandran and rotamer evaluation, we should move toward analyzing all backbone and side-chain torsional dimensions together, including allowance for the influence of secondary structure and local motifs” (10).

(Others are discussed in Results, summarized in SI Appendix, Table ST1).

To overcome these limitations, we introduce “PhiSiCal ($\varphi \psi \chi$al) Checkup,” a comprehensive Bayesian and information-theoretic framework for validation of amino-acid conformations. Supporting PhiSiCal-Checkup are twenty new joint probability distributions, one for each of the canonical amino acids; each distribution treats all mainchain and sidechain dihedral angles together. These are inferred as statistical mixture models using our recently published inference methodology, PhiSiCal (11), based on the unsupervised model selection criterion of Minimum Message Length (12, 13). [Although PhiSiCal provides the core methodology for inference of joint mixture models, specializing these models for the conformation-validation implemented by PhiSiCal-Checkup required several new methodological advances previously unexplored (Materials and Methods)].

In PhiSiCal-Checkup, each joint mixture model can be factorized into mutually consistent probability distributions to analyze any possible combination of dihedral angle terms. Specifically, for any amino-acid type containing d dihedral angle terms ($\underset{d\text{dihedral angle terms}}{\underbrace{\left(〈\omega ,\varphi ,\psi ,{\chi }_1,{\chi }_2,\dots 〉\right)}}$), the corresponding joint mixture model (defining a distribution in a d-dimensional toroidal space, ${\mathbb{T}}^d$) can be transformed “on-the-fly” using Bayesian axioms of probability to derive $2^d-2$ marginal and $3^d-2^{d+1}+1$ conditional probability distributions. The formal relationship between joint, marginal, and conditional distributions is governed by the Bayes theorem (14) making all amino acid–specific distributions mutually consistent.

Marginal probability distributions allow the unconstrained evaluation of any combination of the d dihedral angle terms, of which the assessment of main chain $〈\varphi ,\psi 〉$ and side chain $〈{\chi }_1,{\chi }_2,\dots 〉$ conformations are but two possibilities. Conditional distributions allow the evaluation of any subset of dihedral angles subject to observing specific values for others, providing meticulous new ways of analyzing amino-acid conformations. For example, the conditional probability distribution of the $〈{\chi }_1,{\chi }_2,\dots 〉$ terms given some specific observed values for $〈\varphi ,\psi 〉$ (shorthand, $〈{\chi }_1,{\chi }_2,\dots 〉|〈\varphi ,\psi 〉$) can be used to evaluate the favorability of the observed $〈{\chi }_1,{\chi }_2,\dots 〉$ dihedral angles conditioned on the $〈\varphi ,\psi 〉$ observation. When the (continuous) $〈\varphi ,\psi 〉$ values correspond to, say, a (discrete) secondary structural state, the $〈{\chi }_1,{\chi }_2,\dots 〉|〈\varphi ,\psi 〉$ probability distribution will automatically embody the secondary-structural constraint on the $〈{\chi }_1,{\chi }_2,\dots 〉$ angles. In extension, the conditional distribution $〈{\chi }_1,{\chi }_2,\dots 〉|〈\omega ,\varphi ,\psi 〉$ constrains the evaluation of $〈{\chi }_1,{\chi }_2,\dots 〉$ additionally on the cis/trans/twisted peptide state informed by the continuous value of ω, beyond the state informed by $〈\varphi ,\psi 〉$. As another example, the conditional distribution of $〈\varphi ,\psi 〉|\omega$ allows evaluating $〈\varphi ,\psi 〉$ given any observed continuous value for ω, thereby automatically accounting for ω being in cis/trans/twisted states. Data in ref. 15 shows the animations of amino acid–specific $〈\varphi ,\psi 〉|\omega$ distributions for varying ω)

This work also introduces new statistical measures to validate amino-acid conformations. PhiSiCal-Checkup’s validation of dihedral angles (in any combination of terms) is driven by the information-theoretic measure of lossless compression, quantified in bits. Favorability (or unfavorability) of any conformation can objectively be assessed based on the amount of compression gained (or lost) with respect to the raw bit-length of encoding the observed dihedral angles. Statistically, this measure of compression is equivalent to quantifying the log-likelihood-odds of any observation arising from its corresponding (amino acid–specific joint/marginal/conditional) mixture model compared to the raw (uniform) distribution. Further, since all statistical mixture models are continuous and differentiable at every point in the respective distributions’ support, a gradient vector in the probability distribution can be computed for any observed dihedral angles. Using this gradient, sensitivity of the reported compression statistic to perturbation of dihedral angles can be analyzed accounting for the (im)precision of statement of atomic coordinates. This overcomes another limitation in the current state of art: the reliance on hard (non-overlapping) membership assignments of observations to one of three categories—“outlier,” “allowed,” and “favored”—that can yield misleading assignments, especially near the boundaries of those categories.

A configurable web server and standalone software implementing PhiSiCal-Checkup is available for immediate online and offline use: https://lcb.infotech.monash.edu.au/phisical/checkup. Validation of dihedral terms is fully customizable and supported by instructive visualizations.

Results and Discussion

Statistical Mixture Models for Validation of Dihedral Angles.

Twenty new joint statistical mixture models are inferred in this work, one for each of the canonical amino acids, using the reference protein structural data set of Williams et al. (1), curated specifically for the task of structure validation (Materials and Methods and SI Appendix, Table ST2). Each joint mixture model describes a continuous amino acid–specific probability distribution of its corresponding dihedral angle terms, $〈\omega ,\varphi ,\psi ,{\chi }_1,{\chi }_2,\dots 〉$. Amino acid–specific marginal and conditional probability distributions are derived (on demand) from these inferred joint mixture models. Importantly, these distributions are also statistical mixture models, characterized in a reduced number of dimensions defined by the subset they explain (Materials and Methods).

The amino acid–specific joint mixture model, combined with the derived marginal and conditional mixture models provide the “basis set” of probability distributions for PhiSiCal-Checkup to interrogate dihedral angle conformations in any possible combination of terms. The computation of joint/marginal/conditional probabilities and other derived-statistics remains extremely efficient taking between 10 to 60 microseconds-per-computation on a standard (single-thread) computer.

Compressibility of Observed Dihedral Angles.

PhiSiCal-Checkup uses the measures of Shannon information content (16) and compression to quantify the surprise of observing any amino-acid conformation. These measures are derived from the corresponding joint/marginal/conditional mixture model, depending on the combination of dihedral angle terms being observed. Compression (measured in bits) is statistically equivalent to computing the log-odds of an observation arising from its corresponding mixture model, compared to the uniform distribution. Compression (gain) of $+n$ bits yields $2^n$:1 odds in favor of the mixture model, whereas compression (loss) of $-n$ bits yields 1:$2^n$ odds against it. Importantly, PhiSiCal-Checkup also assesses the sensitivity of the computed compression by locally perturbing the observation in the direction of its gradient vector in the probability distribution (Materials and Methods).

To understand compression, its sensitivity, and statistical odds, consider the case study of Chain A Arginine 368 (Arg-368) in the PDB coordinates of arabinoxylan arabinofuranohydrolase (AXAH) from Bacillus subtilis (3C7F). The conformation of Arg-368 in 3C7F (Chain A) yields the following dihedral angle observations: $\omega ={-19.15}^{°}$, $\varphi ={-130.68}^{°}$, $\psi ={120.11}^{°}$, ${\chi }_1={-67.61}^{°}$, ${\chi }_2={178.90}^{°}$, ${\chi }_3={-174.16}^{°}$, ${\chi }_4={-171.50}^{°}$, ${\chi }_5={-179.13}^{°}$. Particularly, $\omega ={-19.15}^{°}$ being discussed here is the rotation around the peptide bond connecting the preceding Glycine (Gly-367) and the current Arg-368. This suggests a strained, energetically unfavorable cis-conformation for Arg-368. However, this is not an error: checking the 1.55 Å crystal structure coordinates of 3C7F, the carbonyl preceding Arg-368 is seen facilitating the binding of the Sodium (${\text{Na}}^{+}$) metal ion (17).

Table 1 shows the compression and sensitivity statistics of observing Arg-368 in 16 combinations of dihedral angle terms (of the total 6,305 possible for Arginine). The individual observation of $\omega ={-19.15}^{°}$ (ignoring all other dihedral angles) results in a loss of compression (−11.53 bits) using its corresponding Arginine-specific marginal mixture model. This gives 1:$2^{11.53}\approx 1$:3,000 odds against that mixture model. The compression’s sensitivity is derived by perturbing $\omega ={-19.15}^{°}\pm {2.5}^{°}$ in the direction of its gradient, causing compression to vary between $[-12.2,-10.9]$ bits (and odds between ≈[1:2,000, 1:4,700]) in the local neighborhood of that observation. Similarly, the observed values for $〈\omega ,\varphi ,\psi 〉$ explained using its marginal mixture model result in a loss of compression of −9.22 bits (odds of $\approx 1:600$) with a sensitivity of $\pm 0.7$ bits. The evaluation of all $〈\omega ,\varphi ,\psi ,{\chi }_1,{\chi }_2,\dots 〉$ terms using the joint mixture model results in a loss of compression of −6.77 bits ($\approx 1:100$ odds). In contrast, observing only the Arg-368’s side chain ($〈{\chi }_1,{\chi }_2,\dots 〉$) terms leads to a gain in compression of $+11.24$ bits (≈2,400:1 odds). All other dihedral angles (in combinations not involving the strained ω) lead to gain in compression. Finally, the Arg-368’s side chain conformation conditioned on the pair of Ramachandran angles taking the values $〈\varphi ,\psi 〉=〈{-130.68}^{°},{120.11}^{°}〉$ (but ignoring the strained cis-$\omega ={-19.15}^{°}$) leads to a gain in compression of $+11.43$ bits. However, when $\omega ={-19.15}^{°}$ is also considered, the compression-gain drops sharply to $2.5$ bits. This is equivalent to the statistical odds dropping from ≈2,800:1 to ≈6:1, thus quantifying the extent of surprise of observing that ω as part of Arg-368’s overall conformation.

Table 1.

Compression and sensitivity of dihedral angles (in varying combinations) of Arg-368 in the protein coordinates of 3C7F Chain A

Observation (mixture model)	Compression (±sensitivity)	Observation (Mixture model)	Compression (±sensitivity)
${\omega }_{\text{(marginal)}}$	−11.5 $\left(\begin{array}{c}\begin{array}{l}+0.6\\ -0.7\end{array}\end{array}\right)$ bits	${〈\omega ,\varphi ,\psi 〉}_{\text{(marginal)}}$	−9.2 $\left(\begin{array}{c}\begin{array}{l}+0.7\\ -0.7\end{array}\end{array}\right)$ bits
${\varphi }_{\text{(marginal)}}$	1.1 $\left(\begin{array}{c}\begin{array}{l}+0.0\\ -0.0\end{array}\end{array}\right)$ bits	${〈\varphi ,\psi 〉}_{\text{(marginal)}}$	2.5 $\left(\begin{array}{c}\begin{array}{l}+0.3\\ -0.3\end{array}\end{array}\right)$ bits
${\psi }_{\text{(marginal)}}$	0.7 $\left(\begin{array}{c}\begin{array}{l}+0.3\\ -0.3\end{array}\end{array}\right)$ bits	${〈{\chi }_1,{\chi }_2,{\chi }_3,{\chi }_4,{\chi }_5〉}_{\text{(marginal)}}$	11.2 $\left(\begin{array}{c}\begin{array}{l}+0.0\\ -1.4\end{array}\end{array}\right)$ bits
${\chi }_{1_{\text{(marginal)}}}$	3.4 $\left(\begin{array}{c}\begin{array}{l}+0.0\\ -0.1\end{array}\end{array}\right)$ bits	${〈\varphi ,\psi 〉|\omega }_{\text{(conditional)}}$	2.3 $\left(\begin{array}{c}\begin{array}{l}+0.1\\ -0.1\end{array}\end{array}\right)$ bits
${\chi }_{2_{\text{(marginal)}}}$	3.2 $\left(\begin{array}{c}\begin{array}{l}+0.0\\ -0.0\end{array}\end{array}\right)$ bits	${{\chi }_1|〈\varphi ,\psi 〉}_{\text{(conditional)}}$	3.0 $\left(\begin{array}{c}\begin{array}{l}+0.3\\ -0.5\end{array}\end{array}\right)$ bits
${\chi }_{3_{\text{(marginal)}}}$	2.6 $\left(\begin{array}{c}\begin{array}{l}+0.2\\ -0.3\end{array}\end{array}\right)$ bits	${〈{\chi }_1,{\chi }_2,{\chi }_3,{\chi }_4,{\chi }_5〉|〈\varphi ,\psi 〉}_{\text{(conditional)}}$	11.4 $\left(\begin{array}{c}\begin{array}{l}+0.0\\ -1.4\end{array}\end{array}\right)$ bits
${\chi }_{4_{\text{(marginal)}}}$	1.5 $\left(\begin{array}{c}\begin{array}{l}+0.1\\ -0.1\end{array}\end{array}\right)$ bits	${〈{\chi }_1,{\chi }_2,{\chi }_3,{\chi }_4,{\chi }_5〉|〈\omega ,\varphi ,\psi 〉}_{\text{(conditional)}}$	2.5 $\left(\begin{array}{c}\begin{array}{l}+0.0\\ -0.0\end{array}\end{array}\right)$ bits
${\chi }_{5_{\text{(marginal)}}}$	2.3 $\left(\begin{array}{c}\begin{array}{l}+0.0\\ -1.5\end{array}\end{array}\right)$ bits	${〈\omega ,\varphi ,\psi ,{\chi }_1,\dots ,{\chi }_5〉}_{\text{(joint)}}$	−6.8 $\left(\begin{array}{c}\begin{array}{l}+0.7\\ -0.7\end{array}\end{array}\right)$ bits

We note that, in the current state of the art (MolProbity), the surprise of the ω value at the peptide bond between Gly-367 and Arg-368 (that gives Arg-368 a cis conformation) is overlooked as it can only validate the $〈\varphi ,\psi 〉$ and $〈{\chi }_1,{\chi }_2,\dots 〉$ terms (independently). In sum, such detailed analysis of observed amino-acid conformations, at varying granularity and constraints, is unique to PhiSiCal-Checkup.

Quantifying Surprise of Observed Dihedral Angles.

From the relationship between statistical odds and compression, it follows that compression at zero bits yields the surprisal-odds of 1:1 (fifty–fifty), thus demarcating an objective boundary for the joint/marginal/conditional distributions: observations below zero become exponentially surprising and those above exponentially favorable.

Here, we explore how compression correlates with the empirical frequencies of dihedral angles. To achieve this we analyzed the percentile-rank distribution of compression across the 1,720,588 amino-acid conformations found in the reference dataset (4).

Fig. 1A tracks the bits of compression at varying percentile levels ($\{min,0.05,0.3,2,5,10,25,50,75,max\}$) for $〈\varphi ,\psi 〉$ and $〈{\chi }_1,{\chi }_2,\dots 〉$ dihedral angle observations of each amino-acid type. (SI Appendix, Table ST3 provides a tabular view of the figure in raw numbers. The last row of this table summarizes the mean and SD of the compression values at each of those percentile levels.) Broadly, the compression values at each percentile mark for $〈\varphi ,\psi 〉$ show relatively lower dispersion about the mean compression values, compared to those of $〈{\chi }_1,{\chi }_2,\dots 〉$. This is expected because the differences in the amino-acid types arise due to their side chain groups that show varying conformational-mobility and energetics (18). In the lower-half of the distribution below the median, the compression statistics of Proline $〈\varphi ,\psi 〉$ and $〈{\chi }_1,{\chi }_2,\dots 〉$ observations diverge the most, followed by Glycine for $〈\varphi ,\psi 〉$ and Arginine for $〈{\chi }_1,{\chi }_2,\dots 〉$. This likely arises due to Proline’s cyclic-pyrrolidine side chain, Glycine’s absence of β-carbon, and Arginine’s side chain length.

Fig. 1.

(A) Bits of compression at varying percentile levels for amino acid–specific Ramachandran $〈\varphi ,\psi 〉$ (Left) and side chain $〈{\chi }_1,{\chi }_2,\dots 〉$ (Right) dihedral angle terms, using 1,720,588 amino-acid conformations observed in the reference dataset (4). Insets show a zoomed-in view of the left-tail of the distributions. (B) Amino acid–specific percentile ranks with <0 bits of compression are shown. Note: Amino Acids Alanine (ALA) and Glycine (GLY) do not have side chain dihedral angle terms.

Focusing on the distributions’ (infrequent) tails, Fig. 1B shows the proportions of $〈\varphi ,\psi 〉$ and $〈{\chi }_1,{\chi }_2,\dots 〉$ observations that lose compression (<0 bits). For $〈\varphi ,\psi 〉$, this accounts for ∼5$\pm 1\%$ of observations for most amino acids, except for Glycine with $9.1\%$, Valine and Isoleucine with $2.8\%$ and Proline with $1.4\%$ that deviate from this trend. For $〈{\chi }_1,{\chi }_2,\dots 〉$, the proportions are more spread out, ranging from 0.1% for Proline to 8.2% for Cysteine. These results quantitatively highlight the differences in the distributions of dihedral angles across the 20 amino-acid types.

These differences were further explored by qualitatively analyzing the compression statistics for each amino acid using all-pairs compression contour maps derived from their corresponding (marginal) mixture models. Specifically, for each amino-acid type with d dihedral terms $〈\omega ,\varphi ,\psi ,{\chi }_1,{\chi }_2,\dots 〉$, $d\text{-choose-}2$ contour plots were generated for each possible pair of dihedral angle terms (19). Each 2D plot shows 1) the empirical distribution for that pair, 2) compression contour lines at 1-bit intervals between $[-5,+5]$ bits, and 3) gradient vectors showing the rate of change of probability (and hence compression) at $5^{°}\times 5^{°}$ intervals. Visual examination of these plots again highlights the significant differences in the amino acid–specific distributions of dihedral angles.

These plots also illustrate the close-fit between the mixture models and the underlying empirical distributions. As an example, Fig. 2 shows two such plots for Proline. Specifically, Fig. 2A displays the contour plot for Proline $〈\varphi ,\psi 〉$ terms. The contours correlate closely with the empirical frequencies of observed $〈\varphi ,\psi 〉$ angles (encoded in light-to-dark shades of blue). The +5-bit compression lines (innermost black contours) encompass two regions of high-probability (20) that peak at $〈\varphi ,\psi 〉\approx 〈{-60}^{°},{+145}^{°}〉$ and $\approx 〈{-60}^{°},{-30}^{°}〉$, where the norm of the gradient vectors in those regions approach 0. At −5 bits of compression (outermost, red line), the contour encompasses infrequent/low-probability regions in the valleys of Proline’s $〈\varphi ,\psi 〉$ distribution. Importantly, points/observations with the same compression values can have significantly different gradients (and hence sensitivities to local perturbations), as illustrated by points labeled P₁ and P₂ in the figure. Locally perturbing P₁ in the direction of its gradient improves compression (and statistical odds) significantly more than it does at P₂.

Fig. 2.

Proline’s empirical observations (shown as scatter points colored in light-to-dark shades of blue colored based on their $1^{°}\times 1^{°}$ grid-frequencies), compression contours (multicolored lines) in 1-bit intervals between $[-5,+5]$, and gradient vectors (black arrows) at $5^{°}\times 5^{°}$ interval are shown above for (A) $〈\varphi ,\psi 〉$ and (B) $〈{\chi }_2,{\chi }_3〉$ dihedral angle terms. The $〈\varphi ,\psi 〉$ plot also shows two points P₁ and P₂ on the −5 bit (outmost, red) compression line. The Insets around these two points highlight the differences in gradients (and hence sensitivities under local-perturbation). For $〈{\chi }_2,{\chi }_3〉$ plot, the axes are truncated to $[{-60}^{°},{+60}^{°}]$, as there are no empirical observations beyond that in the reference dataset. Separately, a set of anomalous observations are highlighted (red points)—refer to the main text for discussion.

Similarly, Fig. 2B shows the compression contours and gradient vectors for Proline’s side chain dihedral angle pair $〈{\chi }_2,{\chi }_3〉$. The contour lines again correlate closely with the empirical frequencies of that pair within the observed Proline conformations. This plot additionally overlays a set of 29 anomalous Proline observations, all from the same protein structure (3H8G), 1.5 Å Bestatin complex structure of leucine aminopeptidase from Pseudomonas putida (21). Surprisingly, in all 29 observations, χ₂ is nearly $0^{°}$ (SI Appendix, Table ST4). Examining the PDB validation report of 3H8G, none of these Prolines were earmarked as outliers by MolProbity. (Note, MolProbity ignores χ₂ and χ₃ dihedral angles in their evaluation for Proline and considers only χ₁.) This illustrates how the contour-plots enabled by PhiSiCal-Checkup can be used to identify and examine surprising conformations that deviate from the observed distributions.

Altogether, the above analyses reveal quantitative and qualitative differences in the empirical distributions of dihedral angles across amino acids. These differences highlight the importance of using amino acid–specific distributions to validate dihedral angle conformations. Further, the analyses also reveal the variation of compression statistic at any fixed percentile threshold across amino-acid types. This throws into question the prevalent use of fixed percentile-thresholds to flag $〈\varphi ,\psi 〉$ and $〈{\chi }_1,{\chi }_2,\dots 〉$ outliers in the current validation protocols (see discussion below).

Comparison with MolProbity.

We compare PhiSiCal-Checkup with MolProbity. SI Appendix, Table ST1 summarizes the key differences between the two systems. Among the differences is the statistical test they rely on to validate amino-acid conformations. Central to MolProbity’s method is the use of fixed percentile rank thresholds that do not change with amino-acid type. Specifically, MolProbity employs a 3-way classification of $〈\varphi ,\psi 〉$ and $〈{\chi }_1,{\chi }_2,\dots 〉$ observations. Each observation is assigned a hard-membership to one of {outlier, allowed, favored} categories based on the percentile rank of the observation’s score (derived from their normalized functions) within a set of scores precomputed for their reference data. MolProbity sets the outlier thresholds of $0.05\%$ (significance-level $0.0005$) for $〈\varphi ,\psi 〉$ and $0.3\%$ (significance-level $0.003$) for $〈{\chi }_1,{\chi }_2,\dots 〉$ observations. For the allowed category, the percentile-threshold is set at $2\%$ (significance-level 0.02) for both. Consequently, observations that fall above the $2$nd percentile as per MolProbity’s scoring earmarks the favored category.

In statistical parlance, MolProbity is relying on a Z-test for a hard 3-way clustering of $〈\varphi ,\psi 〉$ and $〈{\chi }_1,{\chi }_2,\dots 〉$ observations. However, a Z-test is effective only if the test-statistic is normally distributed: only then can the significance-levels of $0.0005$, $0.003$, $0.02$ correspond to Z-scores thresholds of $\sim \pm$3.5σ, $\sim \pm$3σ, and $\sim \pm$2.3σ from the respective mean values of scores in a two-tailed test. Importantly, our analysis finds no evidence of normality of the underlying probabilities (which the MolProbity’s function-scores are approximating). SI Appendix, Figs. SF1 and SF2 show the distribution of probabilities for the $〈\varphi ,\psi 〉$ and $〈{\chi }_1,{\chi }_2,\dots 〉$ conformations observed in the reference dataset, derived using PhiSiCal-Checkup’s mixture models. This is quantitatively supported by the observed variance in the compression statistic across amino-acid types at $0.05$, $0.3$, and $2$ percentile levels (SI Appendix, Table ST3). Therefore, PhiSiCal-Checkup avoids using percentile-rank thresholds in its method of validation.

Instead, PhiSiCal-Checkup relies on the information-theoretic measure of compression to quantify surprisal-odds of any observation. Compression at 0 bits defines an objective threshold below which observations grow exponentially surprising, and above which, exponentially favorable. Thus, PhiSiCal-Checkup uses compression of $\ge 0$ bits to earmark favored observations. For comparison with MolProbity, another threshold to flag outlier conformations requires to be defined, while noting that such a choice remains fully subjective and cannot be formally defended. A subjective outlier threshold at −4 bits of compression (or 1:16 odds) is thus chosen here as a default setting in PhiSiCal-Checkup. (Note, fractional odds (e.g., 1:16) should not be confused with frequentist percentile-based probabilities. In MolProbity, a score ranked at the $0.05$th percentile gives a 1 in 2,000 chance of observing that score in their precomputed set of scores. At the same percentile rank (refer SI Appendix, Table ST3), PhiSiCal-Checkup, on average, yields −6.4 bits of compression for $〈\varphi ,\psi 〉$ observations, or ∼1:85 surprisal-odds.)

This results in a 3-way categorization for PhiSiCal-Checkup based on compression: outlier $<-4$ bits, allowed $[-4,0)$ bits and favored $\ge 0$ bits. A key difference compared with MolProbity’s categorization is that PhiSiCal-Checkup permits overlapping membership-assignment based on the observation’s gradient information (discussed below). The gradient accounts for the uncertainty (imprecision) of stated atomic coordinates and alerts users of observations that are close to the boundaries of PhiSiCal-Checkup’s 3-way classification, and whose membership status is sensitive to minor perturbations of observed dihedral angles.

Table 2 summarizes the agreement/disagreement between the two systems. This is based on 3,624,568 $〈\varphi ,\psi 〉$ (Table 2A) and 3,027,146 $〈{\chi }_1,{\chi }_2,\dots 〉$ (Table 2B) observations derived from 9,419 filtered-PDB structures (23). The corresponding details at the level of individual amino-acid types are available from refs. 24 and 25.

Table 2.

(A and B) $3\times 3$ tables (confusion matrices) displaying the extent of agreement (main diagonal cells) and disagreement (off-diagonal cells) between PhiSiCal-Checkup (compression-based) and MolProbity (percentile-rank based) systems, performing 3-way ({outlier, allowed, and favored}) classification of 3,624,568 $〈\varphi ,\psi 〉$ observations and 3,027,146 $〈{\chi }_1,{\chi }_2,\dots 〉$ observations

Rows and columns represent MolProbity’s and PhiSiCal-Checkup’s respective 3-way assignments. The agreement/disagreement percentages with respect to MolProbity’s membership assignments are shown in parentheses in each cell—row percentages add up to 100. (C and D) Difference matrices (corresponding to the confusion matrices shown above) quantifying the number of observations that change their membership in PhiSiCal-Checkup upon a minor perturbation of each observation in the direction of the observation’s gradient in their probability distributions. Negative numbers indicate the reduction and positive indicate accretion in the corresponding column category for each row (sum of these differences in each row has to add up to 0).

Broadly, ∼$97\%$ of all $〈\varphi ,\psi 〉$ observations and ∼$95\%$ of all $〈{\chi }_1,{\chi }_2,\dots 〉$ observations are in agreement between the two systems. However, this is dominated by the overrepresentation of observations assigned to the favored category by both systems. This arises because 1) the measure-space of this category ($\ge 2\%$ for MolProbity and $\ge 0$ bits for PhiSiCal-Checkup) is disproportionately larger than that of the other two, and 2) the observations come from verified PDB structures which already embody this representational imbalance, with favored conformation forming the very basis for admission into PDB. With this drastic imbalance, any two systems are likely to display overwhelming agreement. We emphasize here that flagging outliers is a business fully in the tails of the distributions, so it becomes necessary to examine more carefully similarities and differences in the ∼$3\%$/∼$5\%$ tails of the $〈\varphi ,\psi 〉$/$〈{\chi }_1,{\chi }_2,\dots 〉$ observations.

Tables 2 A and B show the raw counts of agreement/disagreement between the two systems in the $3\times 3=9$ possible combinations of assignments between the two systems: MolProbity (rows) and PhiSiCal-Checkup (columns). Each cell also shows the agreement/disagreement-percentage with respect to MolProbity’s classification, with the sum of percentages in each row adding up to 100%. Analyzing the differences (off-diagonal cells), we observe that nearly all observations ($99.7\%$ for $〈\varphi ,\psi 〉$ and $98.8\%$ for $〈{\chi }_1,{\chi }_2,\dots 〉$) arise in cells that are $\pm 1$ distance from the main diagonal. This highlights the disagreement arising from observations being assigned to adjacent categories by the two systems: outlier⟷ allowed or allowed⟷favored.

Examining the compression statistics for observations that fall in ±1 off-diagonal cells, we observe that a significant proportion of them are close to the compression-based membership-boundaries defined by PhiSiCal-Checkup. This can be qualitatively visualized in the amino acid–specific contour plots for $〈\varphi ,\psi 〉$ shown in ref. 26. More quantitatively, Table 2 C and D demonstrate the effect of perturbation (in the direction of the gradient) on memberships (Materials and Methods). Each cell tracks the difference in the number of observations after perturbation compared to the raw counts shown in Tables 2 A and B. For $〈\varphi ,\psi 〉$ observations, ∼$38\%$ of the observations previously assigned to the outlier category by PhiSiCal-Checkup (first column) change membership and move into the allowed category (second column). Next, ∼40% of those previously classified as allowed by PhiSiCal-Checkup (second column), change membership and move into the favored category (third column). A similar trend is observed for $〈{\chi }_1,{\chi }_2,\dots 〉$ observations, with ∼$42\%$ moving from outlier to allowed, and ∼44% moving from allowed to favored. This demonstrates the inherent limitation of using hard (non-overlapping) memberships, especially considering the uncertainty implicit in any statement of coordinates. To overcome this, PhiSiCal-Checkup alerts users of conformations with overlapping memberships using its gradient information.

Further, examining the sources of differences at the level of individual amino-acid types, other limitations in the state of the art come to the fore, enumerated below (also refer to refs. 24–26):

1. For $〈\varphi ,\psi 〉$ observations, MolProbity employs the same “general” model for 16 (non-{Glycine, Valine, Isoleucine, Proline}) amino-acid types. This ignores noticeable variations in the empirical distributions of $〈\varphi ,\psi 〉$ for these 16 amino acids, clearly observable from their contour plots (27). For this reason, across all evaluations, PhiSiCal-Checkup employs amino acid–specific probability distributions which model the empirical variations more accurately.

2. Specifically for Proline $〈\varphi ,\psi 〉$ observations, MolProbity infers two additional normalized-functions after grouping the Proline data in the reference dataset into two coarse bins, based on their observed ω value: $\omega \in [{-30}^{°},{+30}^{°}]$ and $\omega \in ([{-180}^{°},{-150}^{°})\vee ({+150}^{°},{+180}^{°}))$ (9). In contrast, PhiSiCal-Checkup employs formal and continuous $〈\varphi ,\psi 〉|\omega$ conditional probability distributions that are more accurate than the discretized (cis-only and trans-only) models of MolProbity.

3. For $〈{\chi }_1,{\chi }_2,\dots 〉$ observations, no clear pattern emerges, with the differences spreading across all amino-acid types. As a proportion of each amino acid’s number of observations, the differences between PhiSiCal-Checkup and MolProbity vary from ∼2 to 3% on the lower side (Tyrosine, Phenylalanine, Leucine, Isoleucine, and Proline) to ∼7 to 8% on the higher (Glutamic acid, Histidine, Asparagine, Lysine, Serine, Cysteine, Methionine). The remaining amino acids (Aspartic acid, Glutamine, Tryptophan, Valine, Threonine, Arginine) differ between $[4,6]\%$. We observe that the PhiSiCal-Checkup’s mixture models fit the empirical distributions accurately (19). On the other hand, as the number of dihedral angles in the side chain increases, MolProbity uses increasingly coarse grid sizes and variable smoothing parameters to fit side chain-specific functions, contributing to the observed differences.

Concluding Remarks and Future Direction.

The results presented above demonstrate the advances PhiSiCal-Checkup achieves to enable a comprehensive, consistent, and accurate evaluation of amino acid conformations of experimental protein coordinates. Where previously only $〈\varphi ,\psi 〉$ and (independently) $〈{\chi }_1,{\chi }_2,\dots 〉$ observations could be validated, to varying consistency and accuracy, the current Bayesian method of PhiSiCal-Checkup supports amino-acid specific validation of dihedral angles in any combination of terms (conditional or otherwise) using strictly formal and mutually consistent statistical models, far-outstripping the scope of investigations currently possible. Further, the information-theoretic measure of ‘favorability’ along with the use of mathematical gradients to enable sensitivity analyses, allows experimentalists to quantify reliably the degree of surprise of any amino acid conformation while accounting for the uncertainty implicit in the protein coordinates.

Beyond these features, a significant effort has been directed toward engineering a configurable software that implements PhiSiCal-Checkup for immediate practical use by experimentalists. This is downloadable both as an open-source program written in C$++$ (for offline use), and as a web-server (for online use): https://lcb.infotech.monash.edu.au/phisical/checkup.

Several extensions to PhSiCal-Checkup are planned, earmarked as future work. Extending the current statistical models that analyze and assess individual amino-acid conformations, more generalized amino acid–specific statistical models are being constructed to permit assessment of short oligopeptide (k ≥ 1-mer) conformations and their spatially interacting k-mer ensembles. The most basic in this line of planned extensions comes in the form of statistical models for pairs of interacting amino acids (i.e., pair of interacting $1$-mers)—these extended models will permit ways to analyze and validate covalent (e.g., disulfide bridge) and non-covalent (e.g., hydrogen (−H) bonds) interactions that currently remain overlooked despite underpinning the protein 3D structure. Another study earmarked for immediate future work is to analyze the distribution of conformational angles of protein structures not determined by experimental methods but predicted using programs such as AlphaFold 3 (22). These predicted structures are increasingly being used in research and it would be interesting to compare the distributions with those derived from experimental coordinates.

Materials and Methods

Statistical Mixture Models.

A statistical mixture model describes a probability distribution expressed as a convex combination (i.e., a mixture) of component probability density functions. Formally, a parametric mixture model $\mathcal{M}$ composed of a mixture containing $|\mathcal{M}|$ component probability density functions is characterized as $\mathcal{M}(x)=\sum _{i=1}^{|\mathcal{M}|}{w}_i{f}_i(x|{\mathrm{\Theta }}_i)$. Here, x denotes any observation, $f_i(x|{\mathrm{\Theta }}_i)$ denotes the i-th probability density function in the mixture with parameters ${\mathrm{\Theta }}_i$, and w_i denotes the component-weight such that $\sum _{i=1}^{|\mathcal{M}|}{w}_i=1$. In unsupervised inference, all mixture parameters $\{|\mathcal{M}|,{\{w_i\}}_{\forall 1\le i\le |\mathcal{M}|},{\{{\mathrm{\Theta }}_i\}}_{\forall 1\le i\le |\mathcal{M}|}\}$ have to be inferred automatically from the observed data (i.e., from a set of observations of the form $X=\{x_1,x_2,\dots ,x_n\}$).

We recently described an unsupervised method to infer amino acid–specific joint mixture models from any given source collection of protein coordinates (11). The inference method relies on the Bayesian and information-theoretic criterion of Minimum Message Length (MML) (12, 13). Each inferred amino acid–specific mixture model describes a continuous joint probability distribution over its (vector of) dihedral angles terms ($〈\omega ,\varphi ,\psi ,{\chi }_1,{\chi }_2,\dots 〉$). Each term in the vector is a continuous random variable in the range $({-180}^{°},{+180}^{°}]$.

For an amino acid type aa with d dihedral angle terms, its corresponding joint mixture model $\begin{array}{c}{\mathcal{M}}^{(aa)}\\ \text{(joint)}\end{array}(\omega ,\varphi ,\psi ,{\chi }_1,{\chi }_2,\dots )$ defines a probability distribution on a wrapped multidimensional d-Torus (${\mathbb{T}}^d$). In our work, each component of $\begin{array}{c}{\mathcal{M}}^{(aa)}\\ \text{(joint)}\end{array}$ is a product of von Mises distributions, one for each dihedral angle term, thus defining a proper continuous probability density function in ${\mathbb{T}}^d$ space. For a set X of observed vector of dihedral angles of an amino acid type aa, $\begin{array}{c}{\mathcal{M}}^{(aa)}\\ \text{(joint)}\end{array}(\omega ,\varphi ,\psi ,{\chi }_1,{\chi }_2,\dots )$ is inferred using the Bayesian criterion of MML as the mixture model that best explains X. (Refer to Amarasinghe et al. (1) for details of the inference method.)

Deriving Marginal Probability Distributions.

Using the axioms of probability, amino acid–specific marginal probability distributions of any proper subset of dihedral angle terms can be derived from its corresponding joint mixture model.

Formally, let $A=\{\omega ,\varphi ,\psi ,{\chi }_1,{\chi }_2,\dots \}$ denote the set of d dihedral angle terms (random variables) for the amino acid type aa. Let B define any non-empty, proper subset of terms in A. Then, the marginal probability distribution of the subset B ⊂ A can be derived from the joint probability distribution of A by (contour) integrating (out of the joint distribution) all dihedral angle terms $\{z_1,\dots z_m\}$ in $B^{\complement }=A\setminus B$. (Note, if $|A|=d$ and $|B|=0<d^{\prime }<d$, then $m=d-d^{\prime }$.)

$$\begin{array}{c}{\mathcal{M}}^{(aa)}\\ \text{(marg.)}\end{array}(B\subset A)=\underset{\forall z_i\in B^{\complement }}{\underbrace{\oint \dots \oint }}\begin{array}{c}{\mathcal{M}}^{(aa)}\\ \text{(joint)}\end{array}(A)d{z}_1\dots d{z}_m$$

From the computational side, we note that the marginal distribution for any subset of dihedral angle terms in B is also a mixture model. It defines a continuous probability distribution in the subspace ${\mathbb{T}}^{d^{\prime }}\subset {\mathbb{T}}^d$. Because each component of the joint distribution is a product of von Mises distributions (over dihedral angle terms), a spatial projection of the joint mixture model $\begin{array}{c}{\mathcal{M}}^{(aa)}\\ \text{(joint)}\end{array}(A)$ from ${\mathbb{T}}^d$ space $\to {\mathbb{T}}^{d^{\prime }<d}$ space results in the corresponding $\begin{array}{c}{\mathcal{M}}^{(aa)}\\ \text{(marg.)}\end{array}(B)$. This allows us to efficiently compute any marginal mixture model on-the-fly (in real time) taking tens of microseconds on modern standalone computers (Results).

Deriving Conditional Probability Distributions.

Amino acid–specific conditional probability distributions of any proper subset of the joint dihedral angle terms, upon observing specific values of another subset of dihedral angles terms can be derived using Bayes theorem.

As introduced above, let B ⊂ A define any non-empty proper subset of A containing $0<d^{\prime }<d$ dihedral angle terms. Further, let $C\subseteq B^{\complement }\subset A$ denote another subset containing $0<d^{″}\le d-d^{\prime }$ terms. Assume the terms (i.e. random variables) in C are observed to take specific dihedral angle values $c=\{c_1,c_2,\dots c_{d^{″}}\}$. Then, the conditional probability distribution for the subset B after observing some specific values c for the terms (random variables) in C can be derived as:

$$\begin{array}{c}{\mathcal{M}}^{(aa)}\\ \text{(cond.)}\end{array}(B|C=c)=\frac{\begin{array}{c}{\mathcal{M}}^{(aa)}\\ \text{(marg.)}\end{array}(B\cup C=c)}{Pr(C=c)}.$$

In the above equation, $\begin{array}{c}{\mathcal{M}}^{(aa)}\\ \text{(marg.)}\end{array}(B\cup C=c)$ denotes a mixture model over the dihedral angle terms (random variables) in Bafter assigning the dihedral angle terms C = c in $\begin{array}{c}{\mathcal{M}}^{(aa)}\\ \text{(marg.)}\end{array}(B\cup C)$. Further, $Pr(C=c)\propto \begin{array}{c}{\mathcal{M}}^{(aa)}\\ \text{(marg.)}\end{array}(C=c)$ is the marginal probability of observing the dihedral angle terms (random variables) C = c.

As can be seen from the equation above, a conditional mixture model is deduced from the corresponding marginal mixture models, each of which is derived as a projection from the amino acid–specific joint mixture model. Hence, as before, the computation of any conditional mixture model in any combination of terms (i.e., subsets B and C of set A) can be performed highly efficiently on standalone computers in real time (Results).

Measures of Shannon Information and Compression.

From the mathematical theory of communication (16), the Shannon information content of any observation O is given by the relationship, $I(O)=-{log}_2(Pr(O))$ bits, where $Pr(O)$ is the probability of that observation drawn from some source probability distribution. In this work, an observation O involves observing specific values for the dihedral angles terms (in any combination of them). In PhiSiCal-Checkup, the information content in any observation O (denoted here by $I_{\text{mixture}}(O)$) is evaluated by computing its probability ${Pr}_{\text{mixture}}(O)$ under its corresponding amino-acid specific joint/marginal/conditional mixture model. The precise mixture model that has to be used to compute $I_{\text{mixture}}(O)$ is fully determined by the amino acid type of the observation and the combination of dihedral terms being evaluated.

Next, the measure of compression for the observation O can be derived by comparing the Shannon information content using the mixture model against its uncompressed raw/null bit content: $\text{Compression}(O)=I_{\text{null}}(O)-I_{\text{mixture}}(O)$ bits. Note, the uncompressed bit content is the same as measuring the Shannon information content of O with the uniform distribution as its source distribution.

Applying the negative-logarithm relationship between Shannon information content and source probabilities, compression is equivalent to computing the log-likelihood (or log-odds) of the observation O arising from each of the two competing distributions as its source (mixture vs. uniform null): $\text{Compression}(O)={log}_2(\frac{{Pr}_{\text{mixture}}(O)}{{Pr}_{\text{null}}(O)})$ bits. Thus, if an observation O results in $+n$ bits of compression (i.e., gain of n bits), the odds are $2^n$:1 in favor of O arising from the mixture model. Conversely, if it results in $-n$ bits of compression (i.e., loss of n bits), the odds are 1:$2^n$ against the observation O arising from the mixture model.

Gradient.

All joint, marginal, and conditional amino acid–specific mixture models are continuous distributions that are differentiable at every point in their respective toroidal support. For example, the gradient vector for $\begin{array}{c}{\mathcal{M}}^{(aa)}\\ \text{(joint)}\end{array}(A)$ at any point $a=\{a_1,a_2,\dots a_d\}\in {\mathbb{T}}^d$, is denoted by

$$\begin{array}{c}\hfill \begin{array}{cc}& \hfill \mathrm{\nabla }\begin{array}{c}{\mathcal{M}}^{(aa)}\\ \text{(joint)}\end{array}(A=a)=\hfill \\ \hfill & \hfill {\left[\frac{\partial \begin{array}{c}{\mathcal{M}}^{(aa)}\\ \text{(joint)}\end{array}(A=a)}{\partial a_1},\frac{\partial \begin{array}{c}{\mathcal{M}}^{(aa)}\\ \text{(joint)}\end{array}(A=a)}{\partial a_2},\dots ,\frac{\partial \begin{array}{c}{\mathcal{M}}^{(aa)}\\ \text{(joint)}\end{array}(A=a)}{\partial a_d}\right]}^{⊺}\hfill \end{array}\end{array}$$

For the mixture models involving the product of von Mises distributions, the result is in a closed mathematical form that can be computed on the fly. We note that the gradients for marginal and conditional mixture models yield similar expressions and characteristics since they also manifest as mixture models (except they are represented in the reduced dimensions $d^{\prime }$ of the subset B ⊂ A).

Thus, for any (multidimensional) point p, its gradient vector ($\overrightarrow{v}=\mathrm{\nabla }\mathcal{M}(p)$) gives the magnitude and direction of steepest ascent at p in its corresponding amino acid–specific probability distribution (joint/marginal/conditional mixture model $\mathcal{M}$). Using this gradient, p can be perturbed to a near-neighboring point $\tilde{p}=p\pm \lambda \widehat{v}$, where $\widehat{v}$ gives the direction cosines of the gradient vector $\overrightarrow{v}$. In practice, PhiSiCal-Checkup chooses λ = 5 for any p (for dihedral terms stated in degrees). This constrains the norm of the projection of $\widehat{p}-p$ in any (dihedral angle) dimension to $\le 5^{°}$.

Difference between PhiSiCal-Checkup and PhiSiCal.

The inference of joint mixture models to support validation of amino acid conformations in PhiSiCal-Checkup is derived using the previously described methodology, PhiSiCal (11). Although PhiSiCal-Checkup builds on the inference-methodology of PhiSiCal, realizing a comprehensive framework specifically for dihedral angle validation required several novel additions and extensions.

The new set of joint mixture models inferred for PhiSiCal-Checkup includes the modeling of ω dihedral angle, along with all other (main chain and side chain) dihedral angles for each amino acid type—previous work, PhiSiCal, ignored ω from its joint probability distribution. Further, the mixture models of PhiSiCal were inferred on PDB50 and PDB50HighRes structural dataset that lacked residue-level filtering. Instead, PhiSiCal-Checkup uses the “Top2018” dataset curated by Williams et al. (4). This dataset comes with residue-level filtering to ensure only the “best parts” of high-quality protein residues are considered with “good electron density support for a physically acceptable model conformation” (4). Furthermore, the factorization of joint mixture models to derive marginal and conditional mixture models, along with the use of compression and gradient information (all described above) to support accurate validation of protein coordinates are unique to PhiSiCal-Checkup.

Supplementary Material

Appendix 01 (PDF)

Data, Materials, and Software Availability

Supplementary Figures and Tables are included in SI Appendix. Supplementary Data has been deposited in FigShare (15, 19, 23, 24, 25, 26, 27). Software for online and offline use is available from https://lcb.infotech.monash.edu.au/phisical/checkup. All other data are included in the manuscript and/or SI Appendix.