BayeStab: Predicting effects of mutations on protein stability with uncertainty quantification

Abstract

Predicting protein thermostability change upon mutation is crucial for understanding diseases and designing therapeutics. However, accurately estimating Gibbs free energy change of the protein remained a challenge. Some methods struggle to generalize on examples with no homology and produce uncalibrated predictions. Here we leverage advances in graph neural networks for protein feature extraction to tackle this structure–property prediction task. Our method, BayeStab, is then tested on four test datasets, including S669, S611, S350, and Myoglobin, showing high generalization and symmetry performance. Meanwhile, we apply concrete dropout enabled Bayesian neural networks to infer plausible models and estimate uncertainty. By decomposing the uncertainty into parts induced by data noise and model, we demonstrate that the probabilistic method allows insights into the inherent noise of the training datasets, which is closely relevant to the upper bound of the task. Finally, the BayeStab web server is created and can be found at: http://www.bayestab.com. The code for this work is available at: https://github.com/HongzhouTang/BayeStab.

1. INTRODUCTION

A critical approach to investigate protein folding is to measure its thermodynamic properties. The folding process might be disturbed in mutated states, leading to changes in Gibbs free energy (∆∆G). This change is sometimes desired in the pharmaceutical industry, as antibody drugs typically need high thermal stability. ¹ Also, such a process is essential to understand how genome variation in drug targets can cause resistance to therapeutic drugs. ² , ³

To predict the stability change of proteins upon mutation with high throughput, computational approaches have been widely used. There were methods based on various evolutionary and physical chemical hypotheses with high performance. Another branch leveraged machine learning for fast identification, using techniques, such as support vector machine (SVM), ⁴ , ⁵ , ⁶ gradient boosting, ⁷ , ⁸ , ⁹ artificial neural network (ANN), ¹⁰ , ¹¹ and combinations of them. ¹² , ¹³ , ¹⁴ , ¹⁵ , ¹⁶ , ¹⁷ , ¹⁸ , ¹⁹ , ²⁰ However, several studies pointed out the significantly biased results of the machine learning‐based methods. ²¹ , ²² , ²³ In other words, they predict the destabilizing mutation more than the stabilizing mutation, and the seemingly high linear correlation between predicted and experimental results might not be shown in the stabilizing mutations.

Recent studies based on deep learning techniques, such as the convolution neural network, seem to handle this issue well, showing symmetric prediction. ²⁴ , ²⁵ , ²⁶ , ²⁷ Generally, deep learning requires large amounts of training data to improve performance. ²⁸ Currently, deep learning‐based approaches have been demonstrated with high performance comparable to classic machine learning methods. With new collected data ²⁹ , ³⁰ and potentially more in the future, it is not yet known how deep learning‐based methods will perform.

One conundrum in this field is how to further improve the representation learning of the models when limited experimental data are available. The graph neural network (GNN) is a powerful tool for extracting information from graph data. ³¹ Graph convolutional networks apply spectral convolution in the graph Fourier domain to aggregate neighboring representations for feature learning. ³² They have been used for protein structure refinement ³³ and protein function prediction. ³⁴ These attempts to encode protein context information make the prediction of mutation induced stability changes possible, yet it is still scarcely investigated.

Overfitting is another critical challenge to consider in machine learning‐based predictions. It happens when only limited experimental data are available, and the well‐trained models might not generalize well on unobserved datasets. Thus, the model must be flexible enough to capture all properties of the data. ³⁵ Probabilistic programming offers a way to generalize the models, allowing much richer representations of the model. It addresses this challenge by developing a distribution that encompasses the models using Bayesian theory. ³⁶ The key idea behind the probabilistic machine learning is to infer plausible models from the data with uncertainty. Compared to a pure deep learning model, which predicts a definite output, Bayesian machine learning's prediction corresponds to the aggregation of different neural networks trained on the same dataset. ³⁷ One advantage of the Bayesian approach is less prone to overfitting, since they are averaged over the parameters.

Meanwhile, the uncertainty quantified by the Bayesian method can be applied to investigate the inherent noise of the dataset, which is related to the upper bound performance. ³⁸ The key difficulty in using Bayesian neural networks (BNNs) is that Bayesian inference is computationally intractable. To reduce computation cost, researchers proposed using dropout at test times to enable uncertainty quantification of the predictive distribution. ³⁹ Concrete dropout is a dropout variant, which can be seen as a continuous relaxation of the discrete dropout. With appropriate regularization terms, this technique allows the dropout probability to be tuned using gradient methods and the uncertainty to be estimated.

Here we demonstrate that BNNs enabled by concrete dropout can be coupled with graph neural networks (GNN) to predict protein mutations' ∆∆Gs and estimate the uncertainties. The molecular representations learned by the feature extractor are operated on graph networks. After being combined with the coordination of the atoms, they are then processed by fully connected layers to map the high‐dimensional features to the low‐dimensional properties. To enable faster training, we retained the mutant part only and trimmed the rest. Our deep learning model is trained end‐to‐end, from protein feature vectors to the output property (Figure 1a).

FIGURE 1.

(a) The BayeStab's processing can be summarized into five steps: input the protein data, trim the nonmutant part, encode the protein vector representation, train the BNN, and predict the ΔΔG and uncertainty. (b) Illustration of the adjacency matrix and molecular information in the feature vectors. (c) The structure of the BayeStab model. (d) The underlying theory of Bayesian method to predict ΔΔG and quantify the uncertainty

We test our method on four public datasets, and the model outperforms previous approaches, showing improved generalization ability. Based on the BNN, we estimate the prediction uncertainty and decompose the uncertainty into parts induced by model data noise, which offers significant insights for investigating the upper bound of the performance. Last, BayeStab web server is presented to serve the broad scientific community.

2. THEORETICAL BACKGROUND

In this section, we first introduce the Bayesian inference model and variational inference as an approximation. Then, we illustrate how to quantify the uncertainty in a BNN. Next, we explain the working principle of our GNN.

2.1. Bayesian inference

Given a training set {X, Y}, where X is the protein feature and Y is ∆∆G upon mutation. p (Y|X, w) is the likelihood of the model and p (w) is the prior distribution. w = { W ₁ , …, W _k } is the model parameters with a structure of k layers structure. In a Bayesian framework, the posterior is calculated as:

$$p\left(\mathbf{w}\mid \mathbf{X},\mathbf{Y}\right)=\frac{p\left(\mathbf{Y}\mid \mathbf{X},\mathbf{w}\right)p\left(\mathbf{w}\right)}{p\left(\mathbf{Y}|\mathbf{X}\right)}$$ (1)

The predictive distribution of the problem can be defined as follows:

$$p\left({\mathbf{y}}^{*}\mid {\mathbf{x}}^{*},\mathbf{X},\mathbf{Y}\right)=\int p\left({\mathbf{y}}^{*}|{\mathbf{x}}^{*},\mathbf{w}\right)p\left(\mathbf{w}\mid \mathbf{X},\mathbf{Y}\right)d\mathbf{w}$$ (2)

where y ^* is the output of input x ^* for a given w.

Direct application of the formula is impractical due to the high computation cost. Variational inference can approximate the posterior using a tractable distribution q _θ (w) parameterized by the parameter θ . By minimizing the Kullback–Leibler (KL) divergence,

$$\mathrm{KL}\left(q_{\theta }\left(\mathbf{w}\right)\Vert p\left(\mathbf{w}\mid \mathbf{X},\mathbf{Y}\right)\right)={\int }_{\mathrm{\Omega }}{q}_{\theta }\left(\mathbf{w}\right)\log \frac{q_{\theta }\left(\mathbf{w}\right)}{p\left(\mathbf{w}\mid \mathbf{X},\mathbf{Y}\right)}d\mathbf{w}$$ (3)

We can combine the intractable posterior distribution in Equation (3) with Equation (1). Then, the variational approximation of the negative evidence lower‐bond becomes:

$${\mathrm{ℒ}}_{\mathit{VI}}\left(\theta \right)=-{\int }_{\mathrm{\Omega }}{q}_{\theta }\left(\mathbf{w}\right)\log \mathrm{p}\left(\mathbf{Y}\mid \mathbf{X},\mathbf{w}\right)d\mathbf{w}+\mathrm{KL}(q_{\theta }\left(\mathbf{w}\right)\Vert p\left(\mathbf{w}\right)$$ (4)

To implement a Bayesian model, q _θ (w) is needed. Concrete dropout inside a neural network can approximate the posterior distribution without extra learnable parameters, and the integral across the full parameter space can be retrieved by Monte Carlo (MC) sampling.

2.2. Quantification of uncertainty with BNN

Given a new input x ^* , the variational distribution of the output, y ^* , can be obtained as:

$$q_{\theta }\left({\mathbf{y}}^{\mathbf{*}}|{\mathbf{x}}^{\mathbf{*}}\right)=\int p\left({\mathbf{y}}^{\mathbf{*}}|f^{\mathbf{w}}\left({\mathbf{x}}^{\mathbf{*}}\right)\right)q_{\theta }\left(\mathbf{w}\right)d\mathbf{w}$$ (5)

where f ^w(x*) is the output of the model for a given w. The predictive mean of this distribution with T times of MC sampling is estimated for regression tasks by:

$$\widehat{E}\left[y^{*}|x^{*}\right]=\frac{1}{T}\sum _{t=1}^T{f}^{{\widehat{\mathbf{w}}}_{\mathbf{t}}}\left({\mathbf{x}}^{*}\right)$$ (6)

and a predictive variance is estimated by:

$$V\widehat{a}r\left[{\mathbf{y}}^{*}|{\mathbf{x}}^{*}\right]=\frac{1}{T}\sum _{t=1}^T{f}^{\widehat{{\mathbf{w}}_t}\mathrm{c}}{\left({\mathbf{x}}^{*}\right)}^T{f}^{\widehat{{\mathbf{w}}_t}}\left({\mathbf{x}}^{*}\right)-\widehat{E}{\left[{\mathbf{y}}^{*}|{\mathbf{x}}^{*}\right]}^T\widehat{E}\left[{\mathbf{y}}^{*}|{\mathbf{x}}^{*}\right]$$ (7)

The uncertainty can be divided in two parts: aleatoric and epistemic uncertainty. The aleatoric uncertainty is inherent in the noise from the datasets, while the epistemic uncertainty is caused by the prediction of the model. The uncertainty's segmentation is as follows:

$$V\widehat{a}r\left[{\mathbf{y}}^{*}|{\mathbf{x}}^{*}\right]=\begin{array}{l}\underset{\text{epostemic}}{\underbrace{\frac{1}{T}\sum _{t=1}^T\left({\widehat{\mathbf{y}}}_t^{*}-\overline{\mathbf{y}}\right){\left({\widehat{\mathbf{y}}}_t^{*}-\overline{\mathbf{y}}\right)}^T}}\\ +\underset{\text{aleatoric}}{\underbrace{\frac{1}{T}\sum _{t=1}^T\left(\mathit{diag}\left({\widehat{\mathbf{y}}}_t^{*}\right)-\left({\widehat{\mathbf{y}}}_t^{*}\right){\left({\widehat{\mathbf{y}}}_t^{*}\right)}^T\right)}}\end{array}$$ (8)

where $\overline{\mathbf{y}}=\sum _{t=1}^T$ ${\widehat{\mathbf{y}}}_t^{*}/\mathrm{T}$. ${\widehat{\mathbf{y}}}_t^{*}$ = softmax(f ^{w t} (x _t )), and f ^{w t} (x _t ) is the neural network's output with input x _t .

2.3. GNN for feature learning

The inputs to the GNN X = H ⁽⁰⁾ are the adjacency matrix, A, and the initial node features, which consisted of atom types, adjacent atoms, number of adjacent hydrogen, implicit valence and aromatic bonds (Figure 1b).

The GNN's message passing through a single layer is as follows:

$$H^{\left(l+1\right)}=\mathit{\text{Leaky}}\_\mathit{\text{relu}}\left(W^{\left(l\right)}{\mathit{AH}}^{\left(l\right)}\right)$$ (9)

where H ^(l) and W ^(l) are node features and trainable parameters at the l‐th layer, l ∈ {0, …, L}, respectively. The GNN updates the node feature H ^(l+1) with information from adjacent nodes for representation learning.

To improve the feature extraction performance, we integrated the gating mechanism into the network as:

$$H_{\mathit{\text{gate}}}^{\left(l+1\right)}={\mathit{GH}}_{\mathit{\text{gate}}}^{\left(l+1\right)}+\left(1-G\right)H_{\mathit{\text{gate}}}^{\left(l\right)}$$ (10)

with

$$G=\mathit{\text{leaky}}\_\mathit{\text{relu}}\left(W_{\mathit{\text{gate}}}\left[H_{\mathit{\text{gate}}}^{\left(l\right)}{H}_{\mathit{\text{gate}}}^{\left(l+1\right)}\right]+B\right)$$ (11)

After updating the mode features L‐times through feedforward computations, the graph feature h _G is obtained by summation of all N node:

$${\mathit{h}}_G=\sum _{n\in N}\mathrm{NN}\left(H_n^{\left(L\right)}\right)$$ (12)

3. EXPERIMENTS AND METHODS

3.1. Datasets

S2648 contains 2,648 single point mutations from 131 different globular proteins. The ProTherm database is the source of the dataset. In this dataset, 2,080 of them are destabilizing and 568 are stabilizing. We use S2648 as the training dataset for BayeStab.

Q3421 includes 3,421 mutations from 150 proteins. We use the dataset for 10‐fold cross‐validation.

S350 consists of 350 mutations in 67 different proteins. It is a subset of the S2648 dataset, so the overlapped part needs to be tailored during training.

S611 is developed by DynaMut2, ¹⁷ which is split from a dataset of 4,633 mutations.

S669 is a latest curated test dataset ⁴⁰ manually cleaned from the ThermoMutDB database. It consists of 669 variants of protein sequences that do not share homology with the S2648 dataset and Varibench.

Myoglobin is the globular protein that regulates the concentration of cellular oxygen. ⁴¹ The dataset consists of 134 mutations scattered throughout the protein chain, which also does not overlap with the training dataset.

S ^sym contains 684 variations, and half of them are reverse variations with crystal structures of the corresponding mutant proteins. ⁴² We use the S^sym dataset to investigate the uncertainty in the dataset and in the model.

3.2. Implementation and evaluation

The schematic view of the BayeStab is shown in Figure 1c, and the sizes of each layer in the architecture are listed in Table 1.

TABLE 1.

The architecture of the BayeStab

Layer type	Specifications
GNN layer + dropout ×4	Size:1400
FC layer + dropout + ReLU	Size:1024
FC layer + dropout + ReLU	Size:512
FC layer + dropout + ReLU	Size:256
FC layer + dropout	Size:1

The two branches for processing wild and mutant proteins are symmetric, with both the GNN module and the FC module. The summation of the atom coordinates is concatenated to the latent feature extracted by the GNN. Finally, the output of the wild protein is subtracted from the mutant protein to obtain the ∆∆G. At each hidden layer, we applied the concrete dropout, which leads to the corresponding uncertainty estimation. The principles for quantifying and decomposing the uncertainty are also illustrated in Figure 1d.

In the training phase, we used the Adam optimizer with the learning rate of 10⁻³ for 400 epochs. The dropout was performed at the inference phases, sampled with T = 10 for Bayesian inference. The model was implemented using Pytorch on a GTX‐3070 processor.

To evaluate the prediction accuracy, we use the Pearson correlation coefficient (r) between experimental and predicted ∆∆Gs and the root mean squared error (σ) of predictions. To quantify the prediction bias, we adopt r between the predicted results for direct mutations and reverse mutations and the error, δ = ΔΔG _rev  + ΔΔG _dir . ⁴²

4. RESULT AND DISCUSSION

4.1. Testing results on four datasets

After 10‐fold cross‐validation of the S2648 dataset, BayeStab showed r = 0.61 and σ = 1.19 kcal/mol. The Pearson correlation coefficient increased to 0.69 and σ decreased to 1.06 kcal/mol after removing 5% of the outliers (Figure 2). When we performed a 10‐fold cross‐validation on the Q3421 dataset, r was 0.68, and σ was reduced to 1.29 kcal/mol, if 5% of the outliers were removed (Figure 3).

FIGURE 2.

Cross validation results of the S2648 dataset. With 5% of the outliers removed (blue dots), r = 0.69, σ = 1.06 kcal/mol

FIGURE 3.

Cross validation results of the Q3421 dataset. With 5% of the outliers removed (blue dots), r = 0.68, σ = 1.29 kcal/mol

Then, we tested the trained model on S611, S350, Myoglobin, and S669 datasets, respectively. Before training, the overlap between the training and testing datasets were tailored for assessment. Since BayeStab can predict with the corresponding uncertainty, we marked the data points with various colors to indicate its probability (Figure 4).

FIGURE 4.

BayeStab's performance when tested on four datasets. The corresponding prediction uncertainty is marked using four different colors. (a) Predicting ΔΔG for direct mutations in S611, (b) reverse mutations in S611, (c) direct versus reverse ΔΔG values in S611. (d) Predicting ΔΔG for direct mutations in S350, (e) reverse mutations in S350, (f) direct versus reverse ΔΔG values in S350. (g) Predicting ΔΔG for direct mutations in Myoglobin, (h) reverse mutations in Myoglobin, (i) direct versus reverse ΔΔG values in Myoglobin, (j) predicting ΔΔG for direct mutations in S669, (k) reverse mutations in S669, (l) direct versus reverse ΔΔG values in S669

When evaluated using the S611 dataset, BayeStab obtained r = 0.73, σ = 0.99 kcal/mol in the direct mutations, r = 0.73, σ = 0.99 kcal/mol in the reverse mutations, and r = −0.97, δ = 0.01 in direct‐reverse prediction (Figure 4a–c). We further analyze the performance of the stabilizing and destabilizing mutations, respectively. BayeStab's performance on destabilizing and stabilizing mutations were r = 0.72, σ = 1.02 kcal/mol and r = 0.48, σ = 1.28 kcal/mol. Comparing with other methods, BayeStab improved performance on the overall (Table 2).

TABLE 2.

Comparison of different methods tested on the S611 dataset

	Overall		Stabilizing mutations		Destabilizing mutations
Method	σ	r	σ	r	σ	r
BayeStab	0.99	0.73	1.28	0.48	1.02	0.72
DUET	1.40	0.48	1.75	0.09	1.00	0.58
DynaMut2	1.14	0.68	1.02	0.51	0.91	0.62
SDM	1.93	0.35	1.62	0.48	−0.77	0.03
mCSM	1.42	0.46	1.81	0.11	0.98	0.56
MAESTRO	1.55	−0.36	1.17	0.27	1.81	0.43
I‐mutant	1.47	0.33	1.83	0.03	1.09	0.49

Next, BayeStab was tested on the S350 dataset and achieved r = 0.75, σ = 1.09 kcal/mol in direct mutations, r = 0.75, σ = 1.05 kcal/mol in reverse mutations, and r = −0.97, δ = −0.02 kcal/mol in direct‐reverse prediction (Figure 4d–f). Meanwhile, we split the results into stabilizing and destabilizing mutations. We found that BayeStab's strong performances on stabilizing mutations were r = 0.66, σ = 1.29 kcal/mol, and destabilizing mutations showed r = 0.62, σ = 1.37 kcal/mol. Our results tested on S350 dataset were also compared with six other methods (Table 3). BayeStab's performance also exceeded prior methods when dealing with the imbalance problem.

TABLE 3.

Comparison of different methods tested on the S350 dataset

	Overall		Stabilizing mutations		Destabilizing mutations
Method	σ	r	σ	r	σ	r
BayeStab	1.09	0.75	1.29	0.66	1.37	0.62
DUET	1.31	0.67	1.00	0.65	2.23	0.28
DynaMut2	1.37	0.66	1.16	0.63	2.01	0.38
SDM	1.80	0.52	1.43	0.42	3.12	0.15
mCSM	1.08	0.66	1.01	0.63	2.48	0.31
MAESTRO	1.79	0.55	1.52	0.43	1.37	0.61
I‐mutant	1.75	0.53	1.42	0.42	2.89	0.25

The Myoglobin dataset does not overlap with the training data, indicating that it is appropriate for estimating overfitting. Our tested results on this dataset were r = 0.47, σ = 1.07 kcal/mol on direct mutations, r = 0.47, σ = 1.07 kcal/mol on reverse mutations, and r = −0.97, δ = −0.01 kcal/mol on the direct‐reverse predictions (Figure 4g–i).

The latest curated dataset, S669, is also highly convincing for performance evaluation, since it is not included in the widely available training datasets. On the S669 dataset, BayeStab also achieved superior symmetry, showing r = −0.97, δ = −0.01 kcal/mol for direct‐reverse prediction. Its performance on direct mutations reached r = 0.54, σ = 1.60 kcal/mol, and MAE = 1.07 kcal/mol. The reverse mutations showed r = 0.53, σ = 1.62 kcal/mol, and MAE = 1.07 kcal/mol (Figure 4j–l). Fifteen recently shown methods were also listed for comparison with BayeStab (Table 4). Our method's performance is highly competitive to be the state‐of‐the‐art approach, showing highest linear correlation and improved symmetry.

TABLE 4.

BayeStab compared with 15 recent methods tested on the S669 dataset. The data are adopted from Reference 40

	Direct			Reverse			Dir‐rev
Method	r	σ	MAE	r	σ	MAE	r d‐r	δ
BayeStab	0.54	1.60	1.07	0.53	1.62	1.07	−0.97	−0.01
ACDC‐NN	0.46	1.49	1.05	0.45	1.50	1.06	−0.98	−0.02
DDGun3D	0.43	1.60	1.11	0.41	1.62	1.14	−0.97	−0.05
PremPS	0.41	1.50	1.08	0.42	1.49	1.05	−0.85	0.09
ThermoNet	0.39	1.62	1.17	0.38	1.66	1.23	−0.85	−0.05
Rosetta	0.39	2.70	2.08	0.40	2.68	2.02	−0.72	−0.61
Dynamut	0.41	1.6	1.19	0.34	1.69	1.24	−0.58	−0.06
INPS3D	0.43	1.5	1.07	0.33	1.77	1.31	−0.50	−0.06
SDM	0.41	1.67	1.26	0.13	2.16	1.64	−0.40	−0.40
PopMuSic	0.41	1.51	1.09	0.24	2.09	1.64	−0.32	−0.69
MAESTRO	0.50	1.44	1.06	0.20	2.10	1.65	0.22	−0.57
FoldX	0.22	2.30	1.56	0.22	2.48	1.50	−0.20	−0.34
DUET	0.41	1.52	1.10	0.23	2.14	1.68	−0.12	−0.67
I‐Mutant3.0	0.36	1.52	1.12	0.15	2.32	1.87	−0.06	−0.81
mCSM	0.36	1.54	1.13	0.22	2.30	1.86	−0.05	−0.85
Dynamut2	0.34	1.58	1.15	0.17	2.16	1.69	0.03	−0.64

4.2. Uncertainty decomposition

We then decomposed the uncertainties obtained from BayeStab and compared the uncertainties with all, 1/2, and 1/4 of the training dataset. When we tested on the S^sym dataset, we found the aleatoric uncertainty remained almost unchanged, whereas the epistemic uncertainty increased as the amount of training data decreased (Table 5). This effect can be explained as the model‐induced uncertainty increased due to the less training data, while the uncertainty inherent in the experimental data remained the same.

TABLE 5.

BayStab estimated the epistemic and aleatoric uncertainties when trained using various amounts of the S2648 dataset and tested on the S^sym dataset

Training dataset	Epistemic	Aleatoric
S2648	0.03	0.25
S2648/2	0.08	0.24
S2648/4	0.13	0.25

Besides, we could estimate how much noise from the dataset contributed to the predicted error. For the past two decades, the performance of the machine learning‐based method seemed to have an upper bound. The prediction error, σ, stagnated at around 1 kcal/mol, yet the inherent noise of the dataset was rarely explored.

With BNN's powerful uncertainty division, we may find the dataset's noise is dominant in the overall uncertainty, indicating the model has almost reached the upper bound performance with the data available. More experimental data with the current measurement accuracy may not lead to higher performance, as the epistemic uncertainty is already very small compared with the aleatoric uncertainty.

4.3. Web server

We built a freely available and user‐friendly web server (http://www.bayestab.com) using Flask. The home page and the result page of the web server are shown in Figure 5.

FIGURE 5.

BayeStab web server. (a) The home page (b) and the result page of the web server

The web server takes the structure information of the protein as the input. Users can upload PDB files of the wild type and mutant types to the server. The mutant type PDB files can be generated by Rosetta. Next, the user needs to fill in the mutation information. For example, L37S indicates that at the position of amino acid number 37, and leucine (L) becomes serine (S). Users also need to fill in the mutant protein chain information, such as A or B. Last, the user can get the predicted ∆∆G after submitting the task.

5. CONCLUSION

Here, we fuse the BNN and GNN‐based methods to predict proteins' stability change upon mutations with quantified uncertainty. Our end‐to‐end deep learning model, BayeStab, can effectively learn molecular feature representations to predict the ∆∆G with significantly high performance.

The cross‐validations on S2648 and Q3421 datasets show high linearity and low errors. Superior performance is also demonstrated when tested on four datasets. The predicted results are highly symmetric between direct and reverse mutations without bias toward predicting destabilization. The test results on the S669 are especially persuasive for proving BayeStab's improved generalization, as it has novel variants never encountered by the prior prediction tools. BayeStab achieved high Pearson correlation coefficients that outperformed state‐of‐the‐art methods.

In addition, we propose to integrate concrete dropout in the GNN as our Bayesian approach to quantify the uncertainty, then we further decompose the uncertainty to model‐induced and data noise‐induced parts. To the best knowledge of the authors, this is a novel work to introduce uncertainty quantification into this field. Using the model trained on S2648 and tested on S^sym, we find the noise from the dataset is dominant in the prediction errors, indicating that the prediction upper bound is almost approaching. We also suspect that even if more experimental data are available, the improvement might still be subtle.

Last, BayeStab is also made accessible to wider users through a free web server. In the future, we hope BayeStab would benefit the research community to study protein dynamics and envision its contribution to deepen the understanding of mutations in diseases.

AUTHOR CONTRIBUTIONS

Shuyu Wang: Conceptualization (lead); formal analysis (lead); funding acquisition (lead); investigation (equal); methodology (lead); project administration (lead); software (equal); supervision (lead); validation (lead); writing – original draft (equal); writing – review and editing (lead). Hongzhou Tang: Investigation (equal); methodology (equal); software (equal); visualization (lead); writing – original draft (equal). Yuliang Zhao: Funding acquisition (supporting); project administration (supporting); resources (supporting); validation (supporting); visualization (supporting). Lei Zuo: Conceptualization (supporting); funding acquisition (supporting); project administration (supporting); writing – review and editing (equal).

CONFLICT OF INTEREST

The authors declare no conflict of interest.

Wang S, Tang H, Zhao Y, Zuo L. BayeStab: Predicting effects of mutations on protein stability with uncertainty quantification. Protein Science. 2022;31(11):e4467. 10.1002/pro.4467