GeoPacker: A novel deep learning framework for protein side‐chain modeling

Abstract

Atomic interactions play essential roles in protein folding, structure stabilization, and function performance. Recent advances in deep learning‐based methods have achieved impressive success not only in protein structure prediction, but also in protein sequence design. However, highly efficient and accurate protein side‐chain prediction methods that can give detailed atomic interactions are still lacking. In the present study, we developed a deep learning based method, GeoPacker, that uses geometric deep learning coupled ResNet for protein side‐chain modeling. GeoPacker explicitly represents atomic interactions with rotational and translational invariance for information extraction of relative locations. GeoPacker outperformed the state‐of‐the‐art energy function‐based methods in side‐chain structure prediction accuracy and runs about 10 and 700 times faster than the deep learning‐based method DLPacker and OPUS‐rota4 with comparable prediction accuracy, respectively. The performance of GeoPacker does not depend on the secondary structures that the residues belong to. GeoPacker gives highly accurate predictions for buried residues in the protein core as well as protein–protein interface, making it a useful tool for protein structure modeling, protein, and interaction design.

Abbreviations

pdf

probabilistic distribution function

rSASA

relative solvent accessible surface area

RMSD

root mean square deviation

1. INTRODUCTION

Protein tertiary structure prediction ¹ , ² , ³ , ⁴ , ⁵ has achieved revolutionary improvements using deep learning methods and tremendous number of homologous sequences. Protein structure is determined by its primary sequence in principle, however, the prediction task is extremely challenging for orphan sequence ⁶ , ⁷ , ⁸ due to the immense conformational space caused by the lacking of constraints from coevolutional signals. Protein de novo design aims to produce sequences which are far away from the existing natural sequence space and compatible with the pre‐determined protein backbones. Thus, the structure prediction validation of de novo designed protein without homologous sequences is difficult. Most recently developed deep learning‐based methods for protein sequence design ⁹ , ¹⁰ , ¹¹ , ¹² , ¹³ , ¹⁴ , ¹⁵ , ¹⁶ , ¹⁷ only restore the amino acid residue components for a given protein backbone without constructing the atom‐level interactions because of the coarse‐grained model used. An alternative way to circumvent the structure prediction for de novo designed sequence is to accurately model side chain conformations at atom level, as the main driving force for soluble protein folding is the inner core hydrophobic interactions. In addition, protein side‐chain conformation is closely linked to protein biological functions. ¹⁸ Hence, developing fast and accurate protein side‐chain modeling method is important for protein structure prediction, protein engineering, and de novo protein design.

Traditional computational tools for protein side‐chain modeling ¹⁹ , ²⁰ , ²¹ , ²² consist of a rotamer library, a search algorithm, and an energy scoring function based on force field or statistical potentials. Most of those tools achieved similar performance in predicting the side‐chain dihedral angles within a stringent tolerance of 20° deviation. ²² However, defining accurate energy function and high‐quality rotamer library remains difficult, which hinders the accuracy of side‐chain modeling methods. ²³ , ²⁴ , ²⁵ , ²⁶ In recent years, application of deep learning approaches brought breakthroughs in protein structure prediction ² as well as other demanding scientific problems. ²⁷ , ²⁸ Deep learning‐based methods have also been used to tackle the side‐chain modeling problem. ²⁹ , ³⁰ , ³¹ , ³² , ³³ For example, DLPacker, ³¹ a 3D convolutional neural network‐based method formulated as an image‐to‐image transformation, gained a large margin of accuracy improvement which is sampling‐based and discrete rotamer library dependent that are computational cost demanding. OPUS‐Rota4, ³² which assembled the DLPacker network architecture and predicted the distance, orientations, and an initial side‐chain model, then those predicted geometries were used as constraints for further accuracy improvement of the side‐chain model. ³ At the same time, OPUS‐Rota4 also employed co‐evolutional information of the target sequence for prediction. Despite high accuracy in side‐chain modeling, the requirement of many hierarchical input features makes OPUS‐Rota4 much slower in calculation and not easy to be applied.

Protein structure modeling and de novo protein design demand both accurate and fast side‐chain modeling calculations. Here we developed a geometric deep learning coupled ResNet ³⁴ model, GeoPacker, to directly predict side‐chain dihedral angles. GeoPacker is rotamer library‐free and rotational and translational invariant. When repacking side chains for both natural proteins and de novo designed proteins, GeoPacker outperforms the state‐of‐the‐art energy function‐based methods in accuracy and DLPacker in speed. GeoPacker provides a user‐friendly tool for large‐scale protein side‐chain modeling tasks.

2. RESULTS AND DISCUSSION

2.1. Model overview

We devise a simplified geometric deep learning coupled ResNet to build a multitask learning architecture where tasks are related and mutually supervised, thus improving the side‐chain dihedral angle prediction accuracy and robustness. Another two auxiliary objectives are achieved which are the distance prediction of pair‐wise distal side‐chain heavy atoms (Table S1) ³⁵ and orientation prediction defined as the dihedral angle of C _β –C _γ ···C _γ –C _β (N–C _α for Gly, C _γ –C _β for Ala, C _γ1 for Ile and Val, and C _γ2 for Thr) from two residues. These two additional constraints are theoretically enough to reconstruct and refine the full‐atom model. Similar to protein distance map prediction, ³ the dihedral angle is split into 48 bins (7.5° per bin) from −180° to 180°. For a protein backbone X and a sequence Y, we simplify and format the conception:

$$P\left(R|X,Y\right)=\begin{array}{l}P\left(r_1,r_2,r_3,\dots ,r_n|Y,X\right)\\ \stackrel{\text{independent}}{=}P\left(r_1|Y,X\right)*P\left(r_2|Y,X\right)*\dots \\ *P\left(r_n|Y,X\right),\end{array}$$ (1)

where $P\left(r_i\mid Y,X\right)={\prod }_{j=1}^{4}P\left({\chi }_i^j\mid Y,X\right),i=1,2,\dots ,n,j=1,2,3,4$. Here, we assume that each residue rotamer is independent, which relies on the whole sequence to accelerate inferences. As a matter of fact, the specific residue rotamer is determined by its local microenvironment, but this hypothesis can be effectively alleviated and approximately reasonable to some extend with the addition of coarse‐grained rotamer‐free residue types. The kernel density estimation is used to approximate the probabilistic distribution function (pdf) of each bin from the curated dataset, then the specific dihedral angle ${\chi }_i^j$ is sampled from the predicted discrete bin,

$${\chi }_i^j\sim \mathrm{pdf}\left[\text{BinLeft},\text{BinRight}\right]$$ (2)

An overview of the algorithm is shown in Figure 1. The GNN module is used for encoding the local geometric features of protein and the ResNet module is applied for extracting the global constraints information of distance map of pair‐wise C _α s.

FIGURE 1.

A deep learning framework for protein side‐chain modeling. (a) The left is a sequence, the medium is GNN module for local information extraction, and the right is ResNet plus attention module for global constraint from distance map. After the ResNet module, two auxiliary objective predictions are, respectively, addressed following a 2D CNN and a fully connected layer. (b) GNN module modeled by protein backbone. The color and aggregation updates to present each time/layer status of the residues and links, respectively. (c) Data processing workflow of ResNet module and self‐attention module. (d) Geometric features for pair‐wise residues r _i and r _j , including backbone atoms C, N, O, C _α , and a pseudo C _β .

2.2. Performance and comparison

The common metric for protein side‐chain modeling is the dihedral angle recovery in a stringent tolerance, that is, within 20° deviations for every χ _i in a residue when compared with the experimental structure. Protein side‐chain recovery is defined as the rate of correctly predicted side‐chain conformation over all in a stringent tolerance,

$$\text{protein side}‐\text{chain recovery rate}=\frac{\sum _i^L{I}_i\left(r\right)}{L},$$ (3)

where L is the length of protein chain, $I_i\left(r\right)={\prod }_{j=1}^4{I}_i^j\left(r^j\right)$ is an indicator function for residue i, $I_i^j$ is an indicator function of the j ^th side‐chain dihedral angle of residue i, respectively. $I_i^j$ equals 1 only if the j ^th side‐chain dihedral angle of residue i is predicted correctly, otherwise equals 0. In addition, the side‐chain recovery for individual type of residues is also used as a metric:

$${\text{residue}}_k\text{side}-\text{chain recovery rate}=\frac{\sum _{i=1}^N\sum _{j=1}^{L_i}{I}_{\mathit{ij}}}{\sum _{i=1}^N\sum _{j=1}^{L_i}{I}_{\left(r_{\mathit{ij}}=={\text{residue}}_k\right)}},$$ (4)

where k = 1,2, …, 20, N is the total number of protein chains in the dataset, L _i is the length of ith protein chain, r _ij is the residue type of j ^th residue in i ^th protein chain, I _ij is an indicator function, respectively. I _ij equals 1 only if jth residue in i ^th protein belongs to residue _k and all of their side‐chain dihedral angles are predicted correctly, otherwise equals 0.

Depending on side‐chain length, we set χ ₁ for Cys, Pro, Ser, Thr, and Val, χ _1–2 for Asp, Asn, Ile, Leu, Phe, Trp, and Tyr, χ _1–3 for Met, Glu, and Gln, and χ _1–4 for Arg and Lys. It is worth pointing out that several side‐chain groups are symmetric under 180° rotations, including Phe, Tyr, Asp, and Glu, while all are periodic, such that −175° and 175° are actually only 10° apart.

In order to avoid overfitting when training the model, the whole data partition is done to shape cross‐validation test. Specifically, we randomly split 14,888 protein chains into a training set containing 13,000 chains for fourfold cross‐validation and an independent testset containing 1888 chains (TS1888). The average performance of the four models is evaluated on the TS1888.

We firstly depict the distribution of each protein side‐chain recovery (Figure 2a), where the overall protein side‐chain recovery is 65.95% in a stringent tolerance within 20° in TS1888. In addition, we also compute the correlation (Figure 2b) between side‐chain recovery and the root mean square deviation (RMSD) of side‐chain heavy atoms from the native structure without the backbone atoms (i.e., N, C, C _α , and O) and a pseudo C _β , which is −0.64. Intuitively, the more accurate the side‐chain recovery is, the smaller the RMSD is, however, the reverse is not true. For example, if only the last predicted dihedral angle of all residues were extremely deviated from the native, the RMSD would be expected to be small, while the side‐chain recovery rate would be zero as we define that the side‐chain recovery needs all the dihedral angles be correctly predicted. Similar phenomenon was demonstrated before in the reference. ²⁶ Further analysis showed that (Figure S2) most of the wrong predictions are side chains on the surface, especially for the χ _3–4. One of the potential reasons is that the sparse microenvironment compared with the crowded inner core region. In addition, side‐chains exposed to solvent are usually flexible which may also take different conformations in different crystal structures, while buried side chains in the hydrophobic core are normally rigid. Thus rotamer recovery rate for exposed and buried side chains should be separately compared. We define the buried residues as those with relative solvent accessible surface area (rSASA) lower than 15% ³⁶ measured by NACCESS ³⁷ and exposed residues as those with rSASA larger than 15%. As shown in Figure 3a, the structures of the buried side chains are more precisely modeled than the exposed residues. We further checked whether side chain modeling accuracy depends on secondary structures as it is well known that favorable side chain rotamers are backbone dependent. ³⁸ , ³⁹ To this end, we reduce the 8‐state DSSP ⁴⁰ secondary structures to 3 states according to the following rules: α‐helix (H) and 3–10 helix (I) to H; extended conformation (E) and isolated bridge (B) to E; others to C. The prediction accuracies for residues in the three types of secondary structures were not much different (Figure 3b). Side chains of the three aromatic residues, Trp, Phe, Tyr on extended structures (E) are more accurately modeled, while side chains of long charged residues are less accurate on loops.

FIGURE 2.

(a) The distribution of protein side‐chain χ _1–4 recovery within 20°; (b) Correlation between side‐chain recovery and RMSD of side‐chain heavy atoms relative to experimental structure

FIGURE 3.

Side‐chain recovery rate of individual residue types with respect to the rSASA (a) and different secondary structures in a stringent tolerance within 20° (b)

We then investigated the importance of different components in the network model. We first removed the distance map and the corresponding 2D task predictions. Interestingly, the side‐chain recovery rate did not change much (64.80% vs. 64.55%), while the standard deviation varies significantly (0.14% vs. 0.17%), indicating that the single task prediction is more fluctuant. We further eliminated the distance prediction of distal heavy atoms and orientation prediction. The performances in both cases were decreased and the distance prediction contributes more. This analysis demonstrates the advantages of multitask learning for accuracy and robustness (Table 1).

TABLE 1.

Components selection of model architecture and recovery of χ _1–4 in a stringent tolerance within 20°

Component excluded	Recovery (%, cross validation)			Recovery (%, test)
	All	Buried	Exposed	All	Buried	Exposed
None	64.80 ± 0.14	82.26 ± 0.27	56.04 ± 0.13	65.95	83.69	57.40
Distance map	64.55 ± 0.17	82.22 ± 0.34	55.79 ± 0.11	66.24	84.19	57.52
Distance prediction	63.32 ± 0.19	80.53 ± 0.32	54.81 ± 0.13	64.94	82.40	56.53
Orientation prediction	64.46 ± 0.16	82.13 ± 0.22	55.70 ± 0.13	66.06	83.94	57.44

We further compared GeoPacker to representative side‐chain modeling methods using the independent testset TS1888 (Table 2). GeoPacker outperformed the state‐of‐the‐art scoring‐function‐based methods, including Scwrl4 ²⁰ and FASPR, ²² and is comparable to DLPacker, ³¹ a representative deep learning‐based method, for side‐chain recovery of χ _1–2, and a little bit worse for χ ₃ or χ ₄ recovery. Figure 4 gives the performance for the 20 types of residues. GeoPacker works better for χ _1–2 of exposed charged or polar residues, particularly for the residue Arg and Lys, than all of the four dihedral angles are considered as χ ₃ and χ ₄ are more influenced by the flexible conformation of the long side chains. GeoPacker performs slightly worse than the other method for Met side chain recovery, which is mainly reflected in the prediction error of the terminal dihedral angle χ ₃. For Trp, GeoPacker gave similar recovery rate to Scwrl4, which is lower than those of FASPR and DLPacker, probably due to its low frequency in nature and the large continuous rotamer exploration sampling space in GeoPacker while other methods were all discrete sampling based. In addition, we also used another dataset with lower pair‐wise sequence identity, that is, 30%, directly from trRosetta, ³ to retrain our model. As shown in Table S2, side‐chain recovery rates of all the test methods were remarkably increased using an independent testset trRosettaTS1833 (see Appendix S1 for data split). GeoPacker outperforms DLPacker in recovery of both χ _1–2 and χ _1–4. The performance of GeoPacker on individual residue is also much improved (Figure S3), especially for the long charged residues including Glu, Arg, and Lys. This might come from the high quality of the trRosetta dataset as well as the early release of it (before May 2018). For a fair comparison, GeoPacker has been trained on the dataset and tested with other methods on CASP14 targets, totally including 34 protein chains. GeoPacker outperforms Scwrl4 and FASPR in a large margin, and is also comparable with DLPacker and OPUS‐rota4 (Table S3).

TABLE 2.

Performance evaluation of protein side‐chain packing modeling in an independent testset TS1888

Method	Recovery of χ 1–4 (%) (stringent tolerance within 20°)			Recovery of χ 1–2 (%) (stringent tolerance within 20°)	Runtime (s)
	All	Buried	Exposed	All
Scwrl4 20	58.47	76.18	49.79	62.77	2.82
FASPR 22	61.65	80.16	52.61	66.46	0.18
DLPacker 31	66.00	86.27	56.19	71.62	34.70
OPUS‐Rota4 32	69.89	89.77	60.27	76.46	2,206.12 a
GeoPacker	65.95	83.69	57.07	74.02	3.29

^a The corresponding job was performed by running 7 parallelizing jobs that lasted for 315.16 s. The total time cost for running a single job as other programs did was estimated by timing 315.16 with 7.

FIGURE 4.

Side‐chain dihedral angle recovery in a stringent tolerance within 20° for individual residue

Given the prospect of promising applications, especially for large‐scale de novo protein sequence design, both accuracy and running speed are important. To our best knowledge, FASPR is currently the fastest tool, though with a somewhat lower prediction accuracy. As the authors of DLPacker pointed out that it spent most of the running time to sample the rotamer with the most matched sidechain atomic density, and OPUS‐Rota4 suffers the same problem due to assembling the network architecture from DLPacker. Using an algorithm that directly generates the side‐chain dihedral angles, GeoPacker is about 10 times faster than DLPacker and 700 times faster than OPUS‐Rota4 with comparable prediction accuracy (Table 2). Due to the expensive time costs, only a part of tasks, including TS1888 and CASP14, are tested using OPUS‐Rota4.

3. DISCUSSION

Structures of protein side chains are important for biological functions. Despite the admirable efficiency, the performance of traditional modeling methods is limited by the inaccurate energy functions and low quality rotamer libraries. Deep learning‐based side‐chain modeling approaches are independent of rotamer libraries, thus are free from the limitations of discrete rotamer sampling. We have developed a geometric deep learning coupled ResNet algorithm to simultaneously predict three mutually supervised tasks. Despite only the predicted side‐chain dihedral angles are used for constructing the full atomic model and the accurate atomic interactions, the other tasks are helpful as constraints for further side‐chain conformational refinement by introducing a potential function, ⁴¹ where many literatures leverage the thought to improve their task performance. ³ , ³² For example, Xu et al. ³² improved around 2% by using the strategy for protein side‐chain packing modeling. On the other hand, our rational prediction procedure is autoregression. The residue conformation is coupled with other residues in the microenvironment. In order to speed up the process, we assumed the residue conformation is independent while the coarse‐grained rotamer‐free residues are coupled, which may produce inaccuracy that could be further improved in the future.

In addition, we focused more on the accurate prediction of buried residues, as solvent exposed residues are more flexible and contribute less to protein folding and structure stability. Proteins that could fold to stable structures form well‐defined hydrophobic cores with most of the buried polar atoms forming satisfactory hydrogen bonds. ⁴² Given the prospect of GeoPacker for de novo protein sequence design, we have tested 10 recently released structures (Table S4) with novel designed backbone topologies and de novo sequences for natural structure. ¹³ , ⁴³ , ⁴⁴ The overall performance of χ _1–4 recovery for all the tested methods decreased by 10% compared with that on natural proteins, yet the accuracy of the core region did not change significantly (Table 3). Thus GeoPacker is useful for large‐scale protein sequence design which is able to form accurate atomic interactions in the hydrophobic core with promising precision.

TABLE 3.

Protein side‐chain recovery on novel structures or new sequences

Method	Recovery of χ 1–4 (%) (stringent tolerance within 20°)
	All	Buried	Exposed
Scwrl4	49.05	76.69	37.76
FASPR	54.20	82.95	42.75
DLPacker	56.38	86.70	44.27
GeoPacker	58.46	86.35	47.34

As introducing perturbation was shown to be useful in flexible‐backbone protein design, ⁴⁵ we want to see how tolerant and generalizable GeoPacker to backbone perturbations. We evaluated the influence using 379 structures comprised five non‐native test sets with different perturbations, ²² , ⁴⁶ where the average RMSD (aRMSD) relative to the native backbone structure is 0.21, 0.57, 0.93, 1.48, and 1.88, respectively. All of the methods could tolerate perturbations below 1 Å (Figure S4).

We also test whether GeoPacker could be used in reconstructing side‐chain conformations at protein–protein interface. Protein–protein interactions are mainly mediated by hydrophobic interactions and hydrogen bonds. The variations to our adaptive model are the graphic representation of the protein complex, where the edges and their corresponding properties are added by the adjacent interface residues, and the distance map will be spanned to the complex, respectively. We tested the performance of GeoPacker using six de novo designed cases (Table S4) reported by two literatures. ⁴⁷ , ⁴⁸ The interface residue side‐chain recovery is 68.57%, indicating the transfer ability of our model, where the interface residues are defined by using distance within 5 Å between two heavy atoms of inter‐residues from different protein chains. To reach the state‐of‐the‐art performance for protein interface modeling problem, it will be necessary to apply further fine‐tuning procedure to the protein–protein interaction interface dataset by transferring our model.

4. CONCLUSION

In this study, we proposed a simplified geometric deep learning coupled ResNet approach, GeoPacker, for protein side‐chain structure prediction, where the inputs are only the geometric features from protein backbone and seven different reduced physicochemical properties for different residues. Our multitask learning framework, including the orientation prediction of C _β –C _γ ···C _γ –C _β , distance prediction of side‐chain distal pair‐wise heavy atoms, and side‐chain dihedral angles prediction, demonstrates that the mutually supervised objectives increase the accuracy and robustness for protein side‐chain modeling. GeoPacker outperforms the state‐of‐the‐art energy scoring function‐based methods in precision and runs about 10 times faster than the deep learning‐based method DLPacker with comparable performance. A series of testing, including de novo designed proteins and different backbone perturbations, further illustrate that GeoPacker is useful for protein structure modeling and de novo protein sequence and interface design, especially for large‐scale sequence side‐chain modeling with admirable efficiency and precision. GeoPacker is available in github at https://github.com/PKUliujl/GeoPacker.

5. MATERIALS AND METHODS

5.1. Data sets

We firstly culled the redundant proteins by PISCES ⁴⁹ with the following rules: (a) protein pair‐wise sequence identity should be <60%; (b) only X‐ray crystallographic structures with resolution <2.0 Å were selected; (c) R‐factor should be <0.25; (d) the protein chain length is longer than 40 but shorter than 1,000 residues. It should be noted that the side‐chain packing modeling of longer proteins is feasible with our pre‐trained model. The cutoff setting is limited to finite memory of machines. Finally, we created a data set with 14,888 proteins.

5.2. Input features

On the one hand, protein side‐chain packing on the whole attributes to the local spatial interactions. Evidently, the local features are of importance for determining the rotamers. Here several features are used: (a) backbone dihedral angle (φ, ψ) with its sine and cosine values; (b) 8‐state secondary structures defined by DSSP ⁴⁰ ; (c) the orientation and distance of pairwise backbone atoms from distinct residues in a cutoff threshold, including orientation {C,N,O,C _α }C _β ···C _β {C _α ,C,N,O} and their distance, correspondingly, where the C _β is built using commonly chemical bond lengths and angles (1.55°A for C _α –C _β bond length, 110.5° for plane angle of C–C _α –C _β and 122.55° for dihedral angle of N–C–C _α –C _β ). ⁵⁰ The cutoff is based on the distance of two central distinct C _α s. In other words, if the distance is larger than the cutoff, all of the orientation and distance will not be computed within the pair residues. (d) Seven reduced physicochemical properties for each residue ⁵¹ are used, including steric parameters, polarizability, volume, hydrophobicity, isoelectric point, helix probability and sheet probability. (e) A basis set of 19 rigid‐body blocks. ⁵² To construct our graph model, these features are embedded into node features and edge features.

On the other hand, the stability of a protein is determined by its overall structure. Hence the distance map as a global constraint, provides an insight into global influence. Significantly, this characteristic is different from the above constructed graph representation. In graphic model, the spatial neighboring residues have link with each other and solely those links include distance statistics whilst the distance map shows the distance of every residue pair.

Because of the relative pair‐wise information, our model is rotational and transformational invariance, which is important for protein tertiary structure reconstruction and corresponding properties.

5.3. Methods

In our GNN module, we need to construct the graph firstly. Each node represents a residue where its center is the C _α atom, and the edge is defined by the distance between C _α s from two residues. In the protein graph modeling, the rotamer of center residue is mostly affected by its neighbors, therefore a spatial‐based graph convolutional neural network is intuitively selected as our model framework by aggregating the information of the neighboring nodes. We set distinct cutoff values from 10 to 12 Å, and interestingly, instead of improving the performance, it increases higher runtime of computation for denser graph. Finally, we set 10 Å as the cutoff. We give a potential interpretation that the more confusing information may additionally merge into the graph to increase the discriminating difficulty. After defining the edges, the orientation and distance between each backbone pairwise atoms are delivered to the edge as extra information/relationship. The updates/iterations of node features and edge features are shown below:

$$x_i^{t+1}=w_i^1{x}_i^t+w_i^2{x}_i^2+\cdots +w_i^{\mathrm{t}}{x}_i^t+\sum _{j\in N_i}\mathrm{\Phi }\left(x_j^t,e_{\mathit{ij}}^t\right)w_j^t,$$ (5)

$$e_{\mathit{ij}}^{t+1}=\mathrm{\Psi }\left(e_{\mathit{ij}}^t,x_j^{t+1}-x_i^{t+1}\right)$$ (6)

where $x_i^{t+1}$ denotes the iteration t + 1 of node i, N _i is the set of neighbors of node i, w _i is a learnable parameters which reflects the importance of residues, Φ,Ψ are both learnable functions and e _ij is the edge. Here Φ,Ψ are linear with activation function elu. Significantly, we add the former weighted information of node $x_i$ to update node $x_i$ at time t + 1 which increases the capability of message passing. In addition, we concatenate the node and edge information so that its correlation is not lost during node update. We further add the informational differences which reflect the flow from node i to j in edge update.

In the ResNet and the self‐attention module, the pair nodes information from the outputs of GNN module will be filtered by 3 * 3 kernels in the ResNet module along with distance map. The number of convolutional layers is set to 20. In each layer, the inputs are from all previous layers' outputs. The activation is elu. Furthermore, each normalization is added before activation layer. After the ResNet module, we add a self‐attention module due to other residues contribute differently to the target residue. To this end, a linear layer and softmax function are used to the output of ResNet module as a weighted matrix, that is, self‐attention matrix, which is further reshaped and multiplied by the output of ResNet module. In order to robustly capture the information, we utilize 2‐head self‐attention, which is corresponding to the workflow depicted in Figure 1c). After concatenating the output of attention operation, a pooling layer with mean operation is used at the specified axis, which can be further to transfer the two dimensional information to the one dimensional for better merging the node information from graph model.

The connections between different modules are demonstrated as below. Any two nodes information from the outputs of GNN modules are concatenated as pairing information, and then this information is fed into ResNet module along with distance map. On the other side, the node outputs of GNN module will pass a three‐head attention indicated by the number of arrows, and then all of the outputs from different modules are concatenated in the node level. Finally, a MLP module is operated on these information to obtain the predicted results of side‐chain dihedral angles.

As each task is nearly equally important, the loss weights of multi tasks in this article are just set to make the initial loss of several random samples nearly equal without many efforts for tuning, which is shown in the following:

$$\text{Total loss}={\text{loss}}_1+{\text{loss}}_2/3.9+{\text{loss}}_3/4,$$ (7)

where the loss₁, loss₂, and loss₃ are for side‐chain dihedral angle prediction, distance prediction of distal heavy atoms, and orientation prediction, correspondingly.

Moreover, the dropout as an overfitting technique, is also employed in our model. We set a 30% drop rate in the final fully connected layer. The optimizer is ADAM ⁵³ with a learning rate of 0.001. Loss function is cross entropy.

AUTHOR CONTRIBUTIONS

Jiale Liu: Conceptualization (equal); investigation (lead); methodology (lead); software (lead); validation (equal); visualization (lead); writing – original draft (lead); writing – review and editing (equal). Changsheng Zhang: Formal analysis (equal); funding acquisition (equal); project administration (equal); supervision (equal); validation (equal); writing – review and editing (equal). Luhua Lai: Conceptualization (equal); formal analysis (equal); funding acquisition (lead); project administration (lead); supervision (lead); validation (equal); writing – review and editing (lead).

CONFLICT OF INTEREST

The authors declare no conflicts of interest.

Supporting information

Appendix S1 Supporting Information.

Liu J, Zhang C, Lai L. GeoPacker: A novel deep learning framework for protein side‐chain modeling. Protein Science. 2022;31(12):e4484. 10.1002/pro.4484

DATA AVAILABILITY STATEMENT

The data, source code and pre‐trained model are available in github at https://github.com/PKUliujl/GeoPacker.