Explainable Deep Hypergraph Learning Modeling the Peptide Secondary Structure Prediction

Abstract

Accurately predicting peptide secondary structures remains a challenging task due to the lack of discriminative information in short peptides. In this study, PHAT is proposed, a deep hypergraph learning framework for the prediction of peptide secondary structures and the exploration of downstream tasks. The framework includes a novel interpretable deep hypergraph multi‐head attention network that uses residue‐based reasoning for structure prediction. The algorithm can incorporate sequential semantic information from large‐scale biological corpus and structural semantic information from multi‐scale structural segmentation, leading to better accuracy and interpretability even with extremely short peptides. The interpretable models are able to highlight the reasoning of structural feature representations and the classification of secondary substructures. The importance of secondary structures in peptide tertiary structure reconstruction and downstream functional analysis is further demonstrated, highlighting the versatility of our models. To facilitate the use of the model, an online server is established which is accessible via http://inner.wei‐group.net/PHAT/. The work is expected to assist in the design of functional peptides and contribute to the advancement of structural biology research.

Accurately predicting peptide secondary structures remains a challenging task due to the lack of discriminative information in short peptides. Based on transfer learning and hypergraph algorithm, sequential semantic information can be incorporated from large‐scale biological corpus and structural. semantic information from multi‐scale structural segmentation, leading to better accuracy and interpretability even with extremely short peptides.

1. Introduction

Peptides have recently emerged as potential therapeutic molecules against various diseases, and have garnered increasing attention due to their many advantages, including high specificity, high penetration, low production cost, and ease of manufacturing and modification.^{[ 1 ]} Various disease‐specific functional peptides have entered the global market, including antiviral peptides (AVPs), antimicrobial peptides (AMPs), and anticancer peptides (ACPs).^{[ 2} , ³ , ^{4 ]} Specifically, a family of peptides known as cell‐penetrating peptides (CPPs) has shown enormous success in the cellular uptake of therapeutic molecules.^{[ 5 ]} Currently, over 40 cyclic peptide drugs are in clinical use, and approximately one new cyclic peptide drug is approved for clinical use each year on average.^{[ 6 ]} Furthermore, predicting the secondary structure of bioactive peptides can provide key insights into the functional mechanisms of peptides and could serve as a basis for designing peptides with desired functions.^{[ 1 ]} Predicting the secondary structure of peptides is an intermediate step in predicting 3D or tertiary structures, all of which are essential determinants of peptide bioactivity.^{[ 7 ]} Therefore, reliable and accurate computational methods for predicting the secondary structures of peptides are urgently needed.

Many efforts have been made to predict the secondary structure of proteins through computational approaches, most of which are based on machine learning algorithms. For instance, Heffernan et al. developed a multi‐task deep learning model^{[ 8 ]} in which a long‐ and short‐term memory bidirectional regression neural network (LSTM‐BRNNS) was constructed to capture both short‐term and long‐term residue interaction relationships.^{[ 9 ]} Li et al. developed the diffusion convolutional recurrent neural network (DCRNN), a hybrid neural network that alleviates the local features derived from convolutional neural networks (CNNs) and the global features captured from stacked bi‐directional gated recurrent units (BIGRU) to predict the secondary structures of proteins.^{[ 10 ]} Similarly, Busia et al. integrated CNN and residual connections to predict the secondary structures of peptides and achieved good performance, demonstrating the importance of the primary protein sequence information in secondary structure prediction.^{[ 11 ]} In addition to the above methods, there are many other protein secondary structure predictors, such as DeepCNF, JPRED, PROTEUS2, RaptorX, and MUfold‐SSW, among others.^{[ 12} , ¹³ , ¹⁴ , ¹⁵ , ¹⁶ , ^{17 ]} However, these methods are designed for the prediction of protein structures and are not applicable for secondary structure prediction due to the inherent structural differences between peptides and proteins. For example, evolutionary information is frequently integrated and used for model training in the prediction of protein secondary structures, and potential biases might be introduced when designing peptide secondary structure models due to the short length of peptides. Additionally, previous studies have demonstrated that even for identical segments of residues in proteins and peptides, their secondary structures might be quite different.^{[ 1 ]} One possible reason is that proteins have more complex tertiary structures, which presumably leads to changes in secondary structures. Particularly, hydrophobic collapse is a major force responsible for a well‐defined tertiary structure. However, this phenomenon is only applicable to proteins and not peptides.^{[ 18 ]} Therefore, developing a peptide‐specific secondary structure prediction method is urgently needed.

Singh et al.^{[ 1 ]} proposed PEP2D, the first peptide‐specific secondary structure predictor that trains a random forest (RF) model with peptide sequential and evolutionary data and achieves good performance. Recently, Cao et al.^{[ 19 ]} designed PSSP‐MVIRT (peptide secondary structure prediction based on multi‐view information, restriction, and transfer learning) for the prediction of peptide secondary structures, employing CNNs and BIGRU to learn high‐latent features and introducing transfer learning to overcome the lack of training data. In addition to the aforementioned methods, there are several other peptide structure prediction methods, such as PEP‐FOLD.^{[ 20 ]} However, existing methods have several limitations. Particularly, most of them rely heavily on feature engineering to design handcrafted features, the quality of which might greatly impact the predictive performance because the feature design is based on the researchers’ prior knowledge. Additionally, existing protein‐specific secondary structure prediction methods focus on long‐distance dependence of sequences with hundreds of residues rather than local fragments, whereas peptide‐specific methods focus more on neighborhood information among residues, thus easily ignoring global information. Ultimately, although deep learning has been successfully used in secondary structure prediction, the current methods still follow a “black box” model and lack good interpretability. These shortcomings limit our ability to predict the relationships between peptide primary sequences and their secondary structures.

In this study, we propose an innovative deep learning framework called PHAT (hypergraph learning and transfer learning‐based framework for peptide structure exploration) to predict peptide secondary structures and explore the related downstream functional analysis. Importantly, our PHAT incorporates several novel features: i) we introduce a powerful pre‐trained protein language model^{[ 21 ]} to transfer semantic knowledge from large‐scale proteins to peptides and learn high‐latent and long‐term features of peptide residues. ii) Considering the local continuity and diversity of peptide secondary structures,^{[ 22} , ^{23 ]} we equip PHAT with a novel HyperGMA (hypergraph multihead attention network), in which we can encode peptide residues with multi‐semantic secondary structural information while capturing contextual features from consecutive regions using multi‐level attention mechanisms. Additionally, our constructed hypergraph effectively prevents over‐smoothing, which is a common issue in conventional graph networks (e.g., GCN,^{[ 24 ]} GAT^{[ 25 ]}). iii) To reveal the predicting mechanisms of PHAT, the transition and emission matrices were visualized in conditional random fields (CRFs) that can automatically learn a set of biologically meaningful knowledge on secondary substructures. This overcomes the limitations of “black‐box” approaches in deep learning‐based models to some extent and provides good interpretability of our PHAT framework. iv) We also demonstrated that the structural predictions obtained from our model can assist in peptide‐related downstream tasks, such as the prediction of peptide toxicity,^{[ 26 ]} T‐cell receptor (TCR) interactions with MHC (major histocompatibility complex)–peptide complexes,^{[ 27 ]} and protein–peptide binding sites. v) A case study demonstrated that our PHAT can also accurately predict distance map and contact map matrices, which can be further used for the reconstruction of peptide 3‐D structures. Benchmarking results indicated that the proposed PHAT significantly outperforms the state‐of‐the‐art methods in either 3‐state or 8‐state secondary structure prediction, demonstrating the superiority and robustness of our model. To facilitate the use of our method, we established a code‐free, interactive, and nonprogrammatic web interface of PHAT at http://inner.wei‐group.net/PHAT/, which can lessen the programming burden for biological and biomedical researchers.

2. Experimental Section

2.1. Datasets

To evaluate the effectiveness of the model, the same benchmark dataset commonly used as a “gold standard” dataset in several studies is used.^{[ 19} , ^{28 ]} The dataset contains 5772 secondary structures of protein data with three structural states: helix (H), strand (E), and coil (C). The dataset processing process is illustrated in Figure 1A. Specifically, the protein structures are derived from X‐ray crystallography, and this process is executed with a resolution of at least 2.5 Å, with no chain breaks and less than five unknown amino acids. The sequence similarity in this dataset is reduced to 25% to ensure a fair performance evaluation. Additionally, there are some sequences containing the “X” symbol, representing unnatural residues in this dataset. Following the same data pre‐processing in,^{[ 19 ]} the unnatural sequences were removed including the “X” symbol, and 4542 protein and peptide sequences were retained. Afterward, among the remaining sequences, the sequences were selected with <100 residues lengths, finally yielding 1285 peptide sequences as the three‐structure‐state dataset. The length of the sequences ranged from 30 to 99 residues. Moreover, previous studies have demonstrated that the secondary structures of protein and peptides can also be defined with eight states, including H (alpha‐helix), G (3₁₀helix), I (π‐helix), E (extended beta‐strand), B (isolated beta‐strand), T (turns), S (bend), and others (C).^{[ 8} , ²⁹ , ^{30 ]} To account for this scenario, a new dataset of 1,060 peptide sequences was further constructed, derived from the DSSP (Dictionary of Protein Secondary Structure) structure database.^{[ 31 ]}

Figure 1.

The workflow and framework of PHAT. A) Dataset processing. We extracted the benchmark datasets from SCRATCH‐1D, where the protein and peptide structures were derived with X‐ray crystallography and operated with a resolution of at least 2.5 angstroms, for three‐state and eight‐state secondary structures. B) Architecture of deep learning network in PHAT. The architecture consists of three modules: (i) Knowledge transfer module, (ii) Hyper Graph embedding module, and (iii) Fusion and classification module. In Knowledge transfer module, the original sequences are encoded by a pretrained protein model to gain the features of peptide residues. In Hyper Graph embedding module, the peptide sequences are constructed into hypergraph structures and embedded by HyperGMA. In fusion and classification module, the outputs of the Knowledge transfer module and the Hyper Graph embedding module are firstly fused through the element‐wise multiplication and better integrated by the Bi‐LSTM. Then the output of Bi‐LSTM is inputted into the CRF layer, and as a result, the secondary structure of related residues can be predicted. C) Illustrates the details of Hyper Graph embedding module. In the part of graph extractor, peptide sequences are firstly sliced into fragments with specific length and constructed as hyperedges of the hypergraph structure. Then the hyperedges are cut into residue groups to be built as hypernodes in the hypergraph structure. Next, the hypergraph structure from graph extractor is inputted into HyperGMA to capture the multi‐scale relationships in view of residue groups and peptide fragments by the multi‐scale attention mechanism. D) Online service. Our web server of PHAT is freely available to provide researchers with peptide details in three‐state or eight‐state secondary structures, statistical analysis, and position analysis. The predictions of our model can be applied in many downstream tasks as in Downstream Analysis.

2.1.1. Training and Testing Datasets

To account for the characteristics of short peptide sequences and fairly evaluate the performance of the methods, the dataset was divided into two categories: >50 residue sequences and ≤50 residue sequences. The sequences with ≤50 residues consisting of 257 peptide sequences (with H of 5,294, E of 1,119, and C of 3,733) were used as the test set. The remaining 1,028 peptide sequences were used as the training dataset. For model training, 10% peptide sequences was randomly selected as the validation set from the training dataset to adjust the parameters of the model. Additionally, the training and testing datasets were labeled with the three‐state secondary structures, with the sequence length of peptides ranging from 30 to 100. For the eight‐structure‐state dataset,1060 sequences were also collected to re‐train and test the model. The details of the datasets are summarized in Tables S1 and S2 (Supporting Information).

2.2. Architecture of the Deep Learning Network of PHAT

The overall network architecture of the deep learning network of PHAT is illustrated in Figure 1B with three main modules: i) knowledge transfer module, ii) hypergraph embedding module, and iii) feature fusion and classification module. Specifically, the PHAT framework only takes peptide sequences as input. In module (i), to address the scarcity of peptides, the model employs and fine‐tuned and pretrained large‐scale protein language model called ProtT5 for the analysis of the peptide datasets. By doing so, rich contextual information can be transferred from large‐scale protein sequences to the model and learn discriminative feature embeddings of peptide sequences. In module (ii), a HyperGMA (hyper graph multi‐head attention network) is proposed to learn local and global features. Specifically, given a peptide sequence, the model first exploits the graph extractor to divide the peptide sequence into fragments with particular lengths as hyperedges and residue groups as hypernodes. Then, by using the hyperedges and hypernodes, the hypergraph structure is constructed and passed it to the HyperGMA to integrate the sequence information of different scales in the hypergraph structure. The model can capture both local and global features at the residue group level and peptide fragment level through the multiscale hypergraph attention mechanism. Afterward, in module (iii), the feature embeddings is integrated from the above two channels (knowledge transfer module and hypergraph embedding module) through an element‐wise multiplication strategy. Furthermore, the model adopts Bi‐LSTM (bidirectional long short‐term memory networks)^{[ 32 ]} to improve and optimize the feature representation ability and exploits CRFs to learn useful correlations among the sub‐secondary structures. Finally, PHAT takes the resulting features from module (iii) as the input of a Viterbi algorithm and predicts the structural state to which each peptide residue belongs.

2.2.1. Feature Embedding from the Pretrained Model ProtT5

Although there are some differences between proteins and peptides in terms of structure, they are similar in many aspects such as the transcription process and residue sequence composition. Therefore, the pre‐trained model ProtT5 was used based on the t5‐3b model,^{[ 33 ]} which was pre‐trained using the UniRef50 database^{[ 34 ]} (i.e., a database consisting of 45 million protein sequences), in a self‐supervised manner to transfer semantic knowledge from proteins to peptides. Its weight was pretrained with a BERT‐like mask language model denoising objective using raw protein sequences without labeling. The model can fully learn the semantic information and generate different residue features belonging to multiple expressions in different context scenarios.

The original peptide sequences are fed into ProtT5, and the output vectors are extracted from many encoder blocks that are dependent on the self‐attention mechanism. Each encoder block calculates the attention for each residue with all residues in the sequence, aiming to obtain the relevance and importance between every two residues. The calculation formula of the self‐attention mechanism is as follows:

$$\mathrm{Attention}\left(Q,K,V\right)=\mathrm{softmax}\left(\frac{Q{K}^T}{\sqrt{{\mathrm{d}}_{\mathrm{Key}}}}\right)V$$ (1)

where Q, K, and V are the query vector, key vector, and value vector of the corresponding individual residues in the peptide sequence, respectively, and d _Key is the dimension of the input key vector.

2.2.2. Hypergraph Multihead Attention Networks

Inspired by the previous studies for hypergraphs in natural language processing,^{[ 35 ]} a hypergraph structure was constructed by taking the peptide residue groups as nodes and the peptide fragments as edges. Based on this structure, a novel HyperGMA was proposed. Figure 1C shows the hypergraph construction process and HyperGMA architecture. i) The peptide sequence was inputted into the graph extractor, which takes a particular length as the sliding window size and moves the sliding window to select the sequence fragments with cross residues. ii) The sequence fragment is divided into smaller residue groups in a similar way as in step i) but with a smaller sliding window size. The residue groups are regarded as hypernodes and the peptide fragments are taken as the hyperedges. iii) The structure of the hypergraph is constructed using the hyperedges and hypernodes generated from steps (i) and (ii). iv) Then, the hypergraph structure is inputted into HyperGMA to extract the graph embeddings of the peptide sequence.

The context of residues in a peptide sequence describes the language characteristics of local co‐occurrence among residues, and its function in sequence representation learning has also been proved to be effective. In the model, two residues were established as a group, based on which 400 types of groups were identified. Moreover, a set of residue groups is regarded as a hyperedge, which is a sequence fragment with a specific length. This enables the model to simultaneously capture structural information both at the residue group level and peptide fragment level. Specifically, a hypergraph is defined as G = (ν, ε), where ν = {v ₁, v ₂, …, v_n } represents a set of n nodes in the graph, and ε = {e ₁, e ₂, …, e_m } represents a set of m hyperedges. Moreover, the model can connect two or more nodes for any hyperedge e_j .

2.2.3. Residue Group‐Level Multihead Attention

$${\left(f_j^l\right)}_m=\sigma \left(\sum _{v_k\in e_j}{\alpha }_{jk}{W}_m{h}_k^{l-1}\right)$$ (2)

where k represents the index of the residue group (hypernode) in the fragment (hyperedge) e_j , j indicates the index of the fragment in edge set ε, v_k ∈ e_j indicates that v_k is contained in fragment e_j , $h_k^l$ is the representation of residue group (hypernode) v_k at layer l, σ is the activation function LeakyReLU, W _m is the weight matrix trained in the m‐head attention, and m represents the head number of multi heads. α _jk is the attention coefficient of the residue group v_k in the fragment e_j , which can be computed by:

$${\alpha }_{jk}=\frac{\exp \left(a_m^T{u}_k\right)}{{\sum }_{v_p\in e_j}\exp \left(a_m^T{u}_p\right)}$$ (3)

where $a_m^T$ is a weight vector for measuring the importance of residue groups in the m‐head attention, v_p ∈ e_j represents that residue group v_p is contained in fragment e_j , and T means “transpose.” u_k represents v_k on the hypergraph defined as:

$$u_k=\mathrm{LeakyReLU}\left(W_m{h}_k^{l-1}\right)$$ (4)

The expression ${(f_j^l)}_m$ represents hyperedge e_j from m‐head attention at layer l. The multi‐head attention mechanism was constructed, connected it, and compressed it to the desired dimension after the layer was fully connected. This structure is aimed to capture residue context information. The output $f_j^l$ represents the connected representation of hyperedge e_j at layer l.

2.2.4. Peptide Fragment‐Level Attention

With the representations of all peptide fragments (hyperedges) as $\{f_j^l|\forall e_j\in {\epsilon }_i\}$ connecting to residue group v_i , the fragment level attention mechanism was introduced to capture the structural information of peptide fragments with distance interval for learning the next‐layer representation of residue group v_i , which is expressed as follows:

$$h_i^l=\sigma \left(\sum _{e_j\in {\epsilon }_i}{\beta }_{ij}{W}_{\mathrm{fragment}}{f}_j^l\right)$$ (5)

where $h_i^l$ is the output representation of residue group (hypernode) v_i (v_i ∈ ν) at layer l, i represents the index of the residue group (hypernode) in the node set ν, all the hyperedges containing residue group v_i are in ε _i , and W _fragment is a weight matrix. e_j is a fragment (hyperedge) divided at a fixed length from peptide sequence, and ε _i is the set of fragments of the peptide.

β _ij shows the attention interaction of peptide fragment (hyperedge) e_j on residue group (hypernode) v_i . The computing process is described below:

$${\beta }_{ij}=\frac{\exp \left(a_{\mathrm{fragment}}^T{V}_j\right)}{{\sum }_{e_p\in {\epsilon }_i}\exp \left(a_{\mathrm{fragment}}^T{V}_p\right)}$$ (6)

where $a_{\mathrm{fragment}}^T$ is a weight vector similar to $a_{\mathrm{m}}^T$ but for measuring the importance of peptide fragments

$$W_{\mathrm{residue}}={(||}_{i=1}^m{W}_{\mathrm{i}})·W_{\mathrm{d}}$$ (7)

$$V_j=\mathrm{LeakyReLU}([W_{\mathrm{fragment}}{f}_j^l||W_{\mathrm{residue}}{h}_i^{l-1}])$$ (8)

|| represents the concatenation operation, · is matrix multiplication, and W _d is a trainable matrix for dimensional reduction.

2.2.5. Bidirectional Long Short‐Term Memory and Conditional Random Field

The secondary structural information in peptide sequences is often related to the residues in the forward and backward peptide fragments. Therefore, Bi‐LSTM (bidirectional long short‐term memory networks) to extract information from two directions in the peptide sequence. Additionally, the previously learned features from ProtT5 and HyperGMA are fused in the form of element‐wise multiplication, which may introduce redundant information. Therefore, a layer of Bi‐LSTM was added to better integrate them and provide a sequence‐level view for the CRF layer. Bi‐LSTM is a deep‐learning architecture with two LSTM layers in different directions, which can capture the dependence of long‐distance residues, and selectively learn and forget information with corresponding importance through training.^{[ 36 ]} Moreover, LSTM has three gate structures (inputting gate, forgetting gate, and outputting gate) and a Cell State's hiding state. In LSTM, the inputting gate is responsible for processing the input of the current sequence position, whereas the forgetting gate controls whether the hidden cell state of the upper layer must be forgotten based on probability. The results of the forgetting gate and inputting gate will act on the cell state. Then, information from the previous sequence, the current sequence, and the cell state will be combined with the activation function and weights to obtain the output. Therefore, the model can better capture semantic information of peptide sequences and the prediction can more accurately select Bi‐LSTM, as shown in Figure S1 (Supporting Information).

To the best of the knowledge, the model is the first to determine the probability of each residue belonging to specific secondary structures by adding a linear layer with the softmax function behind the Bi‐LSTM, after which the label with the highest probability can be obtained. However, this will ignore the correlation among secondary structures and decrease the prediction performance. Alternatively, the CRF approach was chosen, which is widely used in named entity recognition to predict secondary structures, while exploring the context‐related interactions between secondary structures and residue‐level contributions to all secondary structures.

CRFs consist of emission matrices including the probability of residues occupying different secondary structure states and transition matrices including the likelihood of transferring one secondary substructure state to another. During the training process, the model uses the forward and backward algorithms to infer the conditional probability of the secondary structures at each position of the sequence and finally predict the secondary structure by the scoring matrices. The specific calculation process is described below.

There are two kinds of feature functions. The first is referred to as the emission function, which is only related to the current position i in the peptide sequence:

$$e_l\left(y_i,x,i\right)l=1,2,\text{…},L$$ (9)

where x represents all residues of the peptide, y_i represents the secondary structure at position i, and L indicates the number of all secondary structures.

The second function is defined in the context of secondary structures and is referred to as the transition function, which is related to the current structure y_i and the previous structure y _{i − 1}:

$$t_k\left(y_{i-1},y_i,x,i\right)k=1,2,\text{…},K$$ (10)

where K indicates the number of all permutations of two secondary structure states, which is 9 for 3‐state secondary structures and 64 for 8‐state secondary structures.

Assuming thatK ₁ transition functions and K ₂ emission functions, there are a total of K ₁ + K ₂ feature functions. The formula f_k (y _{i − 1}, y_i ,x, i) was then used to express them:

$$f_k\left(y_{i-1},y_i,x,i\right)=\left\{\begin{array}{c}{t}_k\left(y_{i-1},y_i,x,i\right)k=1,2,\text{…}{K}_1\hfill \\ e_l\left(y_i,x,i\right)k=K_1+l,l=1,2,\text{…},K_2\hfill \end{array}\right.$$ (11)

The weight coefficient f_k (y _{i − 1}, y_i ,x, i) was also unified with w_k:

$$w_k=\left\{\begin{array}{c}{\lambda }_k,k=1,2,\text{…}{K}_1\hfill \\ {\mu }_l,k=K_1+l,l=1,2,\text{…},K_2\hfill \end{array}\right.$$ (12)

where λ _k represents the weight coefficient of the k‐th transition function and µ _l represents the weight l‐th coefficient of the emission function.

The parametric form is simplified as:

$$P\left(y|x\right)=\frac{1}{Z\left(x\right)}\exp \sum _{k=1}^{K_1+K_2}{w}_k{f}_k\left(y,x\right)$$ (13)

Z(x) is the normalization factor:

$$Z\left(x\right)=\sum _y\exp \sum _{k=1}^{K_1+K_2}{w}_k{f}_k\left(y,x\right)$$ (14)

In the traditional CRF, it is found that the only global transition matrix is easily affected by the noise from datasets, resulting in unstable prediction results. To solve this problem, the outputs from Bi‐LSTM into linear layers are first arranged, transferring the outputs to local transition matrices with the same dimension as the global transition matrix. Then, the local transition matrices were connected to the global transition matrix, as using the fused transition matrices improved the ability of the model to assess different datasets. The details of the CRF architecture are shown in Figure S2 (Supporting Information).

2.3. Model Training and Predicting Process

2.3.1. Training Process

The Bi‐LSTM‐CRF layer was introduced to fuse features and predict the secondary structure of peptides. In Bi‐LSTM‐CRF, the secondary structure label paths are constructed with the emission and transition matrices. The loss function of the model consists of two parts, the score of the real label path and the total score of all paths, with different secondary structure label combinations. The score of the real path should be the highest in all paths and the goal of the optimization is to minimize the gap between the predicted score and the real score.

If a certain path is a real path and the secondary structure label sequence is the correct prediction result, then it should have the highest score of all possible paths. According to the following loss function, the parameters of the model will be updated continuously with every iteration of the training process, making the ratio of the score of the real path to the total score larger.

$$\mathrm{Loss}=-\log \left(\frac{S_{\mathrm{real}\mathrm{path}}}{S_{\mathrm{total}}}\right)$$ (15)

Assuming that the score of each possible path is S_i , and there are n paths in total, then the total score of all paths is (where e is Euler number):

$$S_{\mathrm{total}}=e^{S_1}+e^{S_2}+···+e^{S_n}$$ (16)

Next, the composition of S_k can be expressed as follows:

$$S_k=\mathrm{EmissionScore}+\mathrm{TransitionScore}$$ (17)

$$\mathrm{EmissionScore}=e_{1\left(x_1\to y_1\right)}+e_{2\left(x_2\to y_2\right)}+···+e_{n\left(x_n\to y_n\right)}$$ (18)

The $e_{i(x_i\to y_i)}$ is the score function resulting in a probability to predict the current residue x_i as the secondary structure y_i .

$$\mathrm{TransitionScore}=t_{\left(\mathrm{start}\to y_1\right)}+t_{\left(y_1\to y_2\right)}+···+t_{\left(y_n\to \mathrm{end}\right)}$$ (19)

where $t_{\mathrm{i}(y_i\to y_j)}$ is the score function in support of generating the probability of transferring the secondary structure y_i to y_j .

2.3.2. Prediction Process

In the prediction process, the Viterbi algorithm^{[ 37 ]} is used to obtain the secondary structure prediction. The Viterbi algorithm is a dynamic programming algorithm that uses the start and end states and the recurrence formula to gain the secondary structure labels. The input of the Viterbi algorithm consists of K feature functions of the model, K weights related to the functions, the observation peptide sequence x = (x ₁,x ₂,…, x_n ), and the number of secondary structure states m. The output of this calculation is the optimal prediction secondary structure label sequence $y^{\ast }=(y_1^{\ast },y_2^{\ast },\text{…},y_n^{\ast })$. The details of the prediction process of the Viterbi algorithm are described below.

First, the start recursive algorithm is initialized as:

$${\delta }_1\left(l\right)=\sum _{k=1}^K{w}_k{f}_k\left(y_0=\mathrm{start},y_1=l,x,i\right),l=1,2,\text{…},L$$ (20)

$${\psi }_1\left(l\right)=\mathrm{start},l=1,2,\text{…},L$$ (21)

where L is the number of secondary structure labels.

For i = 1, 2, …, n − 1, the recursion formula is performed as follows:

$$\begin{array}{ccc}\hfill {\delta }_{i+1}\left(l\right)& =& \underset{1\le j\le L}{\max }\left\{{\delta }_i\left(j\right)+\sum _{k=1}^K{w}_k{f}_k\left(y_i=j,y_{i+1}=l,x,i\right)\right\},\hfill \\ & & l=1,2,\text{…},L\hfill \end{array}$$ (22)

$$\begin{array}{ccc}\hfill {\psi }_{i+1}\left(l\right)& =& \arg \underset{1\le j\le L}{\max }\left\{{\delta }_i\left(j\right)+\sum _{k=1}^K{w}_k{f}_k\left(y_i=j,y_{i+1}=l,x,i\right)\right\},\hfill \\ & & l=1,2,\text{…},L\hfill \end{array}$$ (23)

When the following condition occurs, program recursion is stopped:

$$y_n^{\ast }=\arg \underset{1\le j\le L}{\max }{\delta }_n\left(j\right)$$ (24)

Through the backtracking algorithm, the final prediction structure is obtained:

$$y_i^{\ast }={\psi }_{i+1}\left(y_{i+1}^{\ast }\right),i=n-1,n-2,\text{…},1$$ (25)

In the end, the prediction is:

$$y^{\ast }=\left(y_1^{\ast },y_2^{\ast },\text{…},y_n^{\ast }\right)$$ (26)

2.3.3. Performance Metrics

The performance of our proposed PHAT is evaluated by the accuracy and SOV (segment overlap measure) for each secondary structure state. Acc_i, F1‐score_i {i represents the secondary structure element [H(Helix), E(Sheet) or C(Coil) for 3‐state and H(alpha‐helix), G(3₁₀helix), I(π‐helix), E(extended beta‐strand), B(isolated beta‐strand), T (turns), S (bend) and others (C) for 8‐state]}, the accuracy in all states (hereinafter referred to as Acc), and SOV are calculated as follows:

$${\mathrm{Acc}}_i=\frac{A_{ii}}{A_i}$$ (27)

$$\mathrm{Acc}=\sum {\alpha }_i\frac{{\sum }_{i\in \left\{\mathrm{structure}\mathrm{element}\right\}}{A}_{ii}}{{\sum }_{i\in \left\{\mathrm{structure}\mathrm{element}\right\}}{A}_i}$$ (28)

$$F1-{\mathrm{score}}_i=\frac{2P_i{R}_i}{P_i+R_i}$$ (29)

$$\mathrm{Macro}-F1=\frac{{\sum }_{i\in \left\{\mathrm{structure}\mathrm{element}\right\}}F1-{\mathrm{score}}_i}{n}$$ (30)

$$\mathrm{SOV}=\frac{{\sum }_{i\in \left\{\mathrm{structure}\mathrm{element}\right\}}{\sum }_{si}\frac{\min \mathrm{ov}\left(s1,s2\right)+\delta \left(s1,s2\right)}{\max \mathrm{ov}\left(s1,s2\right)}·\mathrm{len}\left(s1\right)}{N}$$ (31)

where A_i is the sum of correctly predicted residues in each state; A_ii is the number of correctly predicted residues in state i; α _i is the proportion of state i in the entire test set; P_i indicates the proportion of residues correctly predicted to be i among those predicted to be i; R_i is the proportion of residues correctly predicted to be i among residues with the actual i; s1 and s2 are segments corresponding to actual and predicted secondary structures; len(s1) represents the number of residues defining the segment s1; max ov(s1, s2) is the maximum length overlap of s1 and s2 for which either of the segments has a residue in state i; min ov(s1, s2) represents the length overlapping s1 segments and s2 segments. δ(s1, s2) is calculated as follows:

$$\delta \left(s1,s2\right)=\min \left\{\begin{array}{c}\begin{array}{c}\left(\max \mathrm{ov}\left(s1,s2\right)-\min \mathrm{ov}\left(s1,s2\right)\right)\hfill \\ \left(\min ov\left(s1,s2\right)\right)\hfill \end{array}\hfill \\ \begin{array}{c}\left(\frac{\mathrm{int}\left(\mathrm{len}\left(s1\right)\right)}{2}\right)\hfill \\ \left(\frac{\mathrm{int}\left(\mathrm{len}\left(s2\right)\right)}{2}\right)\hfill \end{array}\hfill \end{array}\right.$$ (32)

The normalization value N is a sum of N(i) over the entire set of conformational states:

$$N=\sum _{i\in \left\{\mathrm{structure}\mathrm{element}\right\}}N\left(i\right)$$ (33)

SOV was introduced because the segment overlap measure treats H, E, and C on an equal basis (eight‐state assignment is the same). There are no arbitrary cutoffs on segment length, thus ensuring a consecutive and threshold‐free assessment of prediction accuracy.

3. Results

3.1. PHAT Outperforms Existing Methods When Analyzing an Independent Testing Set

To evaluate the performance of the proposed PHAT in peptide secondary structure prediction, we compared it with four state‐of‐the‐art methods: PROTEUS2,^{[ 14 ]} RaptorX,^{[ 16 ]} Jpred,^{[ 12 ]} and PSSP‐MVIRT.^{[ 19 ]} The first three were designed for protein secondary structure prediction whereas the other is for peptide secondary structure prediction. To ensure a fair comparison, the models were executed and evaluated using the same independent test set. As shown in Table S3 (Supporting Information, PHAT achieved the best performance among all of the tested methods, with an Acc of 84.07%, Acc_H of 89.08%, Acc_E of 71.76%, Acc_C of 80.66%, and SOV of 79.78%. Specifically, compared to other existing methods, our method delivered 1.39% to 19.16% higher SOV values (see Figure 2A and Table S3, Supporting Information), which is an important metric at the segment level and evaluates the overall performance of the methods. The superior SOV performance of our proposed model might be related to the context information of the peptide sequences extracted by the Bi‐LSTM‐CRF layer and multi‐scale features captured by the hypergraph multi‐head attention network. Furthermore, all methods exhibited a relatively low accuracy in the prediction of the structural state E compared to the other two states (H and C). This was due to the low proportion of E in the dataset (see Figure 2B and Table S1, Supporting Information). Therefore, the existing models capture more information for the H and C states, rather than E, during model training. Nevertheless, our PHAT achieved the highest accuracy at E among all of the evaluated methods. This was likely because our multi‐head attention mechanism is capable of capturing a more informative structural representation of E. Additionally, the comparison results in the dataset of the eight‐state secondary structure shown in Table S10 (Supporting Information) also demonstrate the outstanding performance of our method. Therefore, we concluded that our method is more effective than Jpred, PSSP‐MVIRT, PROTEUS2, and RaptorX in the prediction of peptide secondary structures, especially for Acc_E, Acc, and SOV. We also input the testing set into Alphafold^{[ 38 ]} and trRosetta,^{[ 39 ]} which are the most popular software for protein structure prediction, and extract secondary structure from their predicted PDB files using the DSSP algorithm. Interestingly, our method achieves similar or even better performance compared with them while using less computational and time costs (see Table S4, Supporting Information).

Figure 2.

The performances of our method and existing methods on independent test subsets, comparison of different encoding strategies, and visualization of different methods on one peptide: A) SOV, macro‐F1, and F1‐score_H are used as the evaluation metrics; B) F1‐scores under three sub‐structures are used as the evaluation metrics. C) SOV of four methods at the different length intervals. D–G) represent PCA visualization results of individual features of ProtT5, HyperGMA, and the fusion features in multiplication or additive respectively; H–K) represent the comparison between multiplication fusion strategy and other three strategies. I) represents performance comparison between HyperGMA and TextCNN. L) The visualization of predictions by our method and other two methods for the peptide with PDB ID: 2w25.

3.2. Length Preference Investigation for Peptide Secondary Structure Prediction

Previous studies have demonstrated that the functionality of peptides (e.g., affinity) is easily affected by the length of sequences, with most bioactive peptides being normally less than 40 residues long.^{[ 19} , ⁴⁰ , ^{41 ]} To investigate if our model had length biases for peptide secondary structure prediction, we further explored whether peptide length affected the performance of our model. We first divided the test set into four subsets with different length intervals (^{[ 30} , ^{35 ]}, ^{[ 35} , ^{40 ]}, ^{[ 40} , ^{45 ]}, and ^{[ 45} , ^{50 ]}), then separately evaluated our model and existing methods using the subsets. Figure 2C and Figure S3 (Supporting Information) show the SOV, Acc, and F1‐score of the different methods for the prediction of peptide secondary structures using the aforementioned subsets. As illustrated in Figure S3 (Supporting Information), the performance of all of the tested methods clearly decreased as the length of the sequences declined, which indicates that shorter sequences are more difficult to predict as their contextual information is less. Furthermore, as illustrated in Figure 2C, the SOV score of our method was higher than that of the other methods in almost all ranges of peptide sequence lengths. Particularly, our PHAT exhibited an outstanding performance, with average Acc, SOV, and F1‐score values up to 7.02%, 6.21%, and 3.33% higher than the runner‐up PSSP‐MVIRT in different sequence length intervals. These results demonstrate that our method is better at the prediction of shorter peptides.

3.3. Exploration of the Optimal Architecture of Our Model

To investigate the performances of our model using different encoding strategies, we compared the prediction results of different encoding strategies including the two individual feature extractors (HyperGMA and ProtT5) and their different fusion combinations. Table S5 (Supporting Information) shows that our final element‐wise multiplication strategy achieves an Acc of 84.07%, Acc_H of 89.08%, Acc_E of 71.76%, Acc_C of 80.66%, and SOV of 79.78%, outperforming the Acc and SOV of ProtT5 by 1.77% and 5.79% and the fused extractor in the additive strategy by 1.36% and 5.64%, respectively. Furthermore, although ProtT5 performed better than HyperGMA, the model performed better than the individual extractors and the fused extractor in the additive strategy after fusing the features from HyperGMA and ProtT5 with the element‐wise multiplication fusion strategy. This indicated that the different information is complementary to each other in the fusion strategy, thus effectively improving the predictive performance of the model. Moreover, it can be seen from Figure 2H–K that the element‐wise multiplication fusion strategy of HyperGMA and ProtT5 achieved better performance than the fusion strategies of TextCNN and ProtT5 in terms of Acc and SOV.

To further illustrate the effect of different encoding strategies more intuitively, we visualized the distribution of feature representations in the test set, which reveals the discriminability of features for distinguishing different secondary sub‐structure states through dimension reduction. In the principal component analysis (PCA)^{[ 42 ]} in Figure 2D–G, each point represents a site in the peptide sequence and different colors are used to distinguish the helix (H), strand (E), and coil (C) secondary structures. Compared with the two fusion strategies, the distribution of the site samples belonging to different classes in the feature space from the individual ProtT5 and HyperGMA are almost connected, making it difficult to distinguish the region for each secondary sub‐structure class. Regarding the two fusion strategies, the site samples of three classes are more clearly distributed in the feature space of the multiplication fusion strategy (Figure 2F) than in the feature space of the additive fusion strategy (Figure 2G). Furthermore, to avoid biases between different dimension reduction methods, we also applied another non‐linear method T‐SNE^{[ 43 ]} for dimension reduction, and similar results can be seen in Figure S3 (Supporting Information). In conclusion, our results demonstrate that our PHAT with the multiplication fusion strategy can capture more discriminative and high‐quality features.

3.4. The PHAT Has Good Interpretability in Terms of Extracting Multiscale Features and Making Classifications

To verify the effect of the Bi‐LSTM‐CRF layer in our model, we compared the performance of our model under two training strategies (Cross Entropy loss function and Bi‐LSTM‐CRF), and the results are shown in Table S6 (Supporting Information). Clearly, our model with Bi‐LSTM‐CRF layer performed better (especially in terms of SOV) than the model using the Cross‐Entropy loss function. To explain how the Bi‐LSTM‐CRF efficiently predicts the secondary structure at each site in the peptide sequence, we randomly selected and predicted the secondary structures of the peptide sequence with PDB ID 1edm chain B (Protein Data Bank Identity). Afterward, we chose several sites of this peptide and visualized the corresponding weights of the transition matrix and emission matrices from our model in Figure 3A. As illustrated in Figure 3A, the secondary structure labels corresponding to the highest values in the emission matrices match the real secondary structures of the residues. Moreover, the probability of transferring the labels of the current residues to the real labels of the adjacent residues was the highest in the transition matrices.

Figure 3.

The interpretability of our model. A) Visualization of the weights of transition matrix and emission matrix in Bi‐LSTM‐CRF layer. The emission matrix and transition matrix are calculated by our model. The emission matrix shows the possibilities of current site in different classes and the transition matrix indicates the possibility of the secondary structure transformation in adjacent positions. B,C) Visualization of the attention matrices in hypergraph multi‐head attention network, where B represents the attention of peptide fragments to residue groups and C represents the attention of residue groups to peptide fragments. Darker color means stronger attention.

To further explore the role HyperGMA of in our model, we visualized and analyzed the attention matrices from HyperGMA in Figure 3. The HyperGMA includes two main steps, the residue group level attention encoding and the peptide fragment level attention encoding. In the first step, the feature representations of peptide fragments are aggregated from the contained residue groups through the multi‐head attention mechanism at residue group level. The contribution of each residue group to corresponding peptide fragments is shown in Figure 3B. Moreover, Figure 3C illustrates that the peptide fragments are more likely to reflect the characteristics of specific residue groups, meaning that the peptide fragments are more strongly influenced by local information. In the second step, the feature representation of the residue group is encoded by the peptide fragments where it exists through the fragment level attention mechanism. The contribution of the peptide fragment to corresponding residue groups is shown in Figure 3C, which indicates that a given residue group can aggregate the information from different fragments where it exists. Therefore, our model can better capture the local and global information by collecting secondary structure information at the residue group level and peptide fragment level using HyperGMA.

3.5. Application of Our PHAT in Three Peptide‐Related Downstream Tasks

Several experiments were conducted to verify that the secondary structures predicted by our method can be useful for downstream tasks. Figure 4A–C shows the results of prediction of peptide toxicity, prediction of T‐cell receptor interactions with MHC‐peptide complexes, and prediction of protein‐peptide binding sites, respectively. In Figure 4, it can be seen that when fused with the structure predictions of our PHAT model, the evaluated methods (ATSE, NetTCR‐2.0, and PepBCL) achieve higher performance in terms of most metrics than without the PHAT predictions. Similar results were observed with the methods fused with structure predictions from PROTEUS2 and PSSP‐MVIRT in the corresponding task.

Figure 4.

Comparative results for three downstream tasks. A) The results on the task of prediction of peptide toxicity. B) The results on the task of prediction of T‐cell receptor interactions with MHC‐peptide complexes. C) The results on the task of prediction of protein‐peptide binding sites. D) ROC curve and Precision‐Recall cure of comparison experiment in ATSE. E,F) Density of positive and negative examples under different confidence in prediction of peptide toxicity. G) ROC curve and Precision‐Recall cure of comparison experiment in NetTCR‐2.0. H, I) Density of positive and negative examples under different confidence in prediction of TCR interactions with MHC‐peptide complexes.

3.5.1. PHAT Has an Outstanding Performance for Aiding in Predicting Peptide Toxicity

We first used the methods (PSSP‐MVIRT, PROTEUS2, and PHAT) to predict the secondary structures of the dataset in ATSE,^{[ 26 ]} a peptide toxicity predictor, and add the secondary structures from the three methods to ATSE. As shown in Figure 4D and Table S7 (Supporting Information), ATSE with our PHAT model achieved an SN of 95.06%, SP of 93.4%, Acc of 94.74%, MCC of 89.62%, AUC of 96.7% (the definition of these metrics can be found in Supplementary metrics), which constituted a 0.17%, 0.18%, 0.43%, 0.5%, and 1.1% higher performance than ATSE with PROTEUS2, and a 0.25%, 0.37%, 0.88%, 1.87%, and 0.8% higher performance than ATSE with PSSP‐MVIRT, respectively. Additionally, Figure 4E,F shows PHAT had an outstanding performance for the prediction and classification of ATSE, and there was also a general improvement over the original method. These results demonstrate the efficiency of our model to predict secondary structures to assist in peptide toxicity prediction. Particularly, the higher SOV of our method reveals that our model can more accurately capture the integrity and continuity of secondary structures, which may explain the superior performance of our method.

Secondary structure is an important determinant of toxicity.^{[ 44 ]} However, few studies have used the secondary structure of peptides to predict peptide toxicity. Predicting the secondary structures of peptides by various methods can compensate for these limitations and build a bridge between peptide secondary structure and peptide toxicity.

3.5.2. PHAT Achieves Superior Performance for the Prediction of T‐Cell Receptor Interactions with MHC–peptide Complexes

Our prediction of the secondary structure of peptides can also be applied to the study of T‐cell receptor interactions with MHC‐peptide complexes. Here, we used the NetTCR‐2.0 method,^{[ 27 ]} which has a CNN architecture, to predict the interactions between the α/β TCR and MHC–peptide sequences and assess the effect of adding secondary structures predicted from the three methods (PSSP‐MVIRT, PROTEUS2, and our PHAT). As indicated in Figure 4G and Table S8 (Supporting Information), analysis of the NetTCR‐2.0 dataset with PHAT achieved an average Acc of 94.04%, a precision of 45.54%, a recall of 78.6%, an F1‐score of 57.29%, and an AUC of 92.7%, which was higher than the original method by 0.61%, 3.51%, 2.61%, and 2.4%, respectively. Furthermore, our model outperformed the Acc, Precision, F1‐score, and AUC of PSSP‐MVIRT by 0.38%, 1.61%, 1.28%, and 1%, as well as PROTEUS2 by 0.59%, 2.29%, 1.82%, and 1.3%, respectively. Moreover, Figure 4H,I shows that PHAT achieved a better prediction of NetTCR‐2.0 classification.

Additionally, we found that two groups of α/β TCR sequences, which have similar sequences but different secondary structures, cannot be classified correctly using NetTCR‐2.0 without adding secondary structures. Fortunately, they were accurately predicted after introducing the secondary structure features from our PHAT. In Figure S4 (Supporting Information), we visualized the secondary structures of the two peptide sequences predicted by our method. Therefore, our findings demonstrated that the secondary structures predicted by our method provide useful biochemical information and improve the performance of NetTCR‐2.0. In conclusion, the above results can prove that our prediction of peptide secondary structures has a positive effect on promoting the accuracy of TCR tasks and provide a new direction for TCR research.

3.5.3. PHAT Exhibited Competitive Performance for Assisting in the Prediction of Protein–Peptide Binding Sites

Protein–peptide interactions are involved in various fundamental cellular functions and are crucial for designing new peptide drugs. To explore the effect of the secondary structures from our model in the prediction of protein‐peptide binding sites, comparison experiments with the PepBCL model were conducted.^{[ 45 ]} Specifically, we first combined our structure predictions with the features from the PepBCL model. Then, protein–peptide binding site predictions were conducted based on a random forest machine learning method.^{[ 46 ]} In a previous study that used the PepBCL model,^{[ 36 ]} the secondary structure from SPOT‐1D‐Single was introduced to generate structural features, which we generated in the same way. In this context, the efficiency of our prediction can be verified by comparing secondary structures from several different sources (Table S9, Supporting Information). Our findings indicated that the application of peptide secondary structures predicted by our PHAT achieves significantly better performance than other methods. Some researchers have already incorporated secondary structures into their predictions. Moreover, the prediction of more accurate and continuous secondary structures may enhance the efficiency of site mining. As illustrated in Figure 4C, the features from PepBCL combined with the prediction of PHAT can achieve higher AUC and MCC than using peptide secondary structures from other methods.

3.6. The Visualization of Two Cases Demonstrated That Our Proposed PHAT Method Performs Better Than Existing Methods

To intuitively assess the performance of existing methods, we first randomly selected two peptide chains (PDB ID: 2w25A and 1ejbA) with experimental secondary structures, and applied different methods (PHAT, RaptorX, PSSP‐MVIRT, PROTEUS2, and Jpred) for the prediction of the secondary structure of two peptides. As illustrated in Figure 2L and Figure S5 (Supporting Information), the secondary structures from different methods were mapped into the tertiary structures, where the red area represents helix (H), the yellow area represents strand (E), and the green area represents coil (C). The differences between the structures predicted by our method and the experimental ones were smaller than those of the predictions of the other methods described above. In Figure 2L, our model achieved more correct helix (H) and strand (E) predictions, whereas the other methods were more likely to identify the helix (H) and strand (E) structures as a coil (C). Furthermore, in Figure S5 (Supporting Information), the other four methods (RaptorX, PSSP‐MVIRT, PROTEUS2, and Jpred) tended to predict the coil (C) as helix (H), whereas our method made more correct predictions in local consecutive sequence regions. In conclusion, our method can achieve better performance in terms of continuity and accuracy compared to the existing methods.

3.6.1. The Proposed PHAT Model Facilitates the Construction of 3‐D Peptide Structures

To explore the potential of PHAT in capturing 3‐D structure information of peptides, we used our model to predict the distance map and contact map matrices, which is an essential process in protein 3‐D structure prediction. The workflow of exploration is shown in Figure 5A. Specifically, our PHAT model was first trained using a secondary structure dataset to capture the 2‐D structure information of the peptide. Then, a fully connected network was added to our model and fine‐tuned using the contact map dataset (the details are shown in Table S11, Supporting Information) to obtain the distance information of the 3‐D structure. Next, we calculated the distance of each amino acid pair to construct the distance map and contact map of the peptide sequence. Compared with the experimental results from the test set, our model achieved an average variation of less than 1 Å for each amino acid pair in terms of distance map prediction. To intuitively assess the performance of our model, we visualized and compared our predictions with the state‐of‐the‐art method trRosetta^{[ 39} , ⁴⁷ , ^{48 ]} based on the experimental results from a randomly selected peptide with PDB ID 7ve4 (Figure 5B). Our predicted contact map is more accurate in terms of contacting amino acid pairs than the one obtained with trRosetta. Additionally, our predicted distance map is closer to the experimental result than the trRosetta‐generated map, indicating that our model can more accurately capture the distance between amino acids. With our predicted contact maps and distance maps, the 3‐D structures of corresponding peptides can be reconstructed more realistically by folding algorithms.^{[ 49} , ⁵⁰ , ^{51 ]} In this case, we extended our prediction of the secondary structure to the contact map and distance map, thus aiding in the prediction of the peptide 3‐D structure. Therefore, our PHAT model has the potential to promote the development of therapeutic molecules against various diseases, as well as the design of functional peptides.^{[ 42 ]}

Figure 5.

The exploration in constructing 3‐D structure of peptide with our method. A) The workflow of assisting in building 3‐D peptide structure with our predicted contact and distance map matrices. B) The visualization of contact map and distance map matrices of experimental results, trRosetta prediction and our prediction for the peptide with PDB ID: 7ve4.

4. Discussion and Conclusion

In this study, we developed PHAT, a deep learning‐based framework for peptide secondary structure prediction, and systematically evaluated it using benchmark datasets. Compared with other methods designed for protein secondary structure prediction, our model achieved superior performance in most metrics, especially Acc_E and SOV. The conventional methods designed for the prediction of protein structure might be biased toward extracting long‐distance dependence within protein sequences with hundreds of residues. However, the peptides in our dataset are significantly shorter than most proteins, and therefore the neighborhood information in peptides may not be easily captured by these methods. In contrast, our method can capture more contextual information of peptide sequences through the hypergraph multi‐head attention network, and can thus make more correct predictions in local consecutive sequence regions, as demonstrated by the visualization of our predictions for two peptides (PDB ID: 2w25A and 1ejbA).

Similar results can be seen when comparing the peptide‐specific secondary structure predictors (e.g., PSSP‐MVIRT) with our method. This is likely because previous methods designed for peptides focus more on neighborhood information of peptide residues and therefore tend to ignore long‐term information. In contrast, in addition to being capable of capturing contextual information, our method can obtain long‐term and bio‐semantic knowledge for peptide sequences by using ProtT5, a model pre‐trained with millions of protein sequences, thus achieving a good prediction performance. The peptide length preference experiments for secondary structure prediction illustrated that although the prediction performance of the tested methods decreased as the length of the sequences declined, our method achieved better performance than other existing methods when analyzing shorter peptide sequences. This indicated that our model can integrate contextual information and long‐term knowledge to make predictions.

Moreover, to reveal the feature extraction and prediction mechanisms of our PHAT model, we visualized matrices of a hypergraph multi‐head attention network (HyperGMA) and Bi‐LSTM‐CRF, which provide good interpretability while achieving an outstanding prediction performance. Specifically, the visualization of attention matrices in HyperGMA demonstrated that our model can effectively capture the local and global features of peptides at the residue group level and the peptide fragment level, thus providing insights into its attention mechanisms. Similarly, the visualization of the classification layer in Bi‐LSTM‐CRF illustrates that CRFs can guide our model to efficiently predict the secondary structure for each site in the peptide sequences.

Furthermore, to verify the accuracy of the secondary structures predicted by our model in downstream tasks, we applied our predicted structural information to the prediction of peptide toxicity, T‐cell receptor interactions with MHC–peptide complexes, and identification of protein–peptide binding sites. Using the secondary structures predicted by our model enhanced the performances of these tasks, which indicated that our predicted structural information can assist in predicting more accurate properties and is complementary to sequential and evolutionary features in peptide‐related downstream tasks. Additionally, to explore the potential of PHAT in capturing 3‐D structural information of peptides, we applied our model to predict distance map and contact map matrices and achieved an outstanding performance, thus demonstrating that our model can help in the reconstruction of peptide 3‐D structures. We also developed an online service (the workflow is shown in Figure 1D) to implement our PHAT, thus saving researchers the need to write programs or scripts. We hope that this online tool will be helpful to the research community.

Although our PHAT model achieves improved performances for predicting peptide secondary structure, there is still room for improvement. For example, PHAT is meant to be used for general peptide secondary structure prediction, and therefore we focused particularly on sequences with lengths <50. However, for datasets with peptide sequences longer than 50, we cannot ensure that our method will have the same performance. Moreover, when interacting with other targets (e.g., protein, DNA, RNA, etc.), peptide sequences remain the same, but the secondary structure of the peptides may change considerably. However, our PHAT makes its predictions based on the sequence patterns and thus cannot make adjustments to account for potential molecular interactions. Therefore, we are planning to incorporate additional data such as interaction information with other targets to further improve the prediction of peptide secondary structures in different interacting scenarios.

Conflict of Interest

The authors declare no conflict of interest.

Supporting information

Supporting Information

Jiang Y., Wang R., Feng J., Jin J., Liang S., Li Z., Yu Y., Ma A., Su R., Zou Q., Ma Q., Wei L., Explainable Deep Hypergraph Learning Modeling the Peptide Secondary Structure Prediction. Adv. Sci. 2023, 10, 2206151. 10.1002/advs.202206151

Data Availability Statement

The data that support the findings of this study are openly available in SSpro/ACCpro 5: almost perfect prediction of protein secondary structure and relative solvent accessibility using profiles, machine learning and structural similarity at https://doi.org/10.1093/bioinformatics/btu352, reference number 28. The details of data and code can also be found in Supporting Information.