Lactylation prediction models based on protein sequence and structural feature fusion

Abstract

Lysine lactylation (Kla) is a newly discovered posttranslational modification that is involved in important life activities, such as glycolysis-related cell function, macrophage polarization and nervous system regulation, and has received widespread attention due to the Warburg effect in tumor cells. In this work, we first design a natural language processing method to automatically extract the 3D structural features of Kla sites, avoiding potential biases caused by manually designed structural features. Then, we establish two Kla prediction frameworks, Attention-based feature fusion Kla model (ABFF-Kla) and EBFF-Kla, to integrate the sequence features and the structure features based on the attention layer and embedding layer, respectively. The results indicate that ABFF-Kla and Embedding-based feature fusion Kla model (EBFF-Kla), which fuse features from protein sequences and spatial structures, have better predictive performance than that of models that use only sequence features. Our work provides an approach for the automatic extraction of protein structural features, as well as a flexible framework for Kla prediction. The source code and the training data of the ABFF-Kla and the EBFF-Kla are publicly deposited at: https://github.com/ispotato/Lactylation_model.

INTRODUCTION

Lactate is the primary carbon-containing metabolite in the cell glycolysis pathway, and its biological function has been widely studied due to the existence of the Warburg effect in tumor cells [1]. Lysine lactylation (Kla) is a newly discovered posttranslational modification (PTM) in both histone and nonhistone proteins [2, 3] that has been reported to be involved in important life activities, such as glycolysis-related cell function, macrophage polarization and nervous system regulation [4, 5], and plays a key role in neuroinflammation and tumorigenesis [6, 7]. To understand the molecular mechanism and immune regulation pathway of lactylation, high-resolution liquid chromatography–tandem mass spectrometry (LC–MS/MS) has been used to identify Kla sites in human tissue and cell lines [8–11] and other species [12–16]. However, as the experiment is time-consuming and costly, and modified proteins are not abundant, it is difficult to widely achieve the global profiling of lactylation in different species by current experimental strategies.

Some computational methods have been used to predict Kla sites. few-shot learning-based Kla prediction model (FSL-Kla) is a few-shot learning model based on 343 Kla sites of 191 proteins collected from Homo sapiens, Mus musculus and Botryotinia fuckeliana. In feature coding schemes, FSL-Kla adopts 11 manually designed features, including the amino acid composition and physicochemical properties of the sequence, as well as structural features, such as the accessible surface area, protein secondary structure (SS) and backbone torsion angle [17]. However, manually designed features are limited to express more in-depth and complex biological information that may be related to PTMs [18]. Moreover, the features designed based on prior knowledge also limit the deep learning models to exploring more suitable features in new fields. To solve this problem, subsequent models have used embedding layers to automatically extract features from amino acid sequences: DeepKla has been combined with a convolutional neural network, bidirectional gated recurrent units and an attention mechanism to predict Kla sites in rice [19]; auto-Kla predicts Kla sites in gastric cancer cells based on automated machine learning [20]. These studies have explored Kla site prediction, but there are still some areas for improvement. First, these models focus on using features extracted from protein primary sequences [17, 19, 20], but in actual 3D folding structures, residues adjacent to the Kla site and those that affect its modification may be far apart from it in the sequence. Therefore, when only using sequence features, part of the information may be missed [21]. In addition to the FSL-Kla model, other models that integrate structural information for PTM analysis have been introduced. Kamacioglu et al. [22] selected 23 structure-related parameters obtained from the Protein Database (PDB) and UniProt [23] to annotate and reveal functional phosphorylation sites [24]. Fei Zhu et al. [25] presented two deep learning models by incorporating sequence and structural features, including absolute solvent accessibility (ACC) and SS, to elucidate the molecular basis and underlying functional landscape of PTMs.

However, the structural features used in the above studies were still designed manually, due to our current lack of precise understanding of which structure features play an important role in the process of lactylation, so it is difficult to manual design appropriate features for Kla prediction. Due to the complexity of protein structure information representation compared to primary sequences, it is difficult to extract structural features directly through natural language methods like sequence features. Therefore, there is currently no automatic feature extraction method based on protein spatial structure for Kla site prediction. Second, current Kla prediction models use relatively small datasets, and insufficient samples could give rise to overfitting and affect the accuracy of deep learning models.

Fortunately, profiting from the breakthrough of deep learning algorithms, we can obtain protein structural information with experimental accuracy of almost the complete human proteome from the AlphaFold protein structure database (https://AlphaFold.ebi.ac.uk) [26, 27], enabling us to conveniently extract the protein 3D structural-based features directly from PDB files and further reveal the structural background of Kla in the human proteome [21]. Meanwhile, the recently newly identified Kla datasets of hepatocellular carcinoma and human lung tissues through LC–MS/MS also provide more sufficient training samples for human Kla prediction [9, 10].

In this work, we established two different neural network frameworks based on natural language processing (NLP) to fuse features from protein sequences and 3D structures for lactylation prediction while automatically extracting information from protein sequences and 3D structures, avoiding potential biases in manual feature design. Specifically, to process 3D structural features using the NLP method, we selected the nearest contact residue set of the target lysine site from the residue contact map generated by the protein structure and arranged them according to their order in the protein sequence as the input segments. Through this approach, we can not only easily utilize transformer models with attention mechanisms to extract important information hidden in protein structures but we can also use different ways to fuse protein structure features with sequence features, increasing the flexibility of the lactylation prediction model. After constructing the benchmark dataset of the human Kla sites of the global lactylated proteins [8–11], we compared our two kinds of fusion feature models with the other Kla prediction models and achieved good results, indicating the effectiveness of our methods. To our knowledge, this is the first work that utilizes an NLP model to extract protein structure features for Kla site prediction, providing a framework for integrating information from protein structures into the prediction of other kinds of PTMs. The architecture of the feature fusion models is shown in Figure 1.

Figure 1.

The framework of the feature fusion Kla prediction models. (A) The framework of the ABFF-Kla. (B) The framework of the EBFF-Kla.

MATERIALS AND METHODS

Benchmark dataset

Sequence features

In this study, we collected multiple human lactylation data sites identified by LC–MS/MS experiments as the sample set [8–11]. Among the larger datasets which include 10 749 Kla sites of 2856 proteins, we selected with 80% for model training and testing, and 20% for validation [8–10]. For a smaller independent experimental dataset, including 755 Kla sites on 548 proteins, as another independent validation set [11]. Then, we downloaded these protein sequences from the UniProt database [23] and used CD-HIT (program for clustering large protein databases at different sequence identity levels) to remove redundant sequences [28]. Especially, by referring to previous research and our experience [29], we have selected 0.7 as the similarity threshold for CD-HIT, as a high threshold cannot clear similar redundant sequences, while a low threshold will aggregate different sequences into one class, resulting in a loss of sequence diversity. Unsuitable threshold will affect the distribution of samples and the model performance. These sequences were truncated into 35-residue-long peptides with lysine located at the center, and a segment was defined as a positive sample if its central lysine was a Kla site; otherwise, it was defined as a negative sample [19, 20]. Finally, we obtained 9560 positive nonredundant Kcr sites. Meanwhile, to balance the positive and negative samples, 9560 nonredundant normal lysine sites were randomly selected as negative K sties. In order to determine the optimal range of peptide length in the samples, in addition to the 35 residues length samples, we also reduced and increased the upstream and downstream lengths of the central lysine site by two residue positions, with corresponding sample lengths of 31 and 39, respectively (see Supplementary Materials available online at http://bib.oxfordjournals.org).

Structure feature

In this work, we automatically extracted features around Kla sites using an NLP method, including not only its upstream and downstream residues in the primary sequence but also its contact residues in the spatial structure [21, 30]. In the analysis of protein structure, two resides was considered to be in contact if the distance between their corresponding Cα atom is below to a certain threshold, which usually between 8 Å and 10 Å [31–33]. The selection of distance threshold will affects the number of contacts in the protein, and the lower the threshold, the fewer contact residue pairs will be obtained. In this work, to construct the contact maps of a protein, we hope to preserve as much contact residues information as possible around the target lysine site for subsequent calculations, so we adopted the upper limit 10 Å as the threshold [31]. After obtaining the above nonredundant sequences, we downloaded the PDB files of these proteins from the AlphaFold database [26], and constructed the contact maps of them from the 3-dimensional atomic coordinates of the corresponding PDB files [31]. For each target lysine residue in the nonredundant positive and negative datasets, we selected the nearest contact residue set of the target K site from the protein contact map and then arranged these contact residues into a new 35-residue-long segment based on their order on the protein sequence with the target K site in the center of the segment, named the contact segment, as shown in Figure 1A. Finally, we obtained 8884 positive contact segment samples and 8884 negative contact segment samples. Similar to the sequence features, to determine the optimal range of the segment length, we also attempted 31, 35 and 39 residues long segment, respectively (Supplementary Materials available online at http://bib.oxfordjournals.org).

Feature fusion network for Kla prediction

Attention-based feature fusion model

Attention mechanisms have been widely used in NLP tasks and are based on the hidden states of the source and target sides to obtain the dependency relationship between each word on the source side and the target side, thus focusing on the important information that is screened out [34]. Self-attention further reduces the dependence on external information and is better at capturing the internal correlations of features, especially by calculating the mutual influence between words, thus solving the problem of long-distance dependence [35]. Using deep learning models integrating attention methods to mine semantic associations and syntactic features between residues in protein sequences, various types of PTMs can be predicted, and good performance has been achieved [18–20, 36].

In this work, we designed a dual-channel neural network to integrate two self-attention submodules, which focus on the context of protein sequences and 3D structures that affect Kla sites. For each submodule, the 35-residue-long peptides and contact fragments were regarded as sentences in the text corpus and through the token embedding layer input to their self-attention layer. Next, the combination of a convolutional layer and a long short-term memory (LSTM) layer can more accurately obtain the most important semantic features and positional information of the features [37, 38], as shown in Figure 1A. The self-attention layer and the output sequence features of the sequence submodule is shown in Formula 1 and Formula 2, and the structure submodule below, respectively. Similarly, the self-attention layer and the output structure features of the structure submodule is shown in Formulas 3 and 4, respectively. Finally, the output of the two submodules was merged through a new self-attention layer, thereby incorporating the features of the residues that maybe far from the target K site in the protein sequence but could affect the target K site in the spatial structure into the Kla prediction. The calculation of this process can be found in Formulas 5 and 6. We refer to this model as ABFF-Kla, and its architecture is shown in Figure 1A.

(1)

(2)

(3)

(4)

(5)

(6)

Embedding-based feature fusion model

In the ABFF-Kla model, the sequence features and structure features are fused through their respective self-attention submodules. In this section, we attempted to fuse the features through the embedding module. This model was named EBFF-Kla. In the EBFF-Kla model, the input layer is summed by two embeddings: a sequence embedding and a contact segment embedding, which represent upstream and downstream residues of the target K site in the sequence and the nearest contact residues of the target K site in the spatial structure, respectively [30]. The description of the embedding is as follow:

(7)

(8)

Then, a self-attention layer combined with a bidirectional gated recurrent unit (Bi-GRU) layer deeply extracts the most important semantic features and position features of the residues either in the primary sequence or 3D structure that affect the lactylation of target lysine sites, as shown in Figure 1B.

Model training and evaluation

When training the ABFF-Kla model, we adopted the idea of transfer learning. First, we pretrained the sequence-based submodule and structure-based submodule separately and transfer the parameters of these two models to ABFF-Kla for initialization. Then, the entire model is trained by simultaneously inputting the sequence peptides and the contact segments. To evaluate the performance of ABFF-Kla and EBFF-Kla, we calculated the four different parameters: specificity (SPE), sensitivity (SEN), accuracy (ACC) and Matthews correlation coefficient (MSS). The four parameters are defined as follows:

where TP, FP, TN and FN are the numbers of true positive samples, false-positive samples, true negative samples and false-negative samples, respectively. We also calculated the area under receiver operating characteristic curves as evaluation measures in this work.

RESULTS

Feature fusion model performance

To compare the performance of the above feature fusion models for Kla prediction, we randomly selected 80% from the human Kla benchmark dataset as the training and testing set, and using the remaining 20% for validation. After five random partitions were conducted, the mean accuracy and MCC of 10-fold cross-validation showed that the ABFF-Kla model has a higher performance than that of EBFF-Kla, indicating that the attention-based feature fusion framework can provide richer information than that through embedding-based feature fusion in Kla prediction (Figure 2). In addition, to optimize the two models and find the recurrent neural network (RNN) layer types that are more suitable for each of them, we swapped and compared the LSTM layer of ABFF-Kla and the Bi-GRU layer of EBFF-Kla, as shown in Figure 2A and B. The results indicated that after replacing the LSTM layer with the Bi-GRU layer, the Kla prediction performance of EBFF-Kla was improved (Figure 2). The reason is that EBFF-Kla directly overlays complex features through the embedding module, while the GRU can effectively capture the semantic associations of long sequences while suppressing gradient vanishing or exploding, leading to a complexity lower than that of LSTM [35, 36].

Figure 2.

The accuracy and MCC of the feature fuse Kla prediction models. (A) The accuracy of the sequence sub-modules, the structure sub-modules, ABFF-Kla and EBFF-Kla. (B) The MCC of the sequence sub-module, the structure sub-module, ABFF-Kla and EBFF-Kla.

To compare the contribution of protein sequences and contact fragments to Kla prediction, we trained the self-attention submodule of the ABFF-Kla model on the 35-residue-long peptides and contact fragments. From Figure 2A and B, we found that the prediction performance of the submodule using protein sequences was higher than that of the submodule using contact segments but lower than that of feature fusion models, illustrating that protein structure features can provide additional information beyond sequence features for Kla prediction.

Comparison of the feature fusion models with different sample lengths

In order to determine the range of optimal sample length of our feature fusion models, we conducted five rounds of random partitioning on the human Kla benchmark dataset based on samples with 31 residue lengths, 35 residue lengths and 39 residue lengths, respectively, and make independently validated on each round. The mean accuracy and MCC of 10-fold cross-validation on the test set, validation set and the independent validation set showed that both the sequence sub-module and the structure sub-module exhibit higher performance over 35-residue long samples than other two lengths (Figure 3A and B). Especially for independent validation sets, this trend is more pronounced on the structure sub-module than on the sequence sub-module (Figure 3A and B). Therefore, as a fusion model for two sub modules, the trend of ABFF-Kla is also higher than that of EBFF-Kla, which directly performs feature fusion on the embedding layer (Figure 3C and D). This result indicated that as the sample length increases, although the model can obtain more sequence and structural information, the corresponding interference factors introduced will also increase, thereby affecting the generalization ability of the models.

Figure 3.

Compare the feature fusion model on different sample lengths. (A) The ACC and MCC of the sequence sub-module. (B) The ACC and MCC of the structure sub-module. (C) The ACC and MCC of ABFF-Kla. (D) The ACC and MCC of EBFF-Kla.

Comparison of the feature fusion models with other Kla prediction models

Then, we compared our feature fusion models with other Kla prediction models, such as DeepKla [19] and Auto-Kla [20]. We also conducted five rounds of random partitioning on the human Kla benchmark dataset, and make independently validated on each round using the independent dataset. The mean ACC, MCC and AUC of 10-fold cross-validation indicated that ABFF-Kla (LSTM) has the highest performance, followed by EBFF-Kla (Bi-GRU), which is higher than that of auto-Kla and DeepKla, as shown in Figure 4. From Figure 4A–C, it can be observed that the two ABFF-Kla models have higher specificity and sensitivity than those of the other models, which illustrates that the framework of attention-based feature fusion leads to better predictive performance. In these models, DeepKla and Auto-Kla only use features from protein sequences, while ABFF-Kla and EBFF-Kla fuse features from protein sequences and spatial structures, indicating that contact fragments can provide additional information that affects the lactylation of target lysine sites.

Figure 4.

Compare the feature fuse Kla prediction with other Kla models. (A) Compare the accuracy of different Kla models. (B) Compare the MCC of different Kla models. (C) Compare the AUC of different Kla models. (D) The mean ACC, MCC and AUC values of different Kla models.

Prediction the lactylation of human proteome

In this section, we used our feature fusion models to predict the lactylation of all proteins in the human proteome. Firstly, we downloaded the Swiss-Prot human protein sequence from Uniprot database (https://www.uniprot.org/uniprotkb), which a total of 50 302 proteins [23]. Among them, in addition to 2991 lactylation proteins that have been identified by MS [8–11] (Supplementary Table S1, available online at http://bib.oxfordjournals.org), we used the corresponding models to predict Kla sites of the remaining proteins: (1) For the lysine sites with both sequence and structure features, we used the ABFF-Kla model for prediction, and these results are shown in Supplementary Table S2, available online at http://bib.oxfordjournals.org. (2) For the lysine sites with only sequence features but no structure information, we used the sequence sub-module for prediction, this results are listed in Supplementary Table S3, available online at http://bib.oxfordjournals.org. (3) Specially, for the lysine sites that cannot obtain sufficient sequence features due to their proximity to the ends of the protein sequence, if they have structure features, we can use structure sub-models for prediction, which is also an advantage of our feature fusion model compared to other sequence feature prediction models. The prediction results for this part are shown in Supplementary Table S4, available online at http://bib.oxfordjournals.org, and we conducted GO enrichment analysis on these proteins, as shown in Figure 5.

Figure 5.

The GO enrichment analysis of the proteins with Kla sites predicted by the structure sub-module.

CONCLUSION AND DISCUSSION

In this work, we first built a NLP method to automatically extract the 3D structural features of proteins, avoiding the potential biases caused by manually designed structural features in previous studies. Then, we designed two neural network frameworks, the attention-based feature fusion model ABFF-Kla and the embedding-based feature fusion model EBFF-Kla, to integrate the context of protein sequences and 3D structures that affect Kla sites, which improved the performance of Kla prediction. Our work provides an approach for the automatic extraction of protein structural features, as well as a flexible framework for the integration of different types of protein features, not only for lactylation prediction but also for other PTM prediction studies.

However, there are still some shortcomings in this research. We automatically extracted the contact residues adjacent to the target lysine site through contact maps and generated contact segments as input features, reducing the information in a 3D space to a 2D plane, which inevitably leads to the loss of some structural information. How to better utilize the advantages of automatic feature extraction in deep learning models to extract richer and more important structural features for the prediction of lactylation is our next research direction.

Key Points

• Generated the contact residues segments of the lactylated lysine site from the protein structures predicted by Alphafold2.

• Using natural language processing method to automatically extract the 3D structural features of proteins, avoiding manual feature design.

• Establish two lactylation prediction frameworks, ABFF-Kla and EBFF-Kla, to fuse the sequence features and the structure features based on the attention layer and embedding layer, respectively.

Author Biographies

Ye-Hong Yang is a master at the State Key Laboratory of Common Mechanism Research for Major Diseases. Her research interests include machine learning and bioinformatics.

Jun-Tao Yang is a professor at the State Key Laboratory of Common Mechanism Research for Major Diseases. His research interests include proteomics and bioinformatics.

Jiang-Feng Liu is a PhD at the State Key Laboratory of Common Mechanism Research for Major Diseases. Her research interests include proteomics.

FUNDING

This work was supported by grants from the Chinese Academy of Medical Sciences Innovation Fund for Medical Sciences, China (grant numbers CIFMS2022-I2M-2-001, CIFMS2022-I2M-1-011, CIFMS2021-I2M-1-057, CIFMS2021-I2M-1-049, CIFMS2021-I2M-1-044, CIFMS2021-I2M-1-016 and CIFMS2021-12 M-1-001).

DATA AVAILABILITY

The source code and the datasets of this research are all available and can be download from the web site: https://github.com/ispotato/Lactylation_model.