Mining lysine post-translational modification sites by integrating protein language model representations with structural context

Significance

Lysine post-translational modifications (PTMs) are crucial for regulating protein function, yet their experimental identification remains challenging. To address this, we developed an AI framework that uniquely integrates protein sequence information from large language models with structural features extracted via graph neural networks. This hybrid framework demonstrates its capability in identifying diverse lysine PTM sites. When applied to human protein hCLEC12A, our model identified modification sites that were validated through molecular dynamics simulations, confirming their role in modulating antibody binding affinity. Our approach provides a flexible computational framework for PTM site investigation and facilitates the exploration of PTM-associated regulatory features, thereby supporting further studies on protein regulation.

Abstract

Lysine (Lys/K) residues serve as major hubs for post-translational modifications (PTMs) owing to the chemical versatility of their ε-amino groups, giving rise to diverse regulatory functions. Accurate and efficient identification of modified lysine residues therefore requires computational models that can effectively capture both sequence and structural information while minimizing domain-specific feature engineering. In this study, we propose a unified deep learning framework for lysine PTM site identification that integrates sequence representations derived from a protein language model with atom-level three-dimensional structural features. This framework can be consistently applied to multiple lysine PTM types using a shared modeling strategy. As an application, we used the model to predict potential PTM site on human C-type lectin domain family 12 member A (hCLEC12A) and evaluated their functional relevance through all-atom molecular dynamics simulations. The simulations indicate that the predicted lysine residues influence the stability and binding behavior of the hCLEC12A-antibody 50C1 complex. Overall, this work presents an integrative computational framework for lysine PTM site mining and functional analysis.

Post-translational modifications (PTMs) involve the covalent attachment of specific chemical groups to amino acid side chains, mediated by enzymatic or nonenzymatic mechanisms. This unique process modulates gene transcription and expands the structural and functional diversity of proteins (1–4). These modification sites act as “molecular switches”, that control protein conformation, dynamics, and allosteric communications. Among them, lysine (Lys/K) residues emerge as central hubs of PTMs due to the versatile reactivity of their ε-amino groups, which undergo diverse modifications—including acetylation, ubiquitination, methylation, and so on. The frequency and functional complexity of lysine modifications far surpass those of other residues. Systematic mapping of lysine PTM sites not only narrows the functional scope of uncharacterized proteins but also unlocks insights into protein regulation, disease mechanisms, and therapeutic targeting (5–7).

Traditional identification of protein PTM sites mainly relies on mass spectrometry (MS)-based wet-lab techniques, using high-sensitivity MS (e.g., HPLC-MS/MS) to detect modified peptides’ mass shifts (8, 9), coupled with high-resolution MS for systematic site mapping (10). To enhance specificity, antibody-based affinity enrichment and synthetic peptide coelution verification are utilized (9, 11), while quantitative proteomics and isotopic labeling reveal complex modification profiles (9, 10), forming a complete workflow from target identification to functional validation (12). However, these high-throughput MS approaches remain costly, time-consuming, and instrument-intensive (13, 14), highlighting the need for complementary computational approaches to optimize experimental design and resource allocation in PTM site identification research.

In contrast, computational-based methods, particularly machine learning, offer a rapid, cost-effective alternative for PTM site identification. Early studies primarily relied on feature extraction techniques based on sequence information, including: composition-based features such as K-spaced amino acid pairs, pseudo amino acid composition (PseAAC), dipeptide composition (DC), and the BLOSUM62 matrix; physicochemical property-based features such as AAindex database; and evolutionary information-based features such as position weight amino acid composition (PWAA) and enhanced amino acid composition (EAAC) (13–18). These handcrafted features were typically combined with classical machine learning algorithms for prediction, such as Support Vector Machines (SVM), Penalized Logistic Regression (PLR), Random Forest (RF), and Rotation Forest (RoF) (15, 17, 19). Later studies improved performance by integrating multiple feature types, including biochemical and physicochemical features (20), secondary structure (SS), solvent-accessible surface area (ASA), and backbone torsion angles (θ, τ, ψ, ϕ), though limited by accuracy and computational costs (15, 19). Deep learning revolutionized the field through hybrid architectures (CNN-BiLSTM, GRU-DNN) (18, 20) and pretrained language models like ProtBert, which extracts semantic features from sequences (14). Recent advances involve multimodal fusion strategies (ProtFusion, Deepro-Glu) combining pretrained embeddings with handcrafted features (20, 21), alongside optimization techniques like reinforcement learning and joint loss functions (20, 22).

Despite significant progress, current approaches face several limitations. Existing models depend heavily on labor-intensive handcrafted feature engineering involving various sequence, structural, and functional descriptors. Most frameworks are restricted to predicting single modification types rather than offering generalizable multi-PTM frameworks. Crucially, critical three-dimensional structural features including residue spatial arrangement and atomic coordinates remain largely unexploited. Additionally, the potential of advanced protein language models like ESM-2 for automated feature extraction and sequence–structure integration has not been fully exploited. Furthermore, a key limitation in the field is that research has prioritized model performance on benchmark datasets over their prospective application to investigate uncharacterized PTM sites, which necessitates subsequent meaningful verification.

Molecular dynamics (MD) simulation is a powerful tool to bridge the gap between static predictions and dynamic biological validation (23–27). For example, Heppner et al. (23) utilized microsecond-scale MD and metadynamics to demonstrate how redox-sensitive cysteine sulfenylation in Src kinase induces local structural perturbations, promoting activation loop unfolding and disrupting autoinhibitory interactions. Similarly, the Roos et al. (24) review highlights how MD simulations revealed the critical role of hydrogen bonding networks in modulating cysteine pKa values, providing mechanistic insights into redox enzyme function that static structures alone could not capture. In the context of phosphorylation, Ali et al. (27) employed MD to model how lysine acetylation alters the electrostatic potential of RNA polymerase II’s C-terminal domain, enhancing its binding to reader proteins and facilitating downstream dephosphorylation events. These studies demonstrate MD’s capability to validate and refine PTM-related structural predictions, offering mechanistic insights that complement experimental data.

We present a semantic encoding framework for predicting lysine PTM sites by integrating protein language model representations with atom-level structural features. The framework avoids domain-specific feature engineering and supports generalizable prediction across multiple PTM types. Subsequently, we identified potential PTM sites on human C-type lectin domain family 12 member A (hCLEC12A) using this Model, and we performed all atom MD simulations to evaluate the stability and conformational dynamics of the modified protein and its interaction with the antibody 50C1. These simulations showed that the residues predicted by our model indeed modulate the binding affinity between hCLEC-A12A and antibody 50C1. By synergizing AI-driven prediction with MD-based functional annotation, our workflow transcends static sequence-based limitations, offering a dynamic framework for investigating uncharacterized PTM crosstalk mechanisms.

Results

The Prediction Model Incorporates Large Language Model Alongside Neural Networks.

The prediction model we constructed can capture both protein sequence and structural information, as shown in Fig. 1. For sequence feature extraction, we employ a large protein language model by extracting 20 amino acids upstream and downstream of each PTM site as the contextual sequence. The Evolutionary Scale Modeling-2 (ESM-2) (19) model is used to encode these sequences into an initial feature tensor. This tensor is subsequently reduced in dimensionality through a fully connected layer. The resulting tensor is then fed into a three-layer bidirectional long short-term memory (Bi-LSTM) network with a hidden size of 256, yielding an output of size 256 × 2 from the sequence processing module. For structural feature extraction, atom-level based interresidue distances within the protein are considered, accounting for factors such as peptide bonds, centroid distances, and positive/negative charge interactions. A contact map is constructed using the three-dimensional atom-level molecular coordinates of the amino acids, forming a graph for the local environment around each site. This graph is processed by a three-layer graph convolutional neural network (GCN), where the features undergo dimensional expansion followed by reduction, resulting in an output of size 30. The sequence features (256 × 2) and structural features (28) are concatenated and finally processed by a three-layer multilayer perceptron (MLP) to produce the prediction output.

Fig. 1.

Overview of prediction model. The model primarily consists of a structural information processing module and a sequential information processing module. The output feature vectors from both modules are concatenated and then passed through a fully connected network for dimensionality reduction to produce the final output. Specifically, the structural information processing module constructs a contact map from the atom-level three-dimensional coordinates of amino acids, which is subsequently processed using a graph neural network. The sequential information processing module obtains representations through a large language model, followed by feature processing via a linear layer and a bidirectional long short-term memory (Bi-LSTM) network.

Independent Modeling of Sequence and Structural Information with Protein Large Language Model and Graph.

We developed a dual-perspective computational framework that integrates evolutionary information from protein sequences with atom-level three-dimensional structural features to comprehensively characterize PTM sites. To extract sequence information, we employed a protein language model, which performs deep encoding of input sequences to generate a 1,280-dimensional feature representation for each residue. We then applied global average pooling to these representations to obtain a fixed-dimensional global feature vector encapsulating the entire sequence. To further model contextual dependencies along the protein sequence, we applied a two-layer bidirectional LSTM to the projected residue embeddings. This module captures long-range interactions from both N- and C-terminal directions, enabling richer contextual encoding beyond the per-residue representations produced by the language model. The BiLSTM outputs were then averaged to generate a compact, sequence-aware feature vector.

For structural information representation, we constructed a contact map model based on atomic three-dimensional coordinates. This model integrates structural features from three key aspects: peptide bond covalent connections, spatial proximity effects, and electrostatic interactions, establishing graph-structured data containing atomic coordinate node features and distance-based edge attributes. Node features were generated by concatenating the standardized 3D coordinates of all atoms within each amino acid residue. Finally, feature scaling was applied to ensure data consistency, enabling comprehensive capture of the spatial conformation, topological constraints, and molecular interaction networks surrounding modification sites.

We performed Principal Component Analysis (PCA) on the feature vectors to project the high-dimensional features into a three-dimensional space for visualization. Following dimensionality reduction, a three-dimensional scatter plot was constructed using the first three principal components (PC1, PC2, PC3) as axes, in which positive samples (y = 1) are marked in blue and negative samples (y = 0) in red. As illustrated in Fig. 2, the sequence features surrounding the six types of PTM sites do not completely overlap in the projected space, showing varying degrees of distribution separation. Positive and negative samples each occupy relatively concentrated regions, and although some samples overlap, the observable distribution differences suggest strong potential for discriminating between the two classes based on these features. Similarly, the structural features also form distinct clustered regions with characteristic spatial distributions, providing an exploratory basis for subsequent classification between positive and negative samples. Each axis in the figure is explicitly labeled with the corresponding principal component’s variance contribution ratio, reflecting the extent to which the original high-dimensional information is preserved in the reduced space.

Fig. 2.

Sequence and structural feature characterization of six PTM sites via PCA. (A) Succinylation: Sequence Features (Left) and Structural Features (Right). (B) Crotonylation: Sequence Features (Left) and Structural Features (Right). (C) Malonylation: Sequence Features (Left) and Structural Features (Right). (D) 2-Hydroxyisobutyrylation: Sequence Features (Left) and Structural Features (Right). (E) Ubiquitination: Sequence Features (Left) and Structural Features (Right). (F) Acetylation: Sequence Features (Left) and Structural Features (Right).

Performance of the Model under Realistic and Nonredundant Evaluation Settings.

To evaluate the proposed framework under a rigorous training protocol, model optimization and selection were performed using a training–validation–test scheme with early stopping. Proteins with available structural information were partitioned at the protein level into training, validation, and test sets using a ratio of 6:1.5:1.5, ensuring that no protein appeared in more than one subset. To mitigate sequence homology bias during evaluation, a nonredundant protein set was constructed at the protein level using CD-HIT with a 40% sequence identity threshold, and validation and test samples were restricted to this nonredundant set. The training and validation sets were constructed as class-balanced datasets, enabling stable optimization and reliable early-stopping decisions, whereas the test set followed a realistic, highly imbalanced distribution, in which annotated lysine residues were treated as positives and all remaining lysines within the same proteins were considered negatives.

In addition to the nonredundant evaluation setting, the proposed model was further assessed on a raw dataset that preserves the original biological distribution of PTM sites, where homologous proteins and conserved motifs are inherently present. This setting more closely reflects real-world application scenarios and provides a complementary perspective on model robustness. For the raw dataset, the same protein-level data partitioning strategy was applied, allowing a direct comparison with the nonredundant setting. Together, these two evaluation schemes enable a comprehensive assessment of the model’s predictive performance under both stringent and realistic conditions.

As shown in Fig. 3, the model exhibits highly consistent AUC performance between validation and test sets under both nonredundant and raw conditions, indicating strong generalization capability. Although a moderate performance decrease is observed on the nonredundant dataset, which is expected due to the removal of homologous sequence information, the close alignment between validation and test results suggests that the model captures fundamental sequence–structure features underlying PTM site formation rather than relying on redundancy-driven shortcuts. Importantly, the stable performance observed on the raw dataset—designed to approximate real biological data distributions—demonstrates the model’s suitability for practical PTM site prediction and supports its application to downstream analyses, including the identification and verification of uncharacterized modification sites.

Fig. 3.

AUC performance of the proposed model across six PTM types under raw and nonredundant data regimes. (A) Validation AUC across PTMs on the raw dataset. (B) Test AUC across PTMs on the raw dataset. (C) Validation AUC across PTMs on the nonredundant dataset. (D) Test AUC across PTMs on the nonredundant dataset.

In addition to balanced benchmark evaluations, we further examined model behavior under a class-imbalanced setting using nonredundant test proteins, where all lysine residues were treated as prediction candidates (SI Appendix, Table S7). Under this scenario, ranking-based metrics reveal complementary characteristics of model performance. While the model maintains stable discriminative capability across PTM types, precision–recall–based evaluation is strongly influenced by the low prevalence of positive sites. Accordingly, both AUROC and AUPRC are reported in the table for both validation set and test set to provide a comprehensive view of ranking performance under class-imbalanced conditions.

All primary analyses reported above were conducted using a 41-aa sequence window centered on the target lysine residue. To investigate the impact of amino acid context length on predictive performance, we also evaluated another two window sizes centered on the target lysine residue (21 and 31 amino acids) using the raw dataset. As shown in SI Appendix, Fig. S1, the model exhibits highly comparable AUC values for the 21-aa and 31-aa windows on both validation and test sets, with test AUCs consistently falling within approximately 0.70 to 0.82 across different PTM types and only marginal variations observed between window lengths. Together with the results obtained using the 41-aa window in the main analysis, these findings indicate that the proposed framework is not strongly sensitive to moderate changes in sequence context length and is able to capture the core sequence–structure determinants of PTM site formation. Considering the comparable predictive performance across window sizes and the need to balance contextual information with computational efficiency, we adopt the 41-aa window in subsequent analyses, as it provides the richest representation while remaining practical for large-scale PTM site prediction.

The Comprehensive Prediction of Modification Sites Across Six PTMs.

To evaluate the practical predictive capability of the proposed framework, we conducted experiments across six representative PTMs using nondeduplicated datasets, which more closely reflect real proteomic scenarios where homologous proteins and conserved motifs are inherently present. For each PTM, the data were partitioned into training and test sets at a ratio of 7:3 based on site counts, while ensuring that samples derived from the same protein were assigned exclusively to one subset; the split was performed by sequentially assigning proteins until the target ratio was reached, and both subsets were constructed to be class-balanced. This design enables a reliable assessment of the model’s intrinsic discriminative ability by mitigating biases introduced by severe class imbalance and allowing performance metrics such as F1 score to be interpreted consistently across PTM types.

The model was trained for approximately 160 epochs under identical architectural and optimization settings. After 160 epochs of training, the model achieves effective fitting across six PTM training datasets, SI Appendix, Fig. S2 presents the corresponding changes in evaluation metrics, reached nearly 100% around 110 epochs. Model performance on test datasets were summarized in Fig. 4. Overall, the proposed framework demonstrates consistently strong predictive performance across all six PTMs, achieving F1-scores ranging from approximately 71.7 to 80.9%, with corresponding AUC values between 78.5 and 88.3%. Across repeated evaluations, performance variations remain within a narrow margin (±1.0 to 1.5%), indicating robust convergence and stable generalization behavior.

Fig. 4.

Performance metrics of prediction models for different protein post-translational modifications (PTMs). (A) Succinylation. (B) Crotonylation. (C) Malonylation. (D) 2-Hydroxyisobutyrylation. (E) Ubiquitination. (F) Acetylation.

Among the evaluated PTMs, crotonylation exhibits the strongest overall performance, achieving the highest F1-score (80.9 ± 1.5%), AUC (88.3 ±1.5%), and MCC (63.6 ±1.0%), suggesting that crotonylation sites are characterized by highly discriminative sequence–structure patterns effectively captured by the model. Acetylation and succinylation also demonstrate favorable predictive profiles, with F1-scores exceeding 76% (±1.5%) and AUC values above 83%, reflecting a well-balanced trade-off between sensitivity and specificity. In contrast, malonylation and ubiquitination present comparatively lower, yet stable, predictive performance, with F1-scores around 72% and MCC values in the range of 42 to 45% (±1.0%). This observation is consistent with the greater biological heterogeneity and weaker sequence constraints associated with these modification types. Notably, 2-hydroxyisobutyrylation displays an intermediate performance profile, further supporting the generalizability of the proposed framework across diverse PTM categories.

Importantly, the use of balanced training and test datasets facilitates a more reliable assessment of site-level discriminative features, ensuring that the learned representations emphasize intrinsic sequence and structural determinants of modification rather than class prior distributions. These results indicate that the proposed model provides a robust and transferable foundation for PTM site prediction. In the subsequent section, this framework is further applied to identify putative crotonylation and acetylation sites, which are then subjected to computational validation to assess their structural and energetic plausibility.

Benchmarking Deep Learning Models for PTM Site Prediction.

To assess the contribution of structural information to PTM site prediction, we conducted a series of ablation experiments by benchmarking our full model against sequence-only baselines. Specifically, alternative neural architectures that rely exclusively on sequence-derived features were evaluated, including convolutional neural networks (CNN), hybrid CNN–LSTM models, and bidirectional LSTM (Bi-LSTM). The corresponding results are summarized in SI Appendix, Tables S3–S5. Additionally, we reproduced current state-of-the-art PTM site prediction models and applied them to our dataset; the results are presented in SI Appendix, Table S6. These ablation and comparative experiments demonstrate that our proposed model framework not only learns richer representations but also outperforms other methods in predictive accuracy and stability.

The Impacts of PTM on the HCLEC12A-50C1 Binding Affinity.

C-type lectin domain family 12 member A (CLEC12A) is an inhibitory receptor primarily expressed on myeloid cells (such as granulocytes and monocytes) (29,28, 30). It plays a role in immune homeostasis and is closely associated with diseases like acute myeloid leukemia (AML) (30). Its inhibitory antibody, 50C1, has been demonstrated to block the binding of multiple ligands (31). To evaluate the predictive reliability of our AI model, we applied it to identify potential PTM sites on human CLEC12A (hCLEC12A). The model predicted that residue K174 may undergo lysine acetylation and lysine crotonylation, while residue K181 may also be acetylated (SI Appendix, Fig. S4A). Based on these predictions, we using the unmodified complex (Pure) as reference, we constructed 5 modified systems for comparison: K174 monoacetylation (174Kac), K174 monocrotonylation (174Kcr), K181 monoacetylation (181Kac), K174/K181 double acetylation (174-181-Kac), and the combination of K174 crotonylation and K181 acetylation (174Kcr-181Kac). RMSD analysis confirmed that all simulated complexes maintained structural stability (SI Appendix, Fig. S3), with values plateauing after initial fluctuations, indicating conformational stability throughout the simulation.

We further computed binding free energies using the Molecular Mechanics Poisson–Boltzmann Surface Area (MM-PBSA) method (32). The results demonstrate that PTMs significantly modulate the binding affinity between hCLEC12A and 50C1 (Table 1). Among all systems, the unmodified Pure complex exhibited the strongest binding (∆Gbinding = −58.18 ± 1.94 kcal/mol). Per-residue energy decomposition highlighted that residues K181, R201, G202, R204, D206, R232, L233, Y234, and Q236 on hCLEC12A contribute favorably to complex stability (SI Appendix, Fig. S4D). Interaction analysis further revealed that 50C1 engages residues S184, D198, T200, R201, R204, and Q236 via hydrogen bonds, salt bridges, and cation–π interactions, thereby enhancing complex formation and stability (SI Appendix, Fig. S4B). These computational insights align with prior experimental findings (31). Upon the introduction of a single acetylation modification at K181 of hCLEC12A, the binding affinity decreased significantly (∆Gbinding = −43.01 ± 1.50 kcal/mol) compared to the Pure system (∆Gbinding = −58.18 ± 1.94 kcal/mol). Most notably, in the dual-modified system (174Kcr-181Kac), the binding affinity dropped sharply to −40.60 kcal/mol, suggesting a potential synergistic negative effect between these two modifications that collectively impairs the hCLEC12A–501C1 complex interaction. Both K174 and K181 PTM are located near the hCLEC12A–501C1 binding interface. After modification of these residues, the interactions between hCLEC12A and 50C1 were markedly reduced (Fig. 5 A and B), significantly perturbing the interaction network among interfacial residues. Per-residue energy decomposition analysis revealed that upon acetylation, the energy contribution of KL181 shifted from negative to positive, indicating a change from originally favoring binding to impairing it. Meanwhile, several other key residues at the binding interface (such as R201, G202, R204, D206, Y234, and Q236) also exhibited substantially reduced energy contributions in these modified systems. These combined changes ultimately led to the overall decrease in binding affinity. In conclusion, our simulations demonstrate that PTMs at key lysine residues significantly impair the binding between hCLEC12A and the 50C1 antibody, primarily through destabilization of critical interfacial interactions and perturbation of residue-specific energy contributions.

Table 1.

MM-PBSA binding energy calculations of six simulated systems

No.	Model	${\mathrm{\Delta }G}_{\mathit{binding}}$
1	Pure	−58.18 ± 1.94
2	174Kac	−52.09 ± 1.00
3	174Kcr	−48.66 ± 3.82
4	181Kac	−43.01 ± 1.50
5	174-181-Kac	−50.48 ± 2.77
6	174Kcr-181Kac	−40.60 ± 3.58

Fig. 5.

Impact of hCLEC12A PTM on 50C1 antibody recognition. (A) The binding interface between 501C and hCLEC12A (A) K181 acetylation or (B) 174 crotonylation and K181 acetylation. Hydrogen bonds, salt bridges, and π–π stacking interactions are represented by yellow, cyan, and purple dashed lines, respectively. Per residue free energy decomposition changes on the key residues in hClEC12A (C) K181 acetylation and (D) 174 crotonylation and K181 acetylation relative to the Pure system.$\mathrm{\Delta }\mathrm{\Delta }{G}_{\mathit{decomp}}=\mathrm{\Delta }{G}_{\mathit{decomp}}(Modified system)-\mathrm{\Delta }{G}_{\mathit{decomp}}(Pure)$. The data are expressed as means ± SEM. The smaller the value, the greater the contribution of the amino acid to the binding of hCLEC12A to 50C1.

Discussion

PTM sites function as critical molecular switches that regulate protein activity, mediate cellular signaling pathways, and contribute to disease-associated processes. Accurate identification of PTM sites therefore remains a central challenge for understanding protein regulation at the molecular level. In this study, we present a unified deep learning framework that integrates sequence semantic representations derived from protein language models with atom-level structural context, enabling effective prediction across six lysine PTM types.

Compared with existing PTM prediction frameworks that rely primarily on sequence-derived features or handcrafted physicochemical descriptors, our approach emphasizes representation learning and multimodal integration. Protein language models capture evolutionary constraints, contextual dependencies, and long-range interactions embedded in amino acid sequences, while atom-level structural features provide complementary spatial information that is often inaccessible to sequence-only models. The fusion of these two modalities allows the framework to balance local residue environments with global contextual cues, offering a more comprehensive representation for PTM site prediction.

Importantly, the proposed framework is not restricted to lysine PTM prediction. Its modular design enables straightforward adaptation to other scientific questions involving residue-level functional annotation. By modifying the labeling strategy and task-specific output layer, the same architecture could be applied to the prediction of alternative PTMs, protein–protein interaction interfaces, ligand-binding residues, or disease-associated mutation sites, without altering the core representation modules. This flexibility highlights the framework’s potential as a general computational paradigm rather than a task-specific solution. Beyond predictive performance, the framework can also be applied to downstream functional analyses that link PTM site prediction to structural and energetic consequences.

Furthermore, we applied our model to predict PTM sites on the hCLEC12A protein and validated the predictions using MD simulations. The predicted PTM sites are located at the binding interface between hCLEC12A and its antibody 50C1. Our results demonstrate that the PTMs disrupt the interaction network at the hCLEC12A–50C1 interface, thereby compromising the stability of the complex. These findings further validate the reliability of our AI model in predicting the functional impact of PTMs. Our study not only elucidates the structural and energetic mechanisms by which PTMs influence protein–antibody interactions from a computational perspective but also provides a theoretical basis for targeted intervention strategies centered around key modification sites. In the future, integration with additional experimental data could further enhance the model’s predictive and applicative capabilities in complex biological systems.

Collectively, these results demonstrate that integrating protein language model representations with structural context enables both accurate PTM site prediction and meaningful downstream functional interpretation. By decoupling feature learning from task-specific design, the proposed framework provides a scalable and extensible foundation for addressing a broad range of biological questions related to protein regulation and molecular interactions.

Materials and Methods

Dataset.

To train and evaluate the proposed framework with early stopping, proteins with available structural information were first collected as a raw protein set and partitioned at the protein level into training, validation, and test subsets at a ratio of 6:1.5:1.5, with all samples derived from the same protein assigned to the same subset. For evaluation, the validation and test sets were filtered against a global nonredundant protein set generated from the entire raw protein collection using CD-HIT at 40% sequence identity, and only results based on the nonredundant datasets are reported in the main text; detailed dataset statistics are provided in SI Appendix, Table S1. Positive samples were experimentally verified lysine PTM sites obtained from the Compendium of Protein Lysine Modifications (CPLM) database (33, 34). Balanced training and validation sets were generated by randomly selecting unmodified lysine residues from the same proteins as negative samples, supplemented when necessary by negatives drawn from other proteins within the same subset, whereas the test set included all lysine residues from test proteins, resulting in a highly imbalanced distribution.

In addition to the above setting, a raw dataset without protein sequence redundancy removal was constructed to better reflect real-world application scenarios. For experiments without a validation stage, proteins were directly partitioned into training and test sets, and both sets were built as balanced datasets using the same positive and negative sampling strategy, enabling a clearer and more stable assessment of the model’s intrinsic discriminative capability; detailed statistics of this dataset are summarized in SI Appendix, Table S2.

Large Protein Language Model.

For protein sequence information, we employed the Large Protein Language Model Evolutionary Scale Modeling-2 (ESM-2) (19) to extract semantic representations. ESM-2 was selected because it provides contextualized residue-level embeddings directly from protein sequences, capturing evolutionary constraints and long-range dependencies without requiring explicit structural inputs. First, we loaded the esm2_t33_650M_UR50D model, a 33-layer Transformer architecture with 650 million parameters, along with its corresponding alphabet system, and obtained the batch_converter tool for sequence transformation. The model was set to evaluation mode to ensure deterministic encoding, disabling stochastic operations such as dropout. For input protein sequences, we formatted them as identifier-tagged tuples, which were then automatically tokenized by the batch_converter. This process included appending special CLS (start) and EOS (end) tokens, mapping amino acid residues to their respective token indices, and converting them into normalized tensors compatible with the model. During forward propagation, we specifically extracted the hidden representations from the final Transformer layer, which outputs a 1,280-dimensional contextual embedding vector for each token position (including special tokens).

Construction of Contact Maps.

Based on the UniProt ID of each protein, we retrieved the structural data from the AlphaFold Database, primarily including the 3D atomic coordinates of each residue. The raw data were stored in .pdb format. From the original data, we extracted the three-dimensional atomic coordinates of the protein residues at their corresponding positions.

To construct a contact map between residues in the protein, we defined multiple contact criteria. For any two atoms A1(x₁, y₁, z₁) and A2(x₂, y₂, z₂), we computed their spatial proximity using the Euclidean distance in three-dimensional space:

$$d\left(A_1,A_2\right)=\sqrt{\left({\left(x_1-x_2\right)}^2+{\left(y_1-y_2\right)}^2+{\left(z_1-z_2\right)}^2\right)}.$$

First, considering the peptide bonds connecting amino acid residues in protein molecules, we used the Euclidean distance between corresponding “C” and “N” atoms in three-dimensional space as edge attributes. Then, if the distance between Cα atoms of corresponding residues was less than 10Å (35), we added contact edges between these two residues, with the edge attribute being the Euclidean distance between the two Cα atoms in three-dimensional space. Simultaneously, we also considered contacts between charged residues glutamate (E), aspartate (D), lysine (K), arginine (R), and histidine (H). When oppositely charged residues had a Euclidean distance less than 16 Å (36, 37), we added contact edges between them and set the corresponding distance as the edge attribute. Finally, after constructing the contact map between residues for each protein, an individual graph was formed.

Subsequently, the distance values in the contact map are standardized to strengthen the subsequent graph neural network model. Standardization scalers for node features and edge attributes are constructed exclusively from the training set and subsequently applied to all subsets of the dataset (training, validation, and test). For node-level feature processing, a masking strategy is used to identify nonzero nodes, ensuring that only valid atomic coordinates are included when computing the mean and SD via StandardScaler, while zero-padded nodes preserve their original values. For edge attributes, a global standardization approach is adopted in which all edge attributes from the training graphs are vertically aggregated to estimate unified normalization parameters. This procedure strictly follows machine-learning data preprocessing protocols by preventing information leakage across data splits, while maintaining the inherent topological structure of each graph. After standardization, the processed node features and edge attributes are converted back into PyTorch tensor format to ensure compatibility with mainstream graph neural network frameworks.

Neural Networks for Learning and Predicting.

The processing of protein structural information is accomplished through a three-layer graph convolutional neural network (GCN) based on spectral graph theory. Let the input graph be denoted as $\mathcal{G}=\left(\mathcal{V},\mathcal{E}\right)$, which contains N nodes with a node feature matrix $X\in R^{N\times 45}$ and an edge index set $\mathcal{E}$. The first-layer GCN performs spatial domain convolution using a weight matrix $W^{\left(1\right)}\in R^{45\times 120}$.

$$H^{\left(1\right)}=ReLU\left(D^{-\frac{1}{2}}A{D}^{-\frac{1}{2}}X{W}^{\left(1\right)}\right).$$

The normalized adjacency matrix $\widehat{A}=A+I_N$ introduces self-loop connections, where $\widehat{D}$ is the corresponding degree matrix. This layer uses the ReLU activation function ReLU(x) = max(0,x) for nonlinear transformation and applies dropout regularization (P = 0.2) to improve generalization. The second GCN layer further reduces features to 60 dimensions:

$$H^{\left(2\right)}=\text{ReLU}\left(D^{-\frac{1}{2}}A{D}^{-\frac{1}{2}}{H}^{\left(1\right)}{W}^{\left(2\right)}\right).$$

The hierarchical dimensionality reduction design (120 → 60) progressively extracts higher-order graph features. The third GCN layer generates 30-dimensional node embeddings:

$$H^{\left(3\right)}=D^{-\frac{1}{2}}A{D}^{-\frac{1}{2}}{H}^{\left(2\right)}{W}^{\left(3\right)}.$$

The process yields a low-dimensional representation $H^{\left(3\right)}\in R^{N\times 30}$. Subsequently, an unsupervised pooling strategy is employed, where global average pooling (GAP) is applied to achieve permutation invariance:

$$h_G=\frac{1}{N}{\sum }_{i=1}^N{h}_i^{\left(3\right)}\in R^{30}.$$

The final output undergoes dimensional expansion to form a tensor structure $h_G\in R^{1\times 1\times 30}$.

The protein sequence representation processing module consists of Multilayer Perceptrons (MLPs) and a Bidirectional Long Short-Term Memory (Bi-LSTM) network. This module first employs a linear transformation to map the input sequence features from 1,280 dimensions to 512 dimensions.

$$x\in R^{1\times 43\times \mathrm{1,280}}\to Linear\left(\mathrm{1,280}\to 512\right)\to \tilde{x}\in R^{1\times 43\times 512}.$$

The sequence data are then processed by a two-layer bidirectional LSTM (Bi-LSTM) network with a hidden state dimension of 256. Since each LSTM layer contains both forward and backward directions, the initial hidden state and cell state tensors have a shape of 4 × batch × 256 (where 4 = 2 directions × 2 layers):

$$h_0,c_0\in R^{4\times 1\times 256}.$$

The bidirectional LSTM module employs gated memory cells (input i_t, forget f_t, output o_t gates with σ activations) to process protein sequences, where ${\mathrm{W}}_{\mathrm{ii}}\in R^{512\times 256}$ and ${\mathrm{U}}_{\mathrm{hi}}\in R^{256\times 256}$ transform inputs and hidden states respectively. The cell state updates as c_t = f_t ⊙ c_t−1 + i_t ⊙ tanh (W_vx_t + U_vh_t−1 + b_v), while h_t = o_t ⊙ tanh(c_t), with ⊙ being Hadamard product. This architecture dynamically regulates feature integration (i_t), memory retention (f_t), and state output (o_t), producing 2 * 256-dimension bidirectional features that are mean-pooled to capture protein sequence patterns across varying scales.

In the implemented two-layer bidirectional LSTM architecture, the computational process can be formulated as:

$$\left(h_t,c_t\right)=LSTM\left(x_t,\left(h_{\left\{t-1\right\}},c_{\left\{t-1\right\}}\right)\right).$$

The output tensor $\%\tilde{x}\in R^{1\times 43\times 512}$ (where 512 = 256 × 2 from bidirectional concatenation) subsequently undergoes mean pooling along the sequence dimension (dim = 1) to generate the global sequence representation:

$$Z=1/43{\mathrm{\Sigma }}^{t=43}{\stackrel{\sim }{X}}_t\Rightarrow z{R}^{1\times 1\times 512}.$$

The output tensors of sequence information and structural information are horizontally concatenated. Subsequently, a neural network architecture consisting of three fully connected layers is employed, incorporating standard regularization and nonlinear activation modules. The network first applies a Dropout layer with a rate of 0.35 for regularization. In terms of feature transformation, the architecture progressively maps the input dimension (256 × 2 + 30) to a 128-dimensional hidden space, then compresses it into a 32-dimensional latent representation and finally projects it to a one-dimensional output space. The first two connected layers are followed by a ReLU activation function to introduce nonlinearity. After removing redundant dimensions via the squeeze(0) operation, the network produces the final prediction output suitable for binary classification tasks.

Training Strategy and Hyperparameter Settings.

The proposed model was trained under a supervised learning framework using the Adam optimizer for parameter optimization. The initial learning rate was set to 3 × 10⁻⁴, and an L₂ regularization term with a weight decay coefficient of 1 × 10⁻⁵ was applied to mitigate overfitting. Considering the variable graph topology and sequence length across samples, model parameters were updated using a per-sample training strategy, which ensured stable optimization and avoided excessive padding-induced noise.

To address the pronounced class imbalance inherent, a balanced focal loss was adopted as the training objective. This loss function extends the binary cross-entropy formulation by dynamically down-weighting well-classified samples while emphasizing harder examples. The focal loss was parameterized with a positive-class weighting factor α = 0.75 and a focusing parameter γ = 2.0, enabling effective handling of skewed class distributions. In addition, a mild label smoothing strategy was incorporated during training to improve numerical stability and generalization, where positive and negative labels were softly adjusted with a smoothing factor of 0.05.

During backpropagation, gradients were computed via automatic differentiation, and gradient norm clipping with a maximum norm of 3.0 was applied to prevent gradient explosion and stabilize optimization. All linear and recurrent layers were initialized using Xavier initialization to facilitate faster convergence and consistent training behavior.

The model was trained for a maximum of 160 epochs. To prevent overfitting, an early stopping strategy based on the Matthews Correlation Coefficient (MCC) evaluated on the validation set was employed. Training was terminated if no improvement in validation MCC was observed for 15 consecutive evaluation steps, and the model parameters corresponding to the best validation MCC were retained for final performance assessment.

All sequence representation extraction using the ESM-2 model and neural network computations were performed on GPU devices to accelerate training, while graph construction, feature standardization, and data preprocessing were conducted on the CPU.

Computational Method.

To assess the impact of acetylation and crotonylation on CLEC12A, MD simulations were employed to characterize the conformational changes of the protein and its binding interface with antibody 50C1. The complex structure of CLEC12A and antibody 50C1 was obtained from the PDB databank (PDB ID: 8W9J). Based on the prediction results, acyl modifications (acetylation and crotonylation) were introduced at residues K174 and K181. A total of six models were constructed: the Pure (unmodified complex), 174Kac (K174 acetylation), 174Kcr (K174 crotonylation), 181Kac (K181 acetylation), 174-181-Kac (both K174 and K181 acetylation), and 174Kcr-181Kac (174 crotonylation and K181 acetylation). All modified residues were treated as nonstandard acid residues during simulations. MD simulations were performed with GROMACS suite of programs, version 2023.1 (38–40). The GAFF force field (41) was applied to parameterize the nonstandard residues N6-acetyllysine (Kac) and N6-crotonyllysine (Kcr) residues. RESP partial atomic charges (42) for these residues were derived from quantum mechanical optimization at the B3LYP/6-311G(d,p) level of theory. Standard residues of the CLEC12A protein were parameterized with the AMBER ff14SB force field (43). Each system was solvated in a cubic box of TIP3P water molecules, maintaining a minimum 10 Å between the protein and box edges (44). Physiological ionic conditions were established by adding 150 mM NaCl to neutralize system charge. Periodic boundary condition was applied along all three dimensions. The initial energy minimization was performed using the steepest descent algorithm. After that, the NVT simulation was performed for 125 ps and followed by the NPT ensemble for 100 ns. The pressure of the models was maintained at 1 bar with a Parrinello–Rahman barostat (45), and the temperature was kept at 310 K by a V-rescale thermostat (46). The electrostatic interactions with the particle mesh Ewald method (47) were treated, and a cutoff distance of 9 Å was used for van der Waals interaction calculation. Hydrogen atoms were constrained using the LINCS (48) algorithm. For each system, the binding free energy was obtained through the Molecular Mechanics Poisson–Boltzmann surface area (MM-PBSA) method (32). Model images were generated and rendered by the Visual Molecular Dynamics (49) and Pymol (50) software. Each system was simulated for 3 times.

Supplementary Material

Appendix 01 (PDF)

Data, Materials, and Software Availability

Code and data have been deposited in GitHub (https://github.com/qi29/lysine-PTM-site-Mining) (51). All study data are included in the article and/or SI Appendix.