Peptide turnover prediction using transformer architectures on large-scale time-series proteomic data

Abstract

Background

Protein turnover is essential for maintaining cellular homeostasis and is closely linked to regulatory and disease-related mechanisms. Recent advances in mass spectrometry have enabled high-precision peptide-level measurements; however, identifying sequence-intrinsic signals that determine protein lifespan remains challenging due to the diversity of degradation pathways and cellular variability. Meanwhile, transformer-based protein language models have demonstrated strong capabilities for learning biologically relevant features from amino acid sequences, motivating the development of predictive frameworks for turnover dynamics directly from sequence information.

Results

We developed two architectures that integrate embeddings from a pre-trained protein language model with time-series prediction models: TimeSeq and AASeq. TimeSeq uses averaged sequence embeddings to predict all time points (1–48 h) in a single pass, while AASeq retains residue-level embeddings to enable sequential prediction and attribution analysis. TimeSeq achieved high predictive performance on a large public dataset (R² = 0.809 ± 0.015 on the full dataset; R² = 0.760 ± 0.023 on the sampled dataset). Attribution analysis from AASeq suggested that known degradation-associated features, including PEST-like motifs, may contribute to the model’s predictions, suggesting that the learned representations may capture certain sequence-related determinants of turnover.

Conclusions

This study demonstrates that integrating transformer-derived sequence embeddings with time-series models enables accurate prediction of peptide-level turnover dynamics and may contribute to understanding how sequence-encoded features relate to degradation behavior. The proposed framework provides a foundation for exploring determinants of protein lifespan directly from amino acid sequences.

Supplementary Information

The online version contains supplementary material available at 10.1186/s12864-026-12558-5.

Introduction

Biomolecules, such as proteins, have inherent lifespans. Once a molecule completes its function, it is degraded and replaced with a newly synthesised equivalent. This replacement of equivalent molecules is referred to as protein turnover, which plays a critical role in maintaining the dynamic equilibrium essential for cellular homeostasis [1]. Protein abundance is regulated by a series of finely controlled processes including mRNA transcription, translation, post-translational modifications, and degradation [2]. Understanding turnover is vital for elucidating disease mechanisms and developing novel therapeutic strategies.

Protein degradation proceeds through two primary pathways: the ubiquitin–proteasome system and the autophagy–lysosome system [3]. The ubiquitin–proteasome system is known for its rapid removal of short-lived and abnormal proteins and is involved in cell cycle control and signal transduction [4, 5]. In contrast, the autophagy–lysosome system is responsible for degrading long-lived proteins and even entire organelles, and plays an important role in maintaining cellular energy balance and metabolic regulation [6]. Moreover, certain amino acid sequence motifs (e.g. degrons) act as key signals for protein recognition via these degradation pathways. Although partial insights into these mechanisms have been obtained [7], a universal law or pattern that fully explains protein lifespan is yet to be elucidated. This challenge is due to the complexity of degradation mechanisms and the diversity of cellular environments [8].

Mass spectrometry (MS) has recently been established as a major analytical method for the experimental study of protein turnover. In particular, the development of quantitative techniques using isobaric tags, such as stable isotope labelling with amino acids in cell culture (SILAC) and tandem mass tags (TMT), has enabled large-scale high-sensitivity analyses [9–11].

Recent advances in natural language processing (NLP) have attracted significant attention toward large language models (LLMs) [12]. Protein language models (PLMs), which target protein sequences, have proven useful in various bioinformatics tasks, such as protein function and structure prediction, by efficiently learning evolutionary and structural information from primary amino acid sequences [13–15]. These models treat amino acid residues as fundamental processing units (tokens) and leverage self-supervised learning in large-scale protein sequence databases to capture characteristic patterns that determine protein function and structure.

Although the evolution of sequence-based deep-learning models has revolutionised the prediction of protein structure and function, their application in predicting dynamic protein properties, particularly turnover, remains underexplored. High-precision time-series data obtained through SILAC-TMT and mass spectrometry allows for detailed tracking of protein synthesis and degradation dynamics. However, a comprehensive understanding of lifespan-determining signals in protein amino acid sequences remains insufficient.

Traditional kinetic approaches generally approximate protein turnover as a first-order exponential process and derive a single rate constant (k). While such models provide a concise quantitative description, they inherently assume a homogeneous and time-invariant turnover behavior. However, experimental observations often reveal multi-phase or non-linear turnover dynamics driven by factors such as post-translational modifications, cell-cycle–dependent regulation, and subcellular localization.

Therefore, rather than summarizing the process into a single constant, modeling the temporal trajectory of turnover itself may be useful for capturing the biological diversity of protein stability.

In this study, we sought to integrate protein feature representations extracted using PLMs with deep learning architectures specialised for time-series prediction to identify protein lifespan signals in an information-driven manner. First, we focus on building a predictive model for turnover dynamics.

Methods

Dataset

In this study, we used a peptide-level protein turnover MS dataset measured and collected using SILAC-TMT [16] (PXD023218). The dataset provided the grouping of protein entries. When a peptide sequence matches multiple proteins, the proteins are pre-grouped and the first protein in each group serves as a representative. The dataset includes two key metrics for each peptide over 10 time points (0 h, 1 h, 3 h, 6 h, 10 h, 16 h, 24 h, 34 h, 48 h, and infinity): “Label loss” (the proportion of heavy isotope label lost as existing components are degraded) and “Label incorporation” (the proportion of newly synthesised proteins incorporating heavy isotope labels). Label loss decreased over time and approached 0, whereas label incorporation increased over time and approached 1. Because the ratio variables are inherently bounded between 0 and 1, clipping was applied to remove the influence of outliers.

Two types of datasets were used for the analysis: the whole dataset, which included all data, and a sampled dataset, in which sampling was performed to correct for biases in the turnover patterns present in the complete dataset. For the sampled dataset, peptides were randomly sampled from each cluster to match the size of the smallest cluster (C0: Very Rapid) to address class imbalance across turnover rate clusters. For all experiments, data splitting for cross-validation was performed at the protein level, using the representative UniProt [17] accession as the grouping unit. Within each fold, the data were split such that all peptides derived from the same protein were always assigned together to a single subset (training, validation, or test) and were never distributed across multiple subsets. The resulting peptide counts in each split across all folds are summarised in Supplementary Table S1. To further assess potential homology leakage beyond exact protein-level separation, we quantified sequence similarity between training and test proteins within each fold using MMseqs2 [18] clustering at 90% and 50% sequence identity thresholds, and reported the overlap ratio per fold. Time-series k-means clustering [19] was performed on the whole dataset to assign turnover-pattern cluster labels. The elbow method was used to determine the optimal number of clusters. After observing diminishing returns in the within-cluster variance and considering the interpretability of the clusters, we set the number of clusters to 5. Based on these cluster labels, we then constructed the sampled dataset by randomly extracting an equal number of peptides from each of the five clusters (matching the size of the smallest cluster), thereby avoiding bias toward specific turnover patterns.

Peptide-level turnover prediction architectures

In this study, we developed two architectures to predict peptide-level turnover dynamics: peptide-level turnover prediction using time-sequential modelling (TimeSeq) and peptide-level turnover prediction using amino acid-sequential modelling (AASeq) (Fig. 1). Both architectures were designed as regression tasks to simultaneously predict two processes: label loss and label incorporation.

Fig. 1.

Architecture for peptide-level turnover prediction. The pre-processing component is integrated with various prediction components

Common pre-processing component

The pre-processing component extracts the feature representations of peptide sequences that account for the protein context using the ESM2 model [14]. The full-length amino acid sequence of each peptide was retrieved using the UniProt ID. The ESM2 model was applied to the full-length protein sequences to generate embedded representations. For each amino acid residue in the full-length sequence, a 1,280-dimensional embedding vector was generated. Subsequently, the embeddings corresponding to the target peptide were extracted from the full-length embeddings. This approach enabled the acquisition of feature representations that consider not only the local characteristics of a peptide, but also its context within the entire protein. This full-sequence encoding followed by peptide-window extraction corresponds to the Full-Sequence Window Embeddings (FSWE) strategy, which formalizes context-aware window representations derived from whole-protein embeddings [20]. We adopted this approach to retain long-range contextual signals that may influence local degradation behaviour.

Formally, let the amino acid sequence of a protein be represented as an ordered list of length :

The residue-level embeddings produced by ESM2 are defined as:

For a peptide corresponding to the subsequence interval [𝑠, 𝑒] on the protein, the peptide-level residue embeddings are extracted as:

TimeSeq: comprehensive prediction using time-series models

TimeSeq is based on common time-series prediction methods and is designed to accommodate various predictive components, such as long short-term memory (LSTM) [21] and transformer-based models [22, 23] (Fig. 1). These models take feature representations for each amino acid obtained from the pre-processing component and average them to create a 1,280-dimensional feature vector. Formally, the residue-level peptide embeddings are aggregated by mean pooling to obtain a peptide-level representation:

This feature vector was then used to predict values at eight experimentally measured time points.

from 1 h to 48 h, allowing the models to predict label loss and label incorporation at each time point. (Note that the 0 h and infinite time points were excluded from the prediction because they were used for comparison with the architecture, as discussed later).

The model predicts label loss and label incorporation values at the experimentally measured time points:

The model output is defined as:

Here, the two output channels correspond to label loss and label incorporation, respectively.

This formulation allows TimeSeq to perform multi-timepoint regression in a single forward pass without explicitly modelling residue-level temporal dependencies.

AASeq: design focused on interpretability

AASeq uses a design that prioritises interpretability at the amino acid residue level. Specifically, two types of input vectors were prepared. One combined the feature representation for each amino acid residue with the lag features of label loss, and the other combined it with the lag features of label incorporation. These input vectors were fed separately into the model, where they were processed to predict a target for the next time point by one-step regression.

Formally, let denote the residue-level embedding matrix of a peptide with residues and embedding dimension . At each time step , let and represent the lag feature vectors of label loss and label incorporation, respectively, where .

To condition residue-level representations on past turnover dynamics, the lag features are broadcast to all residues and concatenated with the peptide embeddings to form the following inputs:

Here, 1_m denotes an all-ones column vector of length m, indicating that the lag feature vector is broadcast identically across all residues.

Each input is processed independently by a Transformer-based encoder, producing residue-wise latent representations and . These outputs are aggregated by mean pooling across residues to predict the next-step values:

This formulation enables AASeq to perform one-step-ahead prediction while explicitly linking temporal turnover dynamics to individual amino acid residues, thereby enhancing interpretability at the residue level. Because residue-wise representations are explicitly conditioned on past turnover dynamics, the contribution of individual residues can be examined through residue-level activations or attention weights.

Transformer-based models

The Transformer-based models that we employed rely on a Transformer Encoder and can be regarded as simple, multitask learning models. Specifically, inputs (𝑥), representing peptide-level feature vectors or residue-wise feature sequences, are projected into a common embedding dimension, augmented with positional encoding, and then passed through a Transformer Encoder (which includes a self-attention mechanism). The outputs from this encoder are fed into multiple output heads to simultaneously predict multiple tasks.

In TimeSeq, a single input is passed through the embedding and positional encoding layers and then through self-attention, ultimately yielding predictions of label loss and label incorporation at all eight measured time points (1–48 h). By contrast, AASeq processes two inputs in parallel, applying embedding and self-attention to each before producing outputs for the two tasks. Neither model introduces specialised modules beyond the standard transformer architecture; therefore, both can be classified as relatively simple multitask time-series forecasting models.

Baseline models for comparison

To provide a reference point for performance evaluation, we also constructed baseline models using standard machine learning algorithms. Specifically, we implemented multi-output regression with Random Forest and XGBoost [24]. The models were trained to predict Turnover ratio values across eight time points simultaneously, using either (1) Amino Acid Composition (AAC), which represents each peptide by counts and relative frequencies of its constituent amino acids, or (2) ESM2-derived embeddings averaged across peptide residues. These baselines were used to assess the benefit of integrating advanced sequence embeddings and time-series architectures.

In addition to classical machine-learning baselines, we evaluated a persistence baseline for Turnover ratio, defined as (ŷ(t + 1) = y(t)), which predicts the next time point by carrying forward the previous observation. In single-step time-series prediction, high predictive accuracy can often be achieved by exploiting trivial temporal continuity alone. This baseline was therefore introduced to quantify the contribution of temporal continuity independent of sequence-dependent information.

Training for prediction tasks

As label loss and incorporation exhibit complementary dynamics, a composite loss function that combines the prediction errors for both metrics was used. Specifically, the loss function was defined as follows and the model was trained to minimise this loss:

where L_loss and L_{incorporation} represent the prediction errors (Mean Squared Error, MSE) for label loss and incorporation, respectively. This loss function aims to optimise the prediction accuracy for both metrics simultaneously. To prevent overfitting, L2 regularisation was applied. Additionally, the rectified linear unit (ReLU) activation function was adopted in the hidden layers to improve computational efficiency.

For all neural network models, major hyper-parameters such as learning rate, hidden dimensions, number of layers, and batch size were chosen based on preliminary experiments on the training data and then kept fixed across all folds (see Supplementary Table S2 for details). All experiments were conducted in a 10-fold cross-validation setting. In each fold, as described in the Dataset section, the data were split at the protein level using the representative UniProt accession as the grouping unit, and approximately 70%, 20%, and 10% of the proteins were assigned to the training, validation, and test subsets, respectively with exact proportions varying slightly due to protein-level grouping. The validation set was used only for early stopping and selection of the best epoch, whereas the held-out test set was used once per fold solely for final performance evaluation. Nested cross-validation was not performed, and the test split was never used for hyper-parameter tuning or model selection.

Evaluation

To analyse label loss and incorporation in an integrated manner, a new metric called the Turnover ratio was introduced and calculated using the following formula:

In this study, we evaluated our proposed architectures (TimeSeq/AASeq) using the Turnover ratio, whereas the classical baselines were evaluated on Turnover ratio to align with standard regression setups.

The prediction accuracies of Turnover ratio were measured using the following evaluation metrics: The root mean squared error (RMSE) is a metric that evaluates the magnitude of the difference between the predicted and observed values, representing the prediction accuracy of the model.

The coefficient of determination (R²) indicates how well the prediction model explains variance in the data.

Using these metrics, the overall performance of the model was evaluated.

Physicochemical property analysis

Sequence-derived physicochemical properties were computed for each peptide using Biopython, including peptide length, net charge at pH 7.4, hydrophobicity (GRAVY [25]), instability index, aromaticity, isoelectric point (pI), and amino-acid composition fractions (positively/negatively charged and polar/nonpolar residues). Differences across turnover-pattern clusters (C0–C4) were assessed using the Kruskal–Wallis test, and effect sizes were quantified using η². For prediction-accuracy analysis, peptides were divided into high- and low-accuracy groups based on the median peptide-wise RMSE, and group differences were evaluated using the Mann–Whitney U test.

Gene ontology (GO) enrichment analysis

We first selected the top 10% of peptides by predicting R² in each turnover ratio cluster.

Primary UniProt accessions were used for queries of g: Profiler [26] (API v.1.0.x, Ensembl [27] 110, 28 May 2025) against the GO [28] (Biological Process (BP), Molecular Function (MF), Cellular Component (CC)) and Kyoto Encyclopedia of Genes and Genomes (KEGG) databases [29], with the *hsapiens* (Homo sapiens) background.

Statistical significance was assessed using the built-in hypergeometric test with g: SCS multiple testing correction (adjusted p < 0.05). No hierarchical filtering was applied; therefore, both parent and child terms were retained. The redundant terms have been consolidated in the text and figures.

Plotting conventions (term ordering, marker size, and colour scheme) are described in the caption of Fig. 3. All code and raw g: Profiler outputs are available in Supplementary Tables S3–S6.

Fig. 3.

Integrated analysis of biological signatures and motif contributions to prediction accuracy. Panels A–B show the 13 most significant terms per cluster, ordered by − log₁₀(p-value). Bubble area encodes the number of associated genes, and bubble colour denotes biological category (blue, Cell Cycle; yellow, organelle/lumen; green, nucleotide/metabolic; grey, other). A. GO enrichment analysis comparing peptides from the “Very rapid” and “Very slow” turnover clusters. Dot size indicates gene count; colour represents statistical significance (− log₁₀(p-value)). B. GO enrichment analysis comparing peptides with high vs. low prediction accuracy within “Very rapid” cluster (C0 cluster). Dot size and colour coding are consistent with panel A. C. Analysis of dependency on the PEST motif within the “Very rapid” cluster. The upper panel shows the correlation between motif attribution scores (means) and peptide-wise R² values. The lower panel demonstrates the relationship between motif attribution and the relative drop in R² after masking motif regions

Results and discussion

Before presenting the modelling results, we first outline the rationale for predicting full trajectories instead of relying on a single kinetic constant k. Classical turnover analyses often fit a single first-order exponential function to label loss or label incorporation and report the derived degradation rate constant k. Although convenient, this approach inherently assumes simple first-order behaviour. In practice, however, MS-based proteomic measurements of protein turnover—including SILAC–TMT experiments—are shaped by multiphase dynamics, regulatory pathways, subcellular transport, and experimental noise. Consequently, peptide trajectories frequently deviate from a single-exponential form, and the estimated k becomes highly sensitive to the choice of kinetic model and fitting window. To avoid this model-dependence and better capture the underlying temporal patterns, we trained models to reproduce the full-time courses of label loss and label incorporation. In addition, we utilised the Turnover ratio as a bounded, model-independent summary of these dynamics. This ratio reflects both the degradation of pre-existing proteins and the accumulation of newly synthesised proteins, is naturally restricted between 0 and 1, and can be computed directly from observed or predicted trajectories without assuming any particular kinetic model. We therefore use the Turnover ratio not as a redefinition of “turnover rate” but as a practical, assumption-free representation of temporal behaviour.

Dataset characteristics, cross-validation setup, and time-series k-means analysis

The datasets used in this study included information regarding peptides and their associated proteins (Table 1), as well as statistical data for label loss and label incorporation at each time point (Fig. 2A). For both the whole and sampled datasets, we performed K-fold cross-validation (K = 10) to ensure robust evaluations. To ensure a biologically meaningful data split, we partitioned the proteins based on their UniProt IDs into training, validation, and test sets at a ratio of approximately 7:2:1, respectively, within each fold. All of the peptides derived from a given protein were assigned exclusively to a single set. Thus, within each fold, there was no redundancy of proteins or peptides between the training, validation, and test subsets. As shown in Supplementary Table S1, this approach resulted in a comparable distribution of peptide counts across the splits, with training, validation, and test sets containing approximately 69%, 20%, and 11% of the peptides, respectively.

Table 1.

Dataset characteristics

	Peptides	Proteins
Whole Dataset	38,361	6,194
Sampled Dataset	12,640	4,424

Number of peptides, associated proteins

Fig. 2.

Characteristics of dataset and clustering results from time series k-means. A. Statistical distribution of label loss, incorporation, and Turnover ratio at each time point in the input dataset. B. Representative patterns for each cluster obtained through time-series clustering (50 peptides were sampled from each cluster)

Time-series K-means clustering was performed based on the similarity of the time-series data shapes, allowing the capture of temporal variation patterns that are otherwise difficult to identify using traditional Euclidean distances. Clustering revealed a significant imbalance in the number of samples across the clusters (Table 2).

Table 2.

Characteristics of each cluster

Cluster	Avg. Turnover ratio	Initial slope	Label
0	0.717	0.089	Very rapid
1	0.555	0.058	Rapid
2	0.450	0.038	Moderate
3	0.369	0.027	Slow
4	0.286	0.018	Very slow

Avg. Turnover is the mean Turnover ratio across all time points, the initial slope is the rate of change between the first two measurements, and the label denotes the descriptive speed category

For each cluster, we calculated the average Turnover ratio across all time points and the initial slope of the turnover curve within the first 6 h (Table 2). Based on these characteristics, each cluster was labelled according to its dynamic pattern, as shown in Fig. 2B; Table 2.

In addition to protein-level splitting, we assessed sequence homology between the training and test sets using MMseqs2 on the whole dataset. The overlap ratio—defined as the proportion of test proteins clustered together with training proteins—was 0.42 ± 0.13% at 90% sequence identity and remained limited even at 50% identity (8.91 ± 1.76%) (Supplementary Table S7A). Furthermore, removing test peptides derived from these overlapping proteins resulted in only negligible changes in Turnover ratio prediction performance, with ΔR² < 0.004 across all models (Supplementary Table S7B). These results indicate that the train–test splits used in this study are not substantially affected by sequence homology, and that subsequent analyses are conducted under conditions largely free from homology-related bias.

Comparison of input feature and baseline models

As described in Sect. 2.2.5, we constructed baseline models using Random Forest and XGBoost to provide a reference point for evaluating our proposed architectures. These models were trained to predict Turnover ratio trajectories across eight time points using either Amino Acid Composition (AAC) or embeddings generated by the pre-trained ESM2 model.

As shown in Table 3, when the Random Forest model was used, the AAC-based predictor achieved an RMSE of 0.152 ± 0.005 and R² of 0.030 ± 0.008, whereas the ESM2-embedding model achieved an RMSE of 0.148 ± 0.005 and R² of 0.074 ± 0.020, thus outperforming the AAC-based approach. Moreover, this performance is in the same range as that reported [30] for protein-level degradation rate prediction (mean test R² ≈ 0.062 for the leukaemia dataset and ≈ 0.129 for the yeast dataset), demonstrating that our peptide‐level models provide a competitively accurate baseline for dynamic behaviour prediction.

Table 3.

Performance of baseline models using AAC vs. ESM2 embeddings

Model	Feature Type	RMSE	R²
Random Forest	AAC	0.152 ± 0.005	0.030 ± 0.008
	ESM2 Embedding	0.148 ± 0.005	0.074 ± 0.020
XGBoost	AAC	0.153 ± 0.005	0.010 ± 0.011
	ESM2 Embedding	0.148 ± 0.005	0.073 ± 0.036

These values represent the Turnover ratio prediction performance (RMSE and R²) of each model (Random Forest and XGBoost) using the two input feature types (AAC and ESM2 embeddings)

Performance of TimeSeq

Using TimeSeq, we conducted a comparative evaluation of LSTM and Transformer-based prediction models.

On the whole dataset, the Transformer-based model slightly outperformed the LSTM, and both models achieved R² = 0.809 ± 0.015 (Transformer) and R² = 0.802 ± 0.018 (LSTM) for Turnover ratio, indicating high predictive performance (Table 4). Cluster-specific analysis revealed that all clusters other than Cluster 0 achieved R² ≥ 0.71, whereas Cluster 0 yielded a negative R², underscoring its prediction difficulty.

Table 4.

Performance evaluation of TimeSeq model across datasets and clusters

Dataset			LSTM		Transformer Based
	ALL/Cluster(C)	Peptides	RMSE	R2	RMSE	R2
Whole	ALL	38,361	0.128 ± 0.006	0.802 ± 0.018	0.126 ± 0.006	0.809 ± 0.015
	C0 Very Rapid	2,528	0.298 ± 0.03	−0.136 ± 0.227	0.305 ± 0.009	−0.177 ± 0.066
	C1 Rapid	10,729	0.126 ± 0.029	0.808 ± 0.080	0.122 ± 0.012	0.825 ± 0.033
	C2 Moderate	15,068	0.136 ± 0.022	0.799 ± 0.060	0.125 ± 0.009	0.833 ± 0.022
	C3 Slow	7,413	0.071 ± 0.01	0.929 ± 0.020	0.052 ± 0.002	0.963 ± 0.002
	C4 Very Slow	2,623	0.114 ± 0.03	0.752 ± 0.136	0.125 ± 0.009	0.716 ± 0.042
Sampled	ALL	12,640	0.154 ± 0.011	0.746 ± 0.033	0.15 ± 0.008	0.760 ± 0.023
	C0 Very Rapid	2,528	0.22 ± 0.033	0.371 ± 0.199	0.237 ± 0.019	0.281 ± 0.117
	C1 Rapid	2,528	0.091 ± 0.017	0.900 ± 0.042	0.079 ± 0.006	0.927 ± 0.011
	C2 Moderate	2,528	0.121 ± 0.015	0.842 ± 0.040	0.105 ± 0.006	0.883 ± 0.011
	C3 Slow	2,528	0.114 ± 0.026	0.813 ± 0.090	0.091 ± 0.01	0.885 ± 0.026
	C4 Very Slow	2,528	0.175 ± 0.043	0.417 ± 0.300	0.172 ± 0.014	0.465 ± 0.085

These values represent the Turnover ratio prediction performance (RMSE and R²) of the LSTM- and transformer-based models (TimeSeq) for each dataset (Whole, Sampled) and Cluster

On the sampled dataset, overall R² decreased to R² = 0.760 ± 0.023 (Transformer) and 0.746 ± 0.033 (LSTM). However, Cluster 0 improved markedly from − 0.177 ± 0.066 to 0.281 ± 0.117, and Cluster 1 increased from 0.825 ± 0.033 to 0.927 ± 0.011. By contrast, Cluster 4 declined to 0.47. These findings suggest that the low accuracy observed in specific clusters may stem from a data imbalance (Table 4).

Performance of AASeq

In AASeq, we implemented and evaluated a single-step regression using AASeq (Table 5). Single-step regression predicts the next time point using observed values from previous time points. This approach suppresses the accumulation of prediction errors but requires the sequential acquisition of observed values in practice, making it unsuitable for real-world deployment. However, it is nevertheless useful for analysing the amino acids that contribute the most to each prediction.

Table 5.

Performance of single-step regression (AASeq) model across datasets and clusters

Dataset			Transformer Based
	ALL/Cluster(C)	Peptides	RMSE	R2
Whole	ALL	38,361	0.090 ± 0.007	0.902 ± 0.017
	C0 Very Rapid	2,528	0.111 ± 0.009	0.869 ± 0.022
	C1 Rapid	10,729	0.079 ± 0.008	0.886 ± 0.025
	C2 Moderate	15,068	0.085 ± 0.008	0.901 ± 0.020
	C3 Slow	7,413	0.092 ± 0.007	0.902 ± 0.015
	C4 Very Slow	2,623	0.126 ± 0.009	0.798 ± 0.032
Sampled	ALL	12,640	0.075 ± 0.003	0.940 ± 0.005
	C0 Very Rapid	2,528	0.085 ± 0.006	0.923 ± 0.012
	C1 Rapid	2,528	0.059 ± 0.008	0.936 ± 0.017
	C2 Moderate	2,528	0.060 ± 0.008	0.950 ± 0.013
	C3 Slow	2,528	0.066 ± 0.003	0.950 ± 0.005
	C4 Very Slow	2,528	0.097 ± 0.019	0.880 ± 0.018

These values represent the Turnover ratio prediction performance (RMSE and R²) of the transformer-based single-step regression model (AASeq) evaluated for each cluster on both the Whole and Sampled datasets

Our experimental results showed that the single-step regression model achieved R² = 0.902 ± 0.017 for the whole dataset and R² = 0.940 ± 0.005 for the sampled dataset, demonstrating high predictive accuracy in both settings. Notably, the sampled dataset yielded an improved R² = 0.923 ± 0.012 for Cluster 0 (very rapid), which boosted the overall model performance. These findings suggest that balancing sample counts across clusters stabilises the training on minority patterns and further enhances predictive accuracy.

To assess performance beyond trivial temporal continuity, AASeq was compared with a persistence baseline on the sampled dataset. The persistence baseline achieved R² = 0.821 ± 0.003 for turnover ratio prediction. AASeq significantly outperformed this baseline (R² = 0.940 ± 0.005; paired t-test, p = 6.46 × 10⁻¹³), corresponding to a 42.0% reduction in prediction error (RMSE: 0.129 → 0.075). These results demonstrate that AASeq captures additional predictive information beyond that explained by temporal autocorrelation alone (Supplementary Table S8).

Analysis of inputs

In our predictive models, both the LSTM- and Transformer-based architectures exhibited higher accuracy for Cluster 1 and lower accuracy for Cluster 0. Motivated by this trend, we quantitatively evaluated the cluster separability of peptide-embedding spaces produced by ESM2. After mean-pooling the token embeddings of each peptide, we computed the silhouette score [31] and compared the average values for each cluster (Table 6). The results indicated that the mean silhouette score for every cluster remained < 0.01, revealing no clear clustering structure. Nevertheless, the models still achieved satisfactory R² values for each cluster, suggesting that subtle, informative features are embedded in the high-dimensional space that silhouette analysis alone does not capture.

Table 6.

Mean silhouette scores for each cluster in the ESM2 embedding space

Cluster(C)	Silhouette Score
C0 Very Rapid	–0.005
C1 Rapid	–0.014
C2 Moderate	–0.053
C3 Slow	–0.107
C4 Very Slow	0.004

Silhouette scores were computed on mean-pooled peptide embeddings in the original ESM2 high-dimensional feature space; scores near zero indicate minimal cluster separation and negative scores indicate overlapping clusters

ESM2-derived peptide embeddings reflect only sequence-based information. In contrast, post-translational modifications (PTMs) are known to strongly affect protein stability and degradation dynamics. However, because the aim of this first-step framework was to assess the predictive contribution of the amino-acid sequence itself, peptides annotated with PTMs were excluded. Separating PTM-free and PTM-bearing peptides also provides a future advantage, as it may enables a clear estimation of how much turnover behaviour is explained by sequence-intrinsic features versus additional contributions arising specifically from PTMs. At the same time, excluding PTM-annotated peptides biases the dataset toward more constitutive, less-regulated turnover behaviours. Proteins whose stability is primarily controlled by PTM-dependent mechanisms—such as ubiquitination- or phosphorylation-triggered degradation, endoplasmic reticulum quality control, or lysosomal transport—are therefore likely under-represented in the present analysis. More broadly, turnover regulation and degradation capacity can vary across cellular contexts (e.g., different cell types or in vivo environments), which may further limit direct transferability beyond the source setting. Consequently, the proposed models should be interpreted as capturing the sequence-encoded component of turnover dynamics, and their predictive performance may be reduced for peptide classes whose degradation is dominated by regulatory processes beyond the primary amino acid sequence. Addressing this limitation will require PTM-aware and context-aware extensions, for example by incorporating PTM type and site information and relevant cellular metadata as additional inputs and evaluating model performance in a stratified manner across modification categories and biological contexts.

Comprehensive analysis of Cluster-level biological signatures and sequence motif contributions to predictive modelling

Regarding physicochemical properties, we examined whether the turnover-pattern clusters (C0–C4) were associated with basic sequence-derived features at the peptide level, including length, net charge (pH 7.4), hydrophobicity (GRAVY), instability index, and amino-acid composition. Although multiple properties showed statistically significant differences across clusters (Kruskal–Wallis test), the effect sizes were uniformly small (η² ≤ 0.02 for all properties), indicating that the cluster structure is not primarily driven by simple physicochemical features (Supplementary Fig. S1; Table S9). We observed similarly modest differences when comparing high- vs. low-accuracy peptides (Supplementary Fig. S1; Table S9).

To relate prediction accuracy to protein function, we performed GO enrichment analysis on high-accuracy (top 10% prediction R² values) from two contrasting turnover-ratio clusters: “Very rapid” and “Very slow”.

The results showed that peptides from the “Very rapid” cluster (C0 cluster) were significantly enriched in structural and regulatory factors associated with the cell cycle, including terms such as “mitotic cell cycle”, “spindle”, and “chromosome”. In contrast, peptides from the “Very slow” cluster (C4 cluster) predominantly featured stable proteins associated with structural integrity and intracellular metabolism, including GO terms such as “organelle” and “nucleotide-binding” (Fig. 3A). These terms are consistent with proteins known to have relatively long lifespans because they often contribute to stable intracellular structures or enzymatic complexes involved in cellular homeostasis [32–35].

Furthermore, when comparing groups with different prediction accuracies within the same cluster (Cluster 0), the high-accuracy group was enriched with cell cycle-related terms, whereas the low-accuracy group selectively featured proteins localised to the “organelle lumen” and “membrane-enclosed lumen”, which are highly dependent on transport pathways and post-translational modifications (Fig. 3B). As these lumen-localised proteins rely heavily on non-sequence factors, such as endoplasmic reticulum (ER) -associated degradation and lysosomal transport mechanisms, it has been suggested that accurate lifespan prediction based solely on primary sequence inputs is challenging for this class of proteins [36, 37].

To validate this hypothesis, Integrated Gradients from Captum [38] were applied to the AASeq model to quantify the contributions of individual input amino acids. Peptides containing a well-known degradation signal, the PEST motif [39, 40], were extracted from the “Very rapid” cluster. Considering the reported density-dependent functionality of the PEST motif [41], this motif was defined as a region with consecutive and densely packed Pro (P), Glu (E), Ser (S), and Thr (T) residues. A positive correlation was observed between motif residue contributions and peptide-wise R² scores, although this was not statistically significant (Spearman ρ = 0.14, p = 0.27; Fig. 3C, upper panel). Subsequently, when motifs in highly accurate peptides (top 10% R²) were masked by replacing them with the average embedding, a statistically significant trend emerged: peptides with higher motif contributions exhibited greater reductions in prediction accuracy (ΔR² = original R² − masked R²; Spearman ρ = 0.51, p = 0.035; Fig. 3C, lower panel). This suggests that the AASeq model effectively recognised and utilised the PEST motif as a crucial determinant of protein lifespan.

In conclusion, the AASeq model demonstrated high generalisation performance for short-lived proteins explicitly containing degradation signals in their primary sequences (such as cell cycle-related proteins). Conversely, the predictive accuracy tended to decrease for lumen proteins whose stability was strongly influenced by factors beyond their primary sequence.

Conclusion

In this study, we propose a novel approach that integrates representations obtained from models pre-trained on biological features with time-series prediction models. We demonstrated that although peptide-level turnover prediction in a time-series framework has rarely been attempted, our method achieves far higher accuracy than baseline models designed in accordance with prior work surrounding protein degradation rate prediction in the same domain.

While our goal—identification of biological “lifespan signals”—has not yet been realised, we have successfully built a prototype dynamic turnover prediction model, which was the first milestone outlined in the Introduction.

Among our methods, TimeSeq can predict eight time points, from 1 h to 48 h, in a single pass and delivers stable performance across multiple architectures, including LSTM and Transformer. In particular, Transformer-based TimeSeq achieved high predictive accuracy (R² = 0.809 ± 0.015 on the whole dataset and R² = 0.760 ± 0.023 on the sampled dataset), confirming the effectiveness of integrating sequence-derived and temporal information. AASeq, which retains per-residue feature representations, performed regression predictions and, through attribution analysis, revealed associations with known turnover control mechanisms, such as PEST motifs. Moreover, our analysis of input features revealed that despite the absence of obvious predictive signals in the raw inputs themselves, the model appeared to learn relevant turnover information from high-dimensional embeddings that reflect biological properties.

Nevertheless, several technical challenges remain, including the trade-off between practical applicability and interpretability and the need to generalise across diverse turnover patterns. To overcome these limitations, we will systematically gather more-varied datasets and develop novel model architectures to enhance predictive performance. Ultimately, these efforts will bring us closer to identifying protein lifespan signals. In future work, we plan to incorporate additional biological features, such as protein structural data and post-translational modification states, while maintaining a balance between usability and interpretability, and to perform experimental validation to confirm the biological relevance of our predictions and deepen our understanding of protein turnover mechanisms.

Authors’ contributions

KI conceived the study, performed analyses, and drafted the manuscript. ACY contributed to method design and manuscript revision. YL implemented models and visualizations. SO supervised the project and revised the manuscript. All authors read and approved the final manuscript.

Funding

This work was supported by JSPS KAKENHI [Grant Number 23H04924].

Data availability

The mass spectrometry proteomics dataset re-analysed in this study is available in the PRIDE partner repository [42] under accession PXD023218. The source code and processed data to reproduce the analyses are available at https://github.com/stoneyakoya/PTP.

Declarations

Ethics approval and consent to participate

Not applicable. This study did not involve human participants, human data/tissues, or live animals.

Consent for publication

Not applicable. The manuscript does not contain data from any identifiable individual.

Competing interests

The authors declare no competing interests.