scMix: learning temporal dynamics of gene expression under irregular time intervals

Abstract

Motivation

Understanding temporal gene expression is fundamental in the study of cellular development and differentiation. In practice, temporal single-cell datasets tend to contain only a limited number of measured time points, which are often unevenly spaced, resulting in irregular intervals between observations due to experimental constraints. Existing methods typically address these intervals by sequentially predicting one time point after another, yet lack mechanisms to explicitly model time intervals, leading to error accumulation.

Results

In this work, we introduce scMix, a language-model-based framework for predicting single-cell gene expression, which enables prediction from multiple historical time points. We build scMix on the Receptance Weighted Key Value architecture and use its time decay mechanism to model temporal dependencies over time. Moreover, scMix proposes a delta-time mechanism that allows the model to bypass unmeasured time points, reducing error accumulation and improving robustness. In addition, we incorporate a trend regularization strategy to enhance the temporal coherence of predicted gene expression trajectories. scMix demonstrates state-of-the-art performance in predicting gene expression at unmeasured time points, surpassing existing methods, and also achieves outstanding results on downstream tasks.

Availability and implementation

The code used for this study is available at https://doi.org/10.5281/zenodo.18287184.

Graphical abstract

Graphical Abstract.

1 Introduction

Embryonic development, cellular differentiation, and disease progression are dynamic processes driven by gene expression programs that change over time (Raj and Van Oudenaarden 2008, Trapnell et al. 2014, La Manno et al. 2018). Elucidating these processes requires methods capable of characterizing cellular states and capturing their temporal evolution along developmental trajectories. The advent of single-cell RNA sequencing (scRNA-seq) has made this possible by enabling transcriptome-wide measurements at cellular resolution across multiple time points (Trapnell et al. 2014, Klein et al. 2015, Qiu et al. 2017). This technology enables the reconstruction of developmental pathways and lineage relationships, providing valuable insights into the temporal dimension of biology (Raj and Van Oudenaarden 2008, Treutlein et al. 2016).

Unlike conventional time series data, temporal scRNA-seq has distinct characteristics that create specific challenges for analysis. First, these datasets are often sparse, containing only a limited number of time points (La Manno et al. 2018, Saelens et al. 2019, Bergen et al. 2020). Second, some time points may be missing due to experimental constraints, creating gaps in the timeline (La Manno et al. 2018, Lähnemann et al. 2020, Wagner and Klein 2020). In addition, each cell is profiled only once, which prevents longitudinal tracking and eliminates the possibility of one-to-one correspondence between cells at different time points (Raj and Van Oudenaarden 2008, Schiebinger et al. 2019, Weinreb et al. 2020). These characteristics make modeling temporal single-cell data particularly challenging.

Despite the challenges, several methods have been proposed to characterize dynamic gene expression. Diffusion pseudotime (Haghverdi et al. 2016) reconstructs lineage branching by ordering cells along diffusion trajectories, while Slingshot (Street et al. 2018) reconstructs developmental lineages by fitting smooth curves through clustered cells. RNA velocity (La Manno et al. 2018) infers short-term transcriptional changes from spliced and unspliced mRNAs, while scVelo (Bergen et al. 2020) and Dynamo (Qiu et al. 2022) extend this framework by modeling full splicing dynamics and vector fields, enabling transient-state analysis and fate prediction. Although these methods provide valuable dynamic insights, they do not directly predict temporal gene expression.

A variety of deep learning models have been proposed for temporal single-cell gene expression prediction. PRESCIENT (Yeo et al. 2021) models cell differentiation by learning a potential energy function whose gradient field represents cellular dynamics, providing an energy landscape interpretation of trajectories within the Waddington-optimal transport (OT) framework (Schiebinger et al. 2019). This approach is formulated within a stochastic differential equation (SDE) framework. Building on PRESCIENT, PI-SDE (Jiang and Wan 2024) incorporates the Hamilton–Jacobi equation (Bressloff 2021), yielding a physics-informed SDE. MIOFlow (Huguet et al. 2022) combines OT with neural ordinary differential equations (ODEs) (Chen et al. 2018) in a manifold embedding space and employs a Geodesic Autoencoder (Huguet et al. 2022) to preserve geometry. scNODE (Zhang et al. 2024) integrates a variational autoencoder (Kingma and Welling 2013) with neural ODEs and introduces dynamic regularization to maintain temporal consistency in the latent space. Although these methods differ in strategy, they all rely on SDEs or neural ODEs, which have inherent limitations. First, predictions must be computed sequentially, requiring step-by-step progression between time points. Each step depends only on the previous one, preventing the integration of multiple time points and limiting predictive power. Second, the sequential structure prohibits skipping time points: unobserved intermediates must be estimated before later states, causing errors to accumulate as the prediction range increases.

To address the limitations, we propose scMix, which overcomes the constraints of SDE- and ODE-based models and leverages language models for temporal single-cell gene expression prediction. Specifically, we treat gene expression observations at different time points as temporal tokens, and employ the Time-Mixing module of the Receptance Weighted Key Value (RWKV) (Peng et al. 2023) model to integrate information from multiple previous known time points for prediction. The Time-Mixing module is based on a time decay mechanism, which aligns closely with the dynamic process of cell development by modeling the phenomenon in which the influence of early information gradually diminishes over time. In addition, we introduce a novel $\Delta t$ mechanism as an extension to the original RWKV, enabling the model to perceive irregular time intervals. This mechanism enables time point skipping, allowing the model to directly predict future time points while bypassing intermediate unknown ones, thereby reducing error accumulation. Moreover, our model proposes trend regularization to enhance temporal coherence, ensuring smooth transitions between consecutive time points. Experiments demonstrate that scMix significantly outperforms baseline methods in predicting gene expression at unmeasured time points, as well as on downstream tasks.

2 Materials and methods

2.1 Datasets

We evaluate our model on four benchmark temporal scRNA-seq datasets. Each dataset is derived from a distinct species, providing a diverse evaluation setting: Zebrafish (Danio rerio, 12 time points; Farrell et al. 2018), Drosophila (Drosophila melanogaster, 11 time points; Calderon et al. 2022), Schiebinger2019 (Mus musculus, 19 time points; Schiebinger et al. 2019) and Veres (Homo sapiens, 8 time points; Veres et al. 2019).

For datasets Zebrafish, Drosophila, and Schiebinger2019, we followed the preprocessing strategy described in scNODE (Zhang et al. 2024). For each dataset, we selected the top 2000 highly variable genes (HVGs) to retain the most informative expression patterns, and log transformation is applied to the expression. For dataset Veres, the preprocessing method was adapted from the procedure described in PI-SDE (Jiang and Wan 2024). Gene expression was z-score normalized, and features were filtered based on correlations with the marker gene TOP2A, removing genes with Pearson correlation coefficients above 0.15. In our implementation, we further applied HVG selection, retaining the top 100 genes. Detailed data preprocessing procedures are provided in Supplementary S1, available as supplementary data at Bioinformatics online..

2.2 Problem formulation

Let $\{t_1,t_2,\dots ,t_K\}$ denote the observed time points, where each $t_k\in \mathbb{R}$. Since the time points are not evenly spaced, we define the intervals as $\Delta t=\{0,t_2-t_1,t_3-t_2,\dots ,t_K-t_{K-1}\}$, which has length K. At each time point $t_k$, we observe single-cell gene expression profiles ${\mathcal{X}}_{t_k}={\{x_{t_k}^{(i)}\in {\mathbb{R}}^G\}}_{i=1}^{N_k}$, where G is the fixed number of genes and $N_k$ is the number of cells, which varies across time points. Given a target time point $t_{\text{target}}\in \mathbb{R}$, the goal is to characterize the gene expression distribution of the cell population at $t_{\text{target}}$. This target distribution is denoted as ${\mathcal{X}}_{t_{\text{target}}}={\left\{x_{t_{\text{target}}}^{(i)}\right\}}_{i=1}^{N_{\text{target}}},$ where $N_{\text{target}}$ denotes the number of cells observed at the target time point, and is modeled as a function of the observed time points and the corresponding time information:

$${\widehat{x}}_{t_{\text{target}}}=f_{\theta }\left({\left\{x_{t_k},t_k,t_{\text{target}},\Delta t_k\right\}}_{k=1}^T\right).$$

2.3 Delta-RWKV block

2.3.1 Delta-Time Mixing

To account for unevenly spaced time intervals in time series data, we propose Delta-Time Mixing (Fig. 1b), adapted from the RWKV (Peng et al. 2023) framework. This block extends the original Time-Mixing mechanism by explicitly incorporating the time intervals between observations.

Figure 1.

scMix architecture. (a) scMix overall. For each target time point, several preceding visible time points are taken as input, followed by an appended zero mask token serves as output position. The input sequence is first projected by a linear layer and then fused with time information embeddings. The fused representation, together with $\Delta t$, is fed into Delta-RWKV blocks, which outputs a decayed information sequence, where the final mask token is taken as the prediction for the target time point. The Wasserstein distance between predicted and target expression is computed, and the predicted expression is also compared with visible time points to calculate trend regularization loss. (b) Delta-RWKV block. It consists of a Delta-Time Mixing block and a Channel Mixing block. Delta-Time Mixing extends the RWKV Time-Mixing mechanism by incorporating $\Delta t$ into the decay term, enabling the model to account for uneven time intervals. Channel Mixing captures nonlinear interactions across feature channels.

Let $z_t\in {\mathbb{R}}^C$ denote the feature representation at time step t, and let $z_{t-1}$ denote the representation at the previous step. Following the original RWKV (Peng et al. 2023), the receptance ($r_t$), key ($k_t$), and value ($v_t$) vectors are obtained by linearly mixing the current and previous inputs with learnable parameters ${\mu }_r,{\mu }_k,{\mu }_v\in {\mathbb{R}}^C$ and projecting them as

$$r_t=W_r({\mu }_r\odot z_t+(1-{\mu }_r)\odot z_{t-1}),$$ (1)

$$k_t=W_k({\mu }_k\odot z_t+(1-{\mu }_k)\odot z_{t-1}),$$ (2)

$$v_t=W_v({\mu }_v\odot z_t+(1-{\mu }_v)\odot z_{t-1}),$$ (3)

where $W_r,W_k,W_v\in {\mathbb{R}}^{C\times C}$ are learnable projection matrices, $\odot$ denotes element-wise multiplication.

Next, $k_t$ and $v_t$ are used to compute ${\text{wkv}}_t$ using an attention mechanism that aggregates past information:

$$wk{v}_t=\frac{\sum _{i=1}^{t-1}{e}^{-(t-1-i)w·\Delta t_i+k_i}\odot v_i+e^{u+k_t}\odot v_t}{\sum _{i=1}^{t-1}{e}^{-(t-1-i)w·\Delta t_i+k_i}+e^{u+k_t}}.$$ (4)

Here, $w\in {\mathbb{R}}^C$ is a learnable decay parameter applied to past information, and u is a learnable bias term for current information. We apply the time interval $\Delta t$ to the past term w, making larger intervals induce stronger decay and smaller intervals weaker decay.

The weighted aggregation is then scaled by the receptance and linearly projected to produce the final output:

$$o_t=W_o\left(\sigma (r_t)\odot wk{v}_t\right),$$ (5)

where $\sigma (r_t)$ acts as a gating factor applied element-wise to $wk{v}_t$, and $W_o$ projects the gated result to the final output.

2.3.2 Channel Mixing

Channel Mixing (Fig. 1b) complements Delta-Time Mixing to form the Delta-RWKV block, capturing nonlinear feature interactions at each time step. Given input $z_t\in {\mathbb{R}}^C$, channel-level representations are obtained by mixing the current and preceding states through learnable projections:

$$r_t=W_r\left({\mu }_r\odot z_t+(1-{\mu }_r)\odot z_{t-1}\right),$$ (6)

$$k_t=W_k\left({\mu }_k\odot z_t+(1-{\mu }_k)\odot z_{t-1}\right),$$ (7)

where $W_r,W_k\in {\mathbb{R}}^{C\times C}$ are learnable weights and ${\mu }_r,{\mu }_k$ are learnable mixing factors.

The output is then obtained through a gated transformation:

$$o_t=\sigma (r_t)\odot \left(W_v·{(\max (k_t,0))}^2\right),$$ (8)

with $W_v\in {\mathbb{R}}^{C\times C}$. Here, the gating term $\sigma (r_t)$ controls the contribution of each channel, while the squared ReLU term ${(\max (k_t,0))}^2$ introduces sparsity and nonlinearity. Through this mechanism, Channel Mixing enhances the representational capacity of channel features.

2.4 Overall scMix architecture

Given a target time point $t_{\text{target}}$, the model takes as input the four most recent observed time points, denoted as $t_{i-3},t_{i-2},t_{i-1},t_i$, where $t_i$ is the last observation preceding $t_{\text{target}}$. The number of input steps can be adjusted, but experiments on datasets with 8–18 time points show that using four preceding steps provides the best balance between model performance and the ability to capture gene–cell developmental relationships, with further details provided in the Supplementary S3, available as supplementary data at Bioinformatics online. If fewer than four observations are available, the earliest observed time point is used for padding. The input sequence consists of T observed steps, each represented by a gene expression vector $x_t\in {\mathbb{R}}^G$, where G is the number of genes. For a batch of size B, these sequences form a tensor of shape $B\times T\times G$. To predict the expression at $t_{\text{target}}$, we append an all-zero vector after the observed time points, so that the input has shape $B\times (T+1)\times G$. The model output at the appended position is then used as the prediction for $t_{\text{target}}$.

In addition to gene expression, we incorporate time information as a condition. The time information consists of three components: the observed time $t_{\text{obs}}$, the target time $t_{\text{target}}$, and their distance $\Delta t$. Each component is mapped by an embedding function $\varphi (·)$. The resulting embeddings are fused by applying a linear transformation $W_e$ to the aggregated time embeddings and adding it to the linearly projected gene representation $\text{Linear}(x_t)$:

$$z_t=\text{Linear}(x_t)+W_e(\varphi (t_{\text{obs}})+\varphi (t_{\text{target}})+\varphi (\Delta t)).$$ (9)

The fused embeddings ${\left\{z_t\right\}}_{t=1}^{T+1}$ are processed in temporal order through a stack of Delta-RWKV blocks. At each time t, the hidden representation is updated as:

$$h_t=\text{Delta}-\text{RWKV}(z_t,\Delta t_t).$$ (10)

The Delta-RWKV blocks produce hidden states ${\left\{h_t\right\}}_{t=1}^{T+1}$, and the final state $h_{T+1}$, corresponding to the appended zero vector, is used for prediction:

$${\widehat{x}}_{t_{\text{target}}}=h_{T+1}.$$ (11)

2.5 Loss function

2.5.1 Wasserstein distance loss

Since cells across time points are not one-to-one matched, the evaluation focuses on population-level distribution differences. To this end, we employ the Wasserstein distance (Peyré and Cuturi 2019), which quantifies the discrepancy between two distributions. Formally, the p-Wasserstein distance is defined as:

$$W_p({\widehat{X}}_t,X_t)={\left(\underset{\gamma \in \Pi ({\widehat{X}}_t,X_t)}{\inf }\int \parallel u-v{\parallel }^{p}d\gamma (u,v)\right)}^{1/p},$$ (12)

where $X_t$ and ${\widehat{X}}_t$ denote the observed and predicted distributions, respectively. Here, $\Pi ({\widehat{X}}_t,X_t)$ denotes the set of joint distributions between ${\widehat{X}}_t$ and $X_t$. We use the 2–Wasserstein distance $W_2$, approximated with debiased Sinkhorn divergence.

Based on the Wasserstein distance, the loss is decomposed into two parts: prediction loss and reconstruction loss. Prediction loss aligns the predicted and true distributions at the target time point:

$${\mathcal{L}}_{\text{pred}}=W_2\left({\widehat{X}}_{\text{target}},X_{\text{target}}\right).$$ (13)

However, optimizing only the prediction loss may cause the model to overlook the dynamical consistency at intermediate time points. To address this, we introduce the reconstruction loss that aligns the model outputs with the observations at each observed time point, averaged over time $(1/T)$:

$${\mathcal{L}}_{\text{recon}}=\frac{1}{T}\sum _{t=1}^T{W}_2\left({\widehat{X}}_t,X_t\right).$$ (14)

The overall objective of Wasserstein distance loss is:

$${\mathcal{L}}_{\text{wd}}={\mathcal{L}}_{\text{pred}}+{\mathcal{L}}_{\text{recon}}.$$ (15)

2.5.2 Trend regularization loss

In addition to the prediction and reconstruction losses, we introduce a trend regularization loss to regularize the temporal smoothness of the predicted trajectory. Specifically, we define the discrete temporal change rate between the prediction at the target time and the observation at the input time t as:

$$r_t=\frac{{\widehat{x}}_{\text{target}}-x_t}{t_{\text{target}}-t_{\text{obs}}^{(t)}},$$ (16)

where $x_t$ denotes the observed state, ${\widehat{x}}_{\text{target}}$ is the prediction at the target time point, $t_{\text{obs}}^{(t)}$ is the observed time stamp, $t_{\text{target}}$ is the target time stamp. Based on $r_t$, the trend regularization loss is defined as the sum of squared temporal changes over genes and cells across the observed input time points:

$${\mathcal{L}}_{\text{trend}}=\frac{1}{G}\sum _{t=1}^T\sum _{i=1}^N\sum _{g=1}^G{\left(\frac{{\widehat{x}}_{\text{target},i,g}-x_{t,i,g}}{t_{\text{target}}-t_{\text{obs}}^{(t)}}\right)}^2.$$ (17)

Here, G, N, and T denote the numbers of genes, cells, and observed input time points, respectively. This formulation encourages smooth temporal evolution by penalizing large changes in the temporal change rate.

Combining the above components, the overall training objective is defined as

$${\mathcal{L}}_{\text{total}}={\mathcal{L}}_{\text{wd}}+{\lambda }_{\text{trend}}{\mathcal{L}}_{\text{trend}},$$ (18)

where ${\lambda }_{\text{trend}}$ is a balancing coefficient. The coefficient ${\lambda }_{\text{trend}}$ takes different values across datasets or tasks, and can be adjusted (details in Table S3, available as supplementary data at Bioinformatics online).

3 Results

3.1 scMix accurately predicts gene expressions at unseen time points

To evaluate the performance of scMix in predicting gene expression at unseen time points, we design two tasks corresponding to time point splitting strategies: Random Recovery and Forecasting. For Random Recovery, we randomly mask approximately half of the available time points from the dataset and use the remaining time points as input to recover the missing ones. This task evaluates the model’s ability to recover unobserved data from irregular time points. For Forecasting, we select the last 2 or 3 time points of each dataset as the test set. This task evaluates the model’s extrapolation ability. The specific timepoint splits are summarized in Table S2, available as supplementary data at Bioinformatics online.

We first evaluated the Wasserstein distance between the predicted and true gene expression. Each model was trained and evaluated over five independent runs with different random seeds. As shown in Fig. 2, scMix achieved the best results across almost all time points on the four datasets. At only a few time points (e.g. time point 5 of Schiebinger2019 and time point 2 of Veres), the baselines outperformed scMix, but the differences were minimal. Notably, in the Forecasting tasks, our model significantly outperforms all baselines, showing strong extrapolation ability. In addition, the baselines exhibit a lack of generalizability. For example, PRESCIENT performs well on Veres but poorly on Zebrafish. PI-SDE also performs well on Veres but performs poorly on Schiebinger2019. This indicates that the baselines are sensitive to specific data distributions. Furthermore, the baselines exhibit less stable performance. For example, MIOFlow shows large variance on both Schiebinger2019 and Veres datasets, while scNODE also demonstrates high variance on Veres. By comparison, scMix consistently achieves the best performance across all datasets with low variance, demonstrating strong robustness.

Figure 2.

Comparison of Wasserstein distances between scMix and baselines across four datasets (Zebrafish, Drosophila, Schiebinger 2019, and Veres). Each dataset includes two tasks: Random Recovery task evaluates the ability of models to predict gene expression at randomly masked time points, while Forecasting task assesses extrapolation ability of the models. Across all datasets and tasks, scMix achieves lower Wasserstein distances.

We further visualized the results using Uniform Manifold Approximation and Projection (UMAP) (McInnes et al. 2018) to provide a more intuitive comparison between the predicted and target gene expression. Figure 3 shows the performance of different models on the Random Recovery task of the Zebrafish and Veres datasets (The remaining results are in Fig. S1, available as supplementary data at Bioinformatics online). Across all results, the predictions of scMix are consistently closer to the target distributions.

Figure 3.

UMAP was used to project high-dimensional expression to two dimensions for visualization. (a) Zebrafish dataset on Random Recovery task. (b) Veres dataset on Random Recovery task. (c) Individual visualizations of each time point shown in subfigure (b).

3.2 scMix exhibits greater coherence in trajectory prediction

To evaluate the temporal continuity of the predicted trajectories, we followed the experimental design of scNODE (Zhang et al. 2024). Specifically, we focus on the Random Recovery task, where training and predicted time points are combined, and we examine their joint distribution to assess overall temporal continuity.

For the visualization, we applied partition-based graph abstraction (PAGA) (Wolf et al. 2019), a graph-based method that reconstructs the topological structure of cellular populations. PAGA provides a coarse-grained representation of the connectivity between clusters by constructing a graph of nodes and edges, which enables evaluation of whether predicted cells form smooth connections across neighboring time points. In Fig. 4, the training time points (Training Data Only) are shown, with test time points removed; trajectories exhibit clear discontinuities in the PAGA graph (red circles). We then reconstruct cells at the missing test time points using different models. A reasonable prediction is expected to reconnect these broken trajectories and produce a PAGA structure consistent with the target topology. The results show that our model effectively repairs the broken trajectories and produces a reasonable distribution of predicted cells. In contrast, the MIOFlow, PRESCIENT, and PI-SDE still exhibit trajectory breaks after prediction on the Veres and Zebrafish datasets (highlighted by red circles in Fig. 4 and Fig. S2a, available as supplementary data at Bioinformatics online), while some baseline predictions do not show explicit discontinuities but suffer from severe point aggregation, which is also considered unreasonable (highlighted by yellow circles in Fig. 4, Fig. S2b and c, available as supplementary data at Bioinformatics online). PAGA results of Drosophila and Schiebinger2019 datasets are in Fig. S2, available as supplementary data at Bioinformatics online.

Figure 4.

PAGA visualization was used to simulate cell trajectories and assess gene expression coherence. The Training Data Only panel shows the PAGA topology from training time points. For evaluation, predicted test time points were concatenated with training data to assess overall trajectory continuity. This result is from the Veres dataset on Random Recovery task. Trajectory discontinuities are highlighted by red circles, and over-aggregated regions are highlighted by yellow circles.

To further validate the PAGA results, we adopted the IM distance (Saelens et al. 2019) as a quantitative criterion. The IM distance measures the extent to which the topological structure of the predicted data aligns with that of the target distribution, based on their respective neighborhood graphs. A smaller IM distance indicates closer alignment between predicted and reference manifolds, whereas larger values reflect greater structural discrepancies. The metric ranges from 0, indicating identical structures, to 1, representing maximal dissimilarity. As shown in Fig. 5, we performed 5 trials, and our model achieved the lowest average IM distance. (The remaining results are provided in Fig. S3, available as supplementary data at Bioinformatics online.)

Figure 5.

IM scores comparing PAGA trajectory graphs constructed from real and predicted data. The metric measures how well predicted trajectories align with those from real data, with lower values indicating higher consistency.

3.3 scMix can learn gene–cell interactions

To assess whether our model can capture the interaction between genes and cells, we followed the perturbation experiment design introduced in scNODE (Zhang et al. 2024). This experiment is designed to assess whether the model can capture interactions between genes and cells by analyzing changes in cell proportions after gene perturbation. In this experiment, the analysis of the Zebrafish dataset in scNODE identified TBX16 as a regulator of presomitic mesoderm (PSM) cells (Warga et al. 2013) and SOX3 as a factor associated with Hindbrain cells (Dee et al. 2008).

Building on this design, we trained our model on all time points of the Zebrafish dataset and perturbed the expression of TBX16 and SOX3 at the initial time point by scaling their expression levels with multiplicative factors ranging from ${10}^{-2}$ to ${10}^2$, thereby simulating gene knock-out and overexpression. We then evaluated changes in cell proportions at the final time point of the dataset (time point 11 of Zebrafish dataset). As shown in Fig. 6, TBX16 overexpression led to a pronounced increase in the proportion of PSM cells, reaching nearly 60% at the highest perturbation level, whereas knock-out reduced the proportion compared with the control condition. SOX3 overexpression strongly promoted Hindbrain cell differentiation, with the proportion rising to almost 70%, and knock-out decreased the proportion accordingly. However, perturbing SOX3 showed little influence on PSM cells, and TBX16 had negligible effects on Hindbrain. These results indicate that scMix can capture gene–cell relationships, enabling the identification of additional, unknown gene–cell pairs and supporting downstream biological analyses.

Figure 6.

Perturbation experiments illustrating changes in cell proportions after gene perturbations, to examine whether the model captures gene–cell interactions. (a) Hindbrain percentage after SOX3 perturbation. (b) Hindbrain percentage after TBX16 perturbation. (c) PSM percentage after SOX3 perturbation. (d) PSM percentage after TBX16 perturbation. The horizontal axis indicates perturbation strength $({10}^{-2},\hspace{0.17em}{10}^{-1},\hspace{0.17em}{10}^0,\hspace{0.17em}{10}^1,\hspace{0.17em}{10}^2)$, with ${10}^0$ representing no perturbation, and the vertical axis represents the proportion of the corresponding cell type.

3.4 Ablation study

3.4.1 Comparison of multi- and single-time-point input

A key feature of our model is its use of multiple past time points as input. To evaluate its benefit, we conducted an ablation experiment comparing two input settings: (i) using the most recent four observed time points before the target and (ii) using only the most recent single time point.

We report the results as barplots of Wasserstein distance across five runs on Zebrafish dataset, covering Random Recovery and Forecasting tasks. As shown in Fig. 7a, the multi-time point setting outperforms the single time point setting on the two tasks.

Figure 7.

Ablation studies. (a) Ablation study comparing multi-time point and single-time point settings, illustrated by Wasserstein distances. Results on Zebrafish dataset with Random Recovery task (left) and Forecasting task (right). (b) Ablation study of trend regularization. UMAP visualizations are shown for the ground truth (left), the result without trend regularization (middle), and the result with trend regularization (right).

3.4.2 Comparison of with and without trend regularization

To evaluate the contribution of trend regularization, we conducted an ablation study on Zebrafish dataset for Random Recovery task, comparing performance with and without trend regularization. As illustrated in Fig. 7b, the UMAP visualizations highlight that predictions without trend regularization exhibit noticeable discontinuities, whereas the inclusion of trend regularization yields predictions that are more continuous and better aligned with the target distribution. Figure 7b also shows that in several local regions, the predictions with trend regularization better capture fine-grained structures.

4 Discussion

In this work, we proposed scMix, a language-model-based framework for multi-time point temporal gene expression prediction. Unlike existing approaches that rely on SDEs or neural ODEs, our model innovatively leverages language models, thus avoiding their inherent limitations. scMix captures the information decay observed during cell development and incorporates a $\Delta t$ mechanism that enables it to perceive temporal intervals and skip unknown time points. In addition, we introduced trend regularization to enforce smooth transitions across time points and enhance the temporal coherence of predictions.

The results indicate that across all four datasets, scMix achieves lower Wasserstein distances and exhibits greater stability compared with baselines. Moreover, the PAGA results show that the distributions generated by our model in the trajectory recovery task are more consistent with the target distributions, which is further quantitatively validated by the IM values. The perturbation experiments confirm that our model effectively captures the gene–cell relationships. In addition, the ablation experiments demonstrate that incorporating multiple time points substantially reduces the Wasserstein distance of predictions, while the trend regularization contributes to producing more continuous results.

Despite these encouraging results, scMix exhibits a specific limitation. The model is explicitly designed to emphasize temporal continuity in gene expression dynamics, and the trend regularization further enforces smooth transitions across time points. As a result, scMix is particularly well suited for scenarios where cellular states evolve gradually under natural conditions. However, when cells are exposed to strong stimuli, gene expression may exhibit abrupt changes that deviate from smooth temporal evolution, and in such cases, the model may not achieve reliable predictions.

In future work, cell-level state information such as cell cycle or proliferation status could be considered to provide additional context for temporal gene expression dynamics. These factors are known to influence the pace and pattern of expression changes over time, particularly across longer time spans. This provides promising directions for future research on single-cell temporal modeling.

Author contributions

Shangjin Han (Conceptualization [lead], Data curation [lead], Formal analysis [lead], Investigation [lead], Methodology [lead], Software [lead], Validation [lead], Visualization [lead], Writing—original draft [lead], Writing—review & editing [supporting]) and Dongsup Kim (Funding acquisition [lead], Project administration [lead], Supervision [lead], Writing—review & editing [lead])

Supplementary material

Supplementary material is available at Bioinformatics online.

Conflict of interests

No competing interest is declared.

Funding

This work was supported by the National Research Foundation of Korea grants funded by Korean Government [grant numbers RS-2024-00344154, RS-2024-00399520, RS-2025-00523107].

Data availability

All single-cell RNA-seq datasets used in this study are publicly available. Zebrafish dataset can be accessed through the Single Cell Portal (SCP162). Drosophila dataset is available from the GEO database (GSE190147). Schiebinger2019 dataset is available from the Broad Institute WOT tutorial (WOT). Veres dataset can be accessed through GEO (GSE114412).