MUSE-Net: Missingness-aware mUlti-branching Self-attention Encoder for Irregular Longitudinal Electronic Health Records

Abstract

The era of big data has made vast amounts of clinical data readily available, particularly in the form of electronic health records (EHRs), which provides unprecedented opportunities for developing data-driven diagnostic tools to enhance clinical decision making. However, data-driven modeling of EHRs faces challenges such as irregularly spaced time series, issues of incompleteness, and data imbalance. Realizing the full data potential of EHRs hinges on the development of advanced analytical models. In this paper, we propose a novel Missingness-aware mUlti-branching Self-Attention Encoder (MUSE-Net) to cope with the challenges in modeling longitudinal EHRs for data-driven disease prediction. The proposed MUSE-Net is composed by four novel modules including: (1) a multi-task Gaussian process (MGP) with missing value masks for data imputation; (2) a multi-branching architecture to address the data imbalance problem; (3) a time-aware self-attention encoder to account for the irregularly spaced time interval in longitudinal EHRs; (4) interpretable multi-head attention mechanism that provides insights into the importance of different time points in disease prediction, allowing clinicians to trace model decisions. We evaluate the proposed MUSE-Net using both synthetic and real-world datasets. Experimental results show that our MUSE-Net outperforms existing methods that are widely used to investigate longitudinal signals.

Note to Practitioners—

This article is motivated by the growing need for robust machine learning models capable of handling the complexities of real-world EHRs, including irregular time intervals, missing data, and class imbalance. The proposed MUSE-Net model integrates advanced imputation via MGP with missingness masks, a time-aware self-attention encoder, and a multi-branching framework to enhance predictive robustness. Additionally, MUSE-Net leverages an interpretable multi-head attention mechanism to provide transparent decision-making, allowing clinicians to trace model predictions back to key time points. This framework offers a practical and trustworthy solution for disease prediction and clinical decision support.

I. Introduction

Advancements in sensing and information technology have ushered us into an era of data explosion where a large amount of data is now easily available and accessible in the clinical environment [1], [2], [3]. The wealth of healthcare data offers new avenues for developing data-driven methods for disease diagnosis. For instance, there have been growing research interests in harnessing electronic health records (EHRs) to create solutions for clinical decision support in detecting heart disease [4], sepsis [5], and diabetes [6]. EHRs serve as digital repositories of a patient’s medical information including demographics, medications, vital signs, and lab results [7], [8], [9], curated over time by healthcare providers, leading to a longitudinal database. With rich information about a patient’s health trajectory, longitudinal EHRs present unique opportunities to decipher clinical events and patterns within large populations through data-driven machine learning.

However, data mining of longitudinal EHRs poses distinct challenges due to the observational nature of EHRs. Unlike well-defined, randomized experiments in clinical trials that collect data on a fixed schedule and ensure high data quality, EHRs are recorded only when patients receive care or doctors provide services. The information collected and the timing of its collection are not determined by researchers, resulting in EHRs that are highly heterogeneous and further introducing the following challenges in data-driven decision-making:

(1). Irregularly spaced time series.

EHRs are often documented during irregular patient visits, leading to non-uniform time intervals between successive measurements and a lack of synchronization across various medical variables or among different patients. Traditional time series models face challenges when applied to irregular longitudinal data because they typically assume a parametric form of the temporal variables, making them difficult to effectively account for highly heterogeneous and irregular time intervals across different variables. Additionally, the widely used deep learning models such as convolution neural networks (CNNs) and recurrent neural networks (RNNs) for mining sequential or time series data are designed by assuming consecutive data points are collected at a uniform time interval. Those deep learning models do not consider the elapsed time between records and are less effective in modeling irregular longitudinal EHRs.

(2). Incomplete data and imbalanced class distributions.

EHRs suffer from the issues of missing values and imbalanced data. Due to the nature of clinical practice, not all information is recorded for every patient visit, leading to incomplete datasets. Additionally, EHR data often exhibit a significantly imbalanced distribution, with certain health outcomes or characteristics being underrepresented. For instance, rare diseases or adverse drug reactions [10], [11] may have very few instances compared to more prevalent conditions. In the existing literature, a wide array of statistical and machine learning techniques have been designed to tackle the missing value and imbalanced data issues [12], [13], which, however, are less applicable in the context of modeling irregular longitudinal EHRs. The presence of missing values and imbalanced class distributions will introduce further difficulties in effective model training using longitudinal EHRs, leading to biased or inaccurate predictions if not properly addressed.

To address the challenges presented by longitudinal EHRs, this paper introduces a novel framework of Missingness-aware mUlti-branching Self-Attention Encoder (MUSE-Net) for data-driven disease prediction. First, multi-task Gaussian processes (MGPs) are employed for missing value imputation in irregularly sampled time series. Second, we propose to add missing value masks that record the locations of missing observations as another input stream to our predictive model, which enables the learning of correlations between non-missing and missing values for mitigating the impact of possible imputation errors in the imputation procedure on the prediction performance. Third, we propose to integrate a time-aware self-attention encoder with a multi-branching classifier to address the imbalanced data issue and further classify the irregular longitudinal EHRs for disease prediction. Furthermore, MUSE-Net employs an interpretable multi-head attention mechanism [14] that highlights critical time points in disease prediction, offering transparency in decision-making and enabling clinicians to trace model outputs back to influential time points. This interpretability fosters trust and facilitates the integration of MUSE-Net into real-world clinical applications. We evaluate our proposed framework using both simulation data and real-world EHRs. Experimental results show that our proposed method significantly outperforms existing approaches widely used in current practice.

II. Research Background

Data-Driven Modeling of Non-Longitudinal EHRs:

Extensive research has been conducted to develop data-driven models using non-longitudinal EHRs that consist of static or cross-sectional health information [15], [16], [7], [17]. For example, Huang et al. [18] employed naive Bayes, decision tree, and nearest neighbor algorithms incorporating feature selection methods to determine key factors affecting type II diabetes control and identify individuals who exhibit suboptimal diabetes control status. Hong et al. [19] developed a classification method to identify patients with obesity and various comorbidities using logistic regression, support vector machine, and decision tree. A comprehensive review on machine learning of non-longitudinal EHRs can be referred to [20]. However, those methods are limited in capturing temporal patterns in health status. This limitation can result in less accurate predictions for conditions that are dependent on longitudinal health trajectories [15]. Moreover, traditional machine learning approaches often require manual feature engineering, which is time-consuming and prone to error.

Deep Learning for Longitudinal EHRs:

With the growing availability of longitudinal EHRs, increasing interests have been devoted to developing advanced models to capture temporal information for disease prediction. Owing to the strong capability in pattern recognition, deep learning has been widely explored to mine complexly structured data [21], [22], [23]. Advanced network architectures have been crafted for modeling time series or sequential data. For example, RNNs including long short-term memory (LSTM) are among the most commonly used models to analyze medical time series for various clinical tasks [24], [25]. Additionally, temporal convolutional networks (TCNs) have been recognized as a robust alternative to RNNs for modeling longitudinal signals [6], [5], [26]. However, traditional RNNs and TCNs are designed with the assumption that the records are collected at a constant rate and require the neighboring samples to appear at fixed distances to facilitate the convolution or recurrent operations. This assumption is not valid in many real-world databases, making traditional RNNs or TCNs less effective in modeling irregular-spaced longitudinal EHRs.

Handling Irregularly Sampled Data:

To cope with the issue of irregular time intervals, many modified RNN architectures have been developed. For example, the time-aware LSTM (T-LSTM) was designed to account for non-uniform sampling intervals through a time decay mechanism [27]. Che et al. [28] also implemented a decay mechanism and proposed a GRU-D model, allowing the network to better capture temporal dependencies even when data points were incomplete or irregularly sampled. A comprehensive review of modified RNN architectures for irregular sampled time series can be found in [29]. However, most existing time-aware RNNs assume that the impact of historical risk factors on disease prediction proportionally decays over time, which may not be true in describing complex disease trajectories. Additionally, RNNs have been widely recognized as computationally inefficient for modeling large-scale, long sequential data due to their sequential processing nature.

The self-attention encoder, a cornerstone of the transformer architecture [30], has revolutionized the field of natural language processing (NLP). Unlike RNNs and TCNs that process data sequentially and require uniform sampling intervals, the self-attention mechanism allows the model to weigh the importance of different parts of the input sequences relative to each other. This feature enables parallel processing and enhances the model’s ability to capture long-range dependencies in irregular longitudinal signals. For instance, Li et al. proposed to transform time series data into images and adopted the Vision Transformer to model irregularly sampled time series [31]. Tipirneni and Reddy developed a method that integrated a continuous value embedding technique with self-attention to model irregularly sampled clinical time series by treating them as a set of observation triplets (time, variable, and value) [32]. Huang et al. developed a Deformable Neighborhood Attention Transformer to capture local and global dependencies in medical time series data [33]. Manzini et al. introduced the Diabetic Attention with Relative representation Encoder for Type 2 Diabetes patients [34]. Those self-attention models demonstrate significant improvements over traditional methods, particularly in handling irregular longitudinal datasets.

Missingness and Imbalanced Outcomes in Longitudinal EHRs:

Despite the effectiveness of self-attention encoders, most deep learning models are designed under the assumption of well-structured and regularly sampled data, which makes them less effective for longitudinal EHRs that are typically incomplete, imbalanced, and irregularly collected. In practice, real-world EHR data are notoriously sparse and incomplete (e.g., clinical tests and measurements are not performed at every visit), resulting in substantial missing values in patient trajectories. Importantly, such missingness is rarely random; instead, the patterns of absence themselves carry clinical meaning. For example, the omission of a lab test may indicate that a physician did not suspect a particular condition, while frequent testing of certain biomarkers may reflect heightened clinical concern. Prior studies have demonstrated that missingness patterns in EHRs are informative for predicting patient outcomes, as the presence or absence of measurements often correlates with underlying health status [28], [35].

Traditional imputation strategies, such as forward-fill or mean substitution, tend to ignore inter-variable correlations and temporal dynamics, leading to biased estimates [36]. More advanced approaches, such as multiple imputation by chained equations (MICE) [37] or deep generative models such as GAIN [38], attempt to recover the underlying data distribution more faithfully. Furthermore, specialized architectures such as GRU-D [28] incorporate missingness indicators and time gaps directly into the model, treating absence itself as a predictive signal. Recent works have also proposed variational autoencoders [39], diffusion models [40], and GP-based frameworks [41], [42] to handle sparsity and irregularity in EHRs. While these methods represent significant advances, important gaps remain. For example, MICE are often limited in their ability to model long-range temporal dynamics and personalized missingness patterns [36]. GRU-D incorporates missingness indicators, but relies largely on simplistic decay mechanisms rather than explicitly modeling correlations across variables. Variational autoencoders, GAIN, diffusion models, and GP frameworks typically serve as stand-alone solutions: once imputed, the data are treated as complete, and downstream models rarely account for residual imputation errors.

Beyond missing data, class imbalance represents another critical challenge in longitudinal EHR modeling. Rare clinical outcomes are often underrepresented, leading machine learning models to be biased toward majority classes and to achieve poor sensitivity for minority conditions [13]. While standard strategies such as random over/under-sampling, Synthetic Minority Oversampling Technique (SMOTE), or cost-sensitive learning are widely applied [43], [44], their effectiveness in EHR settings is limited. In particular, naive resampling can result in information loss and biased representation, while cost-sensitive methods often struggle when minority cases are extremely scarce or temporally sparse. More recent approaches have explored ensemble learning [45], focal loss functions [46], and representation learning with tailored regularization [47], [48], yet handling imbalance in conjunction with irregular sampling and informative missingness remains largely unsolved. Effective EHR modeling must therefore jointly address sparsity, irregularity, and imbalance to enable both robust model training and clinically meaningful predictions.

III. Research Methodology

Suppose that there are $N$ patients indexed by $p\in \{1,\dots ,N\}$ and we denote the dataset as $𝒟={\left\{\left({\mathit{t}}_p,Y_p,l_p\right)\right\}}_{p=1}^N$. Each patient is associated with longitudinal data described by the tuple $\left({\mathit{t}}_p,Y_p,l_p\right):{\mathit{t}}_p=\left[t_{p,1},t_{p,2},\cdots ,t_{p,T_p}\right]$ is the time index set for patient $p;Y_p=\left[{\mathit{y}}_{p,1},{\mathit{y}}_{p,2},\cdots ,{\mathit{y}}_{p,M}\right]\in {\mathbb{R}}^{T_p\times M}$ represents the values of $M$ medical variables at ${\mathit{t}}_p;l_p$ is the binary label with $l_p=1$ indicating patient $p$ is positive (e.g., with disease) and otherwise $l_p=0$. The length of the time series for each patient is highly variable (i.e., $T_p\ne T_{p^{\prime }}$, if $p\ne p^{\prime }$), and the times series are often irregularly spaced (i.e., $t_{p,i+1}-t_{p,i}\ne t_{p,j+1}-t_{p,j}$, if $j\ne i$) in longitudinal EHR databases. Additionally, the EHR dataset is often incomplete, and we denote the complete set of values of the $M$ variables as ${\mathit{y}}_p=\text{Vect}\left(Y_p\right)={\left[{\mathit{y}}_{p,1}^T,{\mathit{y}}_{p,2}^T,\cdots ,{\mathit{y}}_{p,M}^T\right]}^T\in {\mathbb{R}}^{T_{p}M}$. Then, we define an index set that contains observed values as $I_o=\left\{(t,m)\mid \text{if}{\left[Y_p\right]}_{t,m}\text{is}\text{observed}\right\}$ and the corresponding values are denoted by ${\mathit{y}}_p^o\in {\mathbb{R}}^{O_p}$. Similarly, we define $I_u=\left\{(t,m)\mid \text{if}{\left[Y_p\right]}_{t,m}\text{is}\text{missing}\right\}$ as the index set of missing values, and the missing value vector is denoted by ${\mathit{y}}_p^u\in {\mathbb{R}}^{U_p}$, where $U_p+O_p=T_{p}M$. Fig. 1 shows the flowchart of the proposed MUSE-Net to classify irregularly spaced and incomplete longitudinal data. Each component in the flowchart is described in detail in the following subsections.

Fig. 1.

(a) MGP for data imputation and missing value masks generation; The imputed data and missing value masks are processed by MUSE-Net, which consist of (b) Missingness-aware Self-attention Encoder with Interpretable Multi-head Attention and (c) Multi-branching output layer.

A. MGP for Missing Value Imputation

A GP is a flexible non-parametric Bayesian model where any collection of random variables follows a joint Gaussian distribution [49], [50], [51], which have been widely used to model complex time series data [52]. GP-based temporal models provide a way to determine the distribution of a variable at any arbitrary point in time, making them intrinsically capable of dealing with missing value imputations that involve irregularly spaced longitudinal data. Note that single-task GPs are limited in their ability to model correlations across multiple related tasks (i.e., different medical variables), making them less effective for modeling multi-variate EHR data. To account for the multi-variate nature, we adopt a multi-task GP (MGP) [53] to capture both variable interactions and temporal correlations for imputing missing values in irregular multi-variate longitudinal EHRs.

We denote $f_{pm}(t)$ as a latent function representing the true values of variable $m$ for patient $p$ at time $t$, and a patientindependent MGP prior is placed over the latent function $f_{pm}(t)$ with a zero mean and the covariance function as:

$$\text{cov}\left(f_{pm}(t),f_{p{m}^{\prime }}\left(t^{\prime }\right)\right)={𝒦}_M\left(m,m^{\prime }\right){𝒦}_t\left(t,t^{\prime }\right)$$

$$y_{pm}(t)~𝒩\left(f_{pm}(t),{\sigma }_m^2\right)$$ (1)

where $y_{pm}(t)$ is the observed value of variable $m$ for patient $p$ at time $t,{\sigma }_{m,}^2$ is the noise term for the $m$th task (variable), ${𝒦}_M\left(m,m^{\prime }\right)$ captures the similarities between tasks, and ${𝒦}_t\left(t,t^{\prime }\right)$ is a temporal correlation function, which is defined as ${𝒦}_t\left(t,t^{\prime }\right)=\text{exp}\left(-\frac{{\left(t-t^{\prime }\right)}^2}{2{\theta }^2}\right)$ with a lengthscale parameter $\theta$. (More discussion on the separable kernel structure can be found in the supplementary material). The prior distribution for the fully observed records, ${\mathit{y}}_p^o$, can be represented by:

$${\mathit{y}}_p^o~𝒩\left(\mathbf{0},{\sum }_p^o\right),{\sum }_p^o={𝒦}_M\odot {𝒦}_{O_p}+E$$ (2)

We formulate the covariance matrix ${\sum }_p^o$ for the observed records as the combination of the Hadamard product (i.e., element-wise multiplication) of ${𝒦}_M$ and ${𝒦}_{O_p}$, and a noise term $E:{𝒦}_M$ is a $O_p\times O_p$ positive semi-definite covariance matrix over medical variables, which is parameterized by a low-rank matrix $B$ as $LB{B}^T{L}^T$ where $B\in {\mathbb{R}}^{M\times q}$, and $L\in {\mathbb{R}}^{O_p\times M}$ is an indicator matrix with $L_{im}=1$ if the $i^{\text{th}}$ observation belongs to task $m$ and $\sum _{m=1}^M{L}_{im}=1;{𝒦}_{O_p}$ is a $O_p\times O_p$ squared exponential correlation matrix over time with elements defined as ${𝒦}_{O_p}\left((m,t),\left(m^{\prime },t^{\prime }\right)\right)={𝒦}_t\left(t,t^{\prime }\right)$ if $(m,t)\in I_o$ and $\left(m^{\prime },t^{\prime }\right)\in I_o;E$ is a noise matrix with $L{\mathit{\sigma }}^2$ on its main diagonal (${\mathit{\sigma }}^2\in {\mathbb{R}}^M$ and ${\left[{\mathit{\sigma }}^2\right]}_m={\sigma }_m^2$).

In the inference step, we impute the missing value for patient $p$ using the posterior mean, ${\mu }_{{\mathit{y}}_p^u}$:

$${\mu }_{{\mathit{y}}_p^u}=\left({𝒦}_{M_{*}M}\odot {𝒦}_{U_p{O}_p}\right){\left({\sum }_p^o\right)}^{-1}{\mathit{y}}_p^o$$ (3)

where ${𝒦}_{M_{*}M}=L_{*}B{B}^T{L}^T\in {\mathbb{R}}^{U_p\times O_p}$ is the task covariance matrix for missing values, and $L_{*}\in {\mathbb{R}}^{U_p\times M},L_{*,im}=1$ if the $i^{\text{th}}$ missing value belongs to task $m;{𝒦}_{U_p{O}_p}$ is the correlation matrix between the time points of the observed and missing values. The MGP hyperparameters include the lower triangular matrix $B$, error term $\text{Diag}(E)$, and the lengthscale $\theta$ in ${𝒦}_{O_p}:𝚯=\{\text{Vect}(B),\text{Diag}(E),\theta \}$, which are learned by minimizing the negative log marginal likelihood given by (with derivative)

$$ℒ(𝚯)=\text{log}\left|{\sum }_p^o\right|+{\left({\mathit{y}}_p^o\right)}^T{\left({\sum }_p^o\right)}^{-1}{\mathit{y}}_p^o+\frac{O_p}{2}\text{log}(2\pi )$$

$$\frac{dℒ}{d𝚯}={\left({\mathit{y}}_p^o\right)}^T{\left({\sum }_p^o\right)}^{-1}\frac{d{\sum }_p^o}{d𝚯}{\left({\sum }_p^o\right)}^{-1}{\mathit{y}}_p^o+\text{Tr}\left({\left({\sum }_p^o\right)}^{-1}\frac{d{\sum }_p^o}{d𝚯}\right)$$ (4)

Finally, the missing values are estimated as ${\stackrel{\mathit{ˆ}}{\mathit{y}}}_p^u={\mu }_{{\mathit{y}}_p^u}$, which are combined with the observed values ${\mathit{y}}_p^o$ to form an imputed dataset $\widehat{𝒟}={\left\{\left({\mathit{t}}_p,{\hat{Y}}_p,l_p\right)\right\}}_{p=1}^N={\left\{\left(\left({\mathit{t}}_p^o,{\text{Vect}}^{-1}\left({\mathit{y}}_p^o\right)\right),\left({\mathit{t}}_p^u,{\text{Vect}}^{-1}\left({\stackrel{\mathit{ˆ}}{\mathit{y}}}_p^u\right)\right),l_p\right)\right\}}_{p=1}^N$, where ${\text{Vect}}^{-1}(\cdot )$ is the inverse operation of $\text{Vect}(\cdot )$. To accelerate MGP Imputation, we adopt the black box matrix-matrix multiplication framework [54] to integrate the modified preconditioned conjugate gradient with GPU acceleration into the data imputation workflow. Please refer to the supplementary material for the detailed procedure for GPU acceleration of MGP imputation.

B. MUSE-Net

We further propose a MUSE-Net to process the imputed longitudinal EHRs. As shown in Fig. 1(b) and (c), this model consists of 3 key modules: time-aware self-attention encoder, missing value masks, and multi-branching outputs.

1). Time-aware Self-attention Encoder:

We propose to adapt the traditional self-attention encoder [30] to incorporate the information of elapsed time between consecutive records in irregular longitudinal EHRs, including three building blocks:

Elapsed time-based positional encoding:

The time order of the longitudinal sequence plays a crucial role in time series analysis. However, this information is often ignored in traditional attention-based encoders because it does not incorporate any recurrent or convolution operations. To address this issue, an elapsed time-based positional encoding is added into the input sequence at the beginning of the self-attention encoder as shown in Fig. 1(b):

$$P{E}_{t_{p,i},2m}=\text{sin}\left(t_{p,i}/{10000}^{2m/M}\right)$$

$$P{E}_{t_{p,i},2m+1}=\text{cos}\left(t_{p,i}/{10000}^{2m/M}\right)$$ (5)

where $t_{p,i}$ is the time position of each observation to account for the irregular time interval, and $2m$ and $2m+1$ are the $(2m)$-th and $(2m+1)$-th variable dimensions, respectively. The sinusoid prevents positional encodings from becoming too large, introducing extra difficulties in network optimization. The embedded elapsed time-based positional features will then be combined with the imputed signals to generate the input of the self-attention module: $X_p=P{E}_p+{\hat{Y}}_p$, where $P{E}_p\in {\mathbb{R}}^{T_p\times M}$ is a matrix with elements defined in Eq. (5).

Intepretable multi-head attention:

The self-attention is a mechanism to allow the network to learn dependencies in the longitudinal data. It maps a query set and a set of key-value pairs to an output. In a single-head self-attention, the key, query, and value matrices, denoted as $K\in {\mathbb{R}}^{T_p\times M},Q\in {\mathbb{R}}^{T_p\times M}$, and $V\in {\mathbb{R}}^{T_p\times M}$, are computed by taking input $X_p$:

$$K={ℱ}_K\left(X_p\right),Q={ℱ}_Q\left(X_p\right),V={ℱ}_V\left(X_p\right)$$ (6)

where ${ℱ}_K(\cdot ),{ℱ}_Q(\cdot )$, and ${ℱ}_V(\cdot )$ represent the network operations to calculate the key, query, and value matrices respectively, which are often selected as linear transformations. The corresponding output, ${\text{Output}}_S\in {\mathbb{R}}^{T_p\times M}$, is computed by applying the scaled-dot production attention:

$${\text{Output}}_S=\text{softmax}\left(\frac{Q{K}^T}{\sqrt{M}}\right)\times V$$ (7)

where softmax $(\mathit{x}{)}_i=\frac{e^{x_i}}{\sum _j{e}^{x_j}}$ outputs the weight assigned to each element in $V$. The dot product in the softmax is scaled down by $\sqrt{M}$, which is essential to prevent the product from growing too large, especially when the dimensionality of $X_p$ is large, introducing instabilities in the training process. The self-attention mechanism enables the network to access all the information of the input with the flexibility of focusing on certain important elements over the entire sequence.

The multi-head attention module is an extension of single-head self-attention to capture different relations among multi-variate time series. Specifically, multiple matrices for the keys, queries, and values are defined by applying multiple network operations, ${ℱ}_K^l(\cdot )$’s, ${ℱ}_Q^l(\cdot )$’s, and ${ℱ}_V^l(\cdot )$’s, to the input:

$$K_l={ℱ}_K^l\left(X_p\right)\in {\mathbb{R}}^{T_p\times \frac{M}{h}},Q_l={ℱ}_Q^l\left(X_p\right)\in {\mathbb{R}}^{T_p\times \frac{M}{h}}$$

$$V_l={ℱ}_V^l\left(X_p\right)\in {\mathbb{R}}^{T_p\times \frac{M}{h}}$$ (8)

where $h$ is the number of attention heads and $l\in \{1,\dots ,h\}$. The output of each attention head is calculated by:

$${\text{Output}}_l=\text{softmax}\left(\frac{Q_l{K}_l^T}{\sqrt{M/h}}\right)\times V_l\in {\mathbb{R}}^{T_p\times \frac{M}{h}}$$ (9)

The final output of the multi-head attention, ${\text{Output}}_M\in {\mathbb{R}}^{T_p\times M}$, is the multiplication of the concatenation of the outputs of all heads and a weight matrix, $W^O\in {\mathbb{R}}^{M\times M}$:

$${\text{Output}}_M=\left[{\text{Output}}_1,{\text{Output}}_2,\cdots ,{\text{Output}}_h\right]W^O$$ (10)

This multi-head attention module enables the network to jointly attend to information from different subspaces of multivariate time series at different time points.

However, traditional multi-head attention suffers from three limitations: (1) Lack of interpretability: the concatenation of head outputs makes it difficult to analyze how each head contributes to the final decision; (2) Independent value matrices: each head has its own $V_l$, leading to inconsistent feature extraction across heads; (3) Extra computation overhead: maintaining separate $V_l$’s increases parameter complexity. To address these issues, we adopt the Interpretable Multi-Head Attention. Instead of maintaining separate value matrices $V_l$, Interpretable Multi-head Attention introduces a shared value matrix $V$ for all heads $V=X_p{W}^V$, where $W^V$ is a single weight matrix applied to the entire input. Additionally, the weighted key-query matrices across all heads are summed as:

$$S=\sum _{l=1}^h\text{softmax}\left(\frac{Q_l{K}_l^T}{\sqrt{M/h}}\right)$$ (11)

Then, the final output is computed by multiplying the aggregated attention score with the shared value matrix $V$:

$${\text{output}}_M=SV{W}^O$$ (12)

This modification improves interpretability by allowing a global attention map $S$ to capture how different query-key interactions influence the shared value matrix. Instead of treating each head’s output separately, interpretable multi-head attention integrates them in a structured and meaningful way.

Feed-forward network:

To stabilize the training process, ${\text{Output}}_M$ will pass through a layer normalization operation [55], and the normalized output will be combined with the original input using skip-connection [56] to generate the output of the “Add & Norm” block (see Fig. 1(b)):

$${\text{Output}}_{AN}=\text{LayerNorm}\left({\text{Output}}_M\right)+X_p$$ (13)

${\text{Output}}_{AN}$ will serve as the input of a feed-forward network with GELU [57] activation and additional “Add & Norm” blocks to further induce nonlinearity degree into the encoder. The multi-head attention and feed-forward modules will be repeated multiple times to improve the generalizability for capturing more informative features of the input sequence.

2). Missing Value Masks:

To further account for potential discrepancies between imputed values and actual observations, the missing value masks (Fig. 1(a)) are incorporated as an additional input to the model. We define the missing value masks for patient $p$ as ${\tilde{X}}_p$ with ${\left[{\tilde{X}}_p\right]}_{t,m}=1$ if ${\left[Y_p\right]}_{t,m}$ is missing; otherwise ${\left[{\tilde{X}}_p\right]}_{t,m}=0$. The masks will be processed by an independent time-aware self-attention encoder in parallel with the imputed sequences as shown in Fig. 1(b). This missingness-aware design enables the network to capture the relationship between non-missing and missing values. As a result, the network is capable of mitigating the effect of potential errors during the MGP imputation process, thereby enhancing the overall predictive performance of the model.

3). Multi-branching Outputs:

The self-attention encoder serves as a powerful feature extractor from irregular longitudinal EHRs for downstream classification. The classifier network, typically with a fully connected layer, is added to interpret the abstracted features and make the final prediction. The imbalanced data issue incurs a more pronounced and direct negative impact on the classifier than on the feature extractor. This is due to the fact that the classifier directly interacts with the labels and is heavily influenced by the majority class during the training process [58]. This leads to a bias towards the majority class, visibly affecting the model’s performance. As such, careful design of the classifier network is needed to cope with the imbalanced data issue.

We propose to incorporate the Multi-branching (MB) architecture [5] (see Fig. 1 (c)) into the classifier to tackle the imbalanced issue. Specifically, in the training phase, the self-attention encoder will be trained with the whole dataset, and each of the MB outputs will be trained with a balanced sub-dataset to mitigate the imbalanced issue. The original imputed dataset $\widehat{𝒟}$ consists of the majority class ${\widehat{𝒟}}_M$, and the minority class ${\widehat{𝒟}}_N$. We create $N_b$ balanced sub-datasets by under-sampling ${\widehat{𝒟}}_M$ to form ${𝒟}_i=\left\{{\widehat{𝒟}}_N,{\widehat{𝒟}}_M^i\right\}$, where the ${\widehat{𝒟}}_M={\widehat{𝒟}}_M^1\cup {\widehat{𝒟}}_M^2\cup \cdots \cup {\widehat{𝒟}}_M^{N_b}$. This under-sampling operation is also applied to the missing value masks. Correspondingly, $N_b$ output branches are created, each aligned with one of the balanced sub-datasets. Thus, each balanced sub-dataset serves as the training data for the respective branch in the output layer, and the self-attention encoder will be optimized by using all the $N_b$ balanced subsets. In the end, the MB output layer will produce $N_b$ predicted probabilities. The optimization process is guided by minimizing the cross-entropy loss function:

$$ℒ(\mathit{\omega };\widehat{𝒟})=-\sum _{p=1}^N\sum _{i=1}^{N_b}I\left(p\in {𝒟}_i\right)\left(l_p\text{log}\left({\hat{P}}^{(i)}\left({\hat{Y}}_p,{\tilde{X}}_p;\mathit{\omega }\right)\right)\right.\left.+\left(1-l_p\right)\text{log}\left(1-{\hat{P}}^{(i)}\left({\hat{Y}}_p,{\tilde{X}}_p;\mathit{\omega }\right)\right)\right)$$ (14)

where $\mathit{\omega }$ is the network parameters; $I(\cdot )$ is an indicator function; ${\hat{P}}^{(i)}\left({\hat{Y}}_p,{\tilde{X}}_p;\mathit{\omega }\right)$ is the prediction by branch $i$ given the input ${\hat{Y}}_p$ and corresponding missing value mask ${\tilde{X}}_p$. The final predicted probability is computed as the average of the $N_b$ predictions: ${\hat{P}}_{MB}\left({\hat{Y}}_p,{\tilde{X}}_p;\mathit{\omega }\right)=\sum _{i=1}^{N_b}{\hat{P}}^{(i)}\left({\hat{Y}}_p,{\tilde{X}}_p;\mathit{\omega }\right)/N_b$, which is a more robust estimator compared to the single-branch (SB) counterpart as stated in Theorem I.

Theorem I: If both MB and SB classifiers are sufficiently trained with the following assumptions:

$$\begin{gathered} {\overline{\hat{P}}}^{(i)}\left(Y_p\right)={\overline{\hat{P}}}_{SB}\left(Y_p\right),\left(i\in \left\{1,\dots ,N_b\right\}\right) \\ {\mathbb{E}}_{\mathit{\omega }}\left[D_{KL}\left({\overline{\hat{P}}}^{(i)}\left(Y_p\right)\Vert {\hat{P}}^{(i)}\left(Y_p;\mathit{\omega }\right)\right)\right]={\mathbb{E}}_{\mathit{\omega }}\left[D_{KL}\left({\overline{\hat{P}}}_{SB}\left(Y_p\right)\Vert {\hat{P}}_{SB}\left(Y_p;\mathit{\omega }\right)\right)\right], \end{gathered}$$ (15)

where ${\hat{P}}_{SB}\left(Y_p;\mathit{\omega }\right)$ is the predicted distribution by the SB classifier for input $Y_p,D_{KL}(\cdot )$ is the Kullback–Leibler divergence, ${\overline{\hat{P}}}_{SB}\left(Y_p\right)={\text{arg}\text{min}}_{P_{*}}{\mathbb{E}}_{\mathit{\omega }}\left[D_{KL}\left(P_{*}\left(Y_p\right)\Vert {\hat{P}}_{SB}\left(Y_p;\mathit{\omega }\right)\right)\right]$, and ${\overline{\hat{P}}}^{(i)}\left(Y_p\right)={\text{arg}\text{min}}_{P_{*}}{\mathbb{E}}_{\mathit{\omega }}\left[D_{KL}\left(P_{*}\left(Y_p\right)\Vert {\hat{P}}^{(i)}\left(Y_p;\mathit{\omega }\right)\right)\right]$, then the variance of the MB classifier is no larger than the variance of the SB classifier, i.e., $V_{MB}\le V_{SB}$, where

$$V_{MB}={\mathbb{E}}_{Y_p~P_{\mathit{\text{data}}},\mathit{w}}\left[D_{KL}\left({\overline{\hat{P}}}_{MB}\left(Y_p\right)\Vert {\hat{P}}_{MB}\left(Y_p;\mathit{w}\right)\right)\right]$$ (16)

$$V_{SB}={\mathbb{E}}_{Y_p~P_{\mathit{\text{data}}},\mathit{w}}\left[D_{KL}\left({\overline{\hat{P}}}_{SB}\left(Y_p\right)\Vert {\hat{P}}_{SB}\left(Y_p;\mathit{w}\right)\right)\right]$$ (17)

Here, $P_{\mathit{\text{data}}}$ denotes the true data distribution and

$${\overline{\hat{P}}}_{MB}\left(Y_p\right)=\text{arg}\underset{P_{*}}{\text{min}}{\mathbb{E}}_{\mathit{\omega }}\left[D_{KL}\left(P_{*}\left(Y_p\right)\Vert {\hat{P}}_{MB}\left(Y_p;\mathit{\omega }\right)\right)\right]$$ (18)

Theorem I shows that the MB classifier is more robust than the SB classifier (see Appendix VIII-B for the theoretical proof).

IV. Experimental Design and Results

We implement our MUSE-Net to investigate both a synthetic and real-world dataset for performance evaluation. MUSE-Net is benchmarked with four models widely used in sequential data mining: LSTM [59], gated recurrent unit (GRU) [60], T-LSTM [27], and TCN [61]. Additionally, we evaluate the performance by different imputation methods, i.e., our MGP+mask, MGP, single GP, GP+mask, mean imputation, and Mean+mask. We also evaluate the impact of branch number on the prediction performance. The dataset is split into training (80%), validation (10%), and testing (10%) sets. The performance is evaluated on four key metrics: area under the receiver operating characteristic curve (AUROC), area under the precision-recall curve (AUPRC), recall, and F1 score. AUROC is a measure of the ability to discriminate between classes. AUPRC is valuable for assessing performance in classifying imbalanced datasets, reflecting the balance between precision and recall. Recall assesses the ability to correctly identify positive instances, while F1 score provides a comprehensive evaluation by accounting for both the sensitivity to positive cases and overall predictive reliability.

A. Experimental Results in Synthetic Case Study

Algorithm 1 shows the process to generate synthetic data from the autoregressive moving average (ARMA) model [62]. Specifically, we first generate three baseline variables by ARMA $\left(p_m,q_m\right)$ with moving average of order $q_m$ and autoregressive of order $p_m:v_{m,t}={\varphi }_{m,1}{v}_{m,t-1}+\cdots +{\varphi }_{m,p_m}{v}_{m,t-p_m}+{\psi }_{m,1}{ϵ}_{m,t-1}+\cdots +{\psi }_{m,q_m}{ϵ}_{m,t-q_m}+{ϵ}_{m,t}$, where $\left\{{ϵ}_{m,t}\right\}$ are uncorrelated random variables for variable $m;{\Omega }_m=\left\{{\varphi }_{m,1},\cdots ,{\varphi }_{m,p_m},{\psi }_{m,1},\cdots ,{\psi }_{m,q_m}\right\}$ is the set of coefficients for variable $m$. We initialize ${\Omega }_m$ randomly from a standard normal distribution and randomly select $p_m,q_m$ from set {1, 2, 3}. We generate the remaining seven variables through feature engineering applied to the three baselines, ensuring correlations exist among the variables and differences exist between different classes. Details on the feature engineering process and the simulation parameters is available in Appendix VIII-A. The generated dataset is a 3-D tensor containing 5,000 samples with 90% from the majority class. Each sample has 10 variables measured across 50 time steps with irregular intervals. Notably, each variable is characterized by a distinct rate of missing values, varying randomly between 30% and 60%, mimicking real-world data complexities.

Algorithm 1.

Synthetic Data Generation Process

Require: Number of time steps: $n\_obs=200$; Number of observations sampled from each time series: $sub\_\mathit{\text{time}}=50$; Number of variables: $n\_\mathit{\text{variables}}=10$; Number of samples to generate: $n\_\mathit{\text{samples}}=5000$; Percentage of the majority samples: $\mathit{\text{percent}}\_\mathit{\text{negative}}=90\%$
1:	Initialize ${\Omega }_m$ for $\text{ARMA}(p_m,q_m)$, $m\in \{1,2,3\}$, and apply feature engineering to generate remaining $\text{ARMA}(p_m,q_m)m\in \{4,5,···,2\times n\_\mathit{\text{variables}}\}$
2:	Generate a label list, labels_list, including $n\_\mathit{\text{samples}}$ binary labels (0 or 1) based on percent_negative
3:	Generate empty 3-D tensor, all_samples, to store the generated data
4:	for $j\leftarrow 1$ to n_samples do
5:	Initialize an empty temporal list: variables ← [ ]
6:	for $m\leftarrow 1$ to n_variables do
7:	if $\mathit{\text{label}}\_\mathit{\text{list}}\left[j\right]$ is 0 (negative instance) then
8:	Generate n_variables time series using $\text{ARMA}(p_m,q_m)$
9:	else
10:	Generate n_variables time series using $\text{ARMA}(p_{m+n\_\mathit{\text{variables}}},q_{m+n\_\mathit{\text{variables}}})$
11:	Add generated time series to variables
12:	Combine variables into a matrix with n_variables columns and add it to all_samples
13:	Initialize an empty tensor: final_dataset
14:	for $i\leftarrow 1$ to n_samples do
15:	subtime_indices ← Maintains the temporal order and randomly selects sub_time unique time steps
16:	sub_ts_i ← all_samples[$i$, subtime_indices, :]
17:	Add generated sub_ts_i to final_dataset
18:	return final_dataset

Fig. 2 shows the AUROC, AUPRC, F1 score, and Recall on validation dataset over training epochs for MUSE-Net-9 (i.e., with 9 branching outputs) using different imputation methods with missing value masks on the validation set of our simulated data. The ‘complete’ scenario represents the case where MUSE-Net is trained on synthetic data without any missing values. Among all methods, MUSE-Net-9+MGP demonstrates the best performance across all four metrics, achieving higher AUROC, AUPRC, F1 score, and Recall, along with the fastest convergence. In contrast, the MUSE-Net-9+GP and MUSE-Net-9+mean methods exhibit slower convergence and lower overall performance, highlighting their limitations in handling missing values effectively.

Fig. 2.

Evaluation scores across epochs for MUSE-Net-9 with different imputation methods on simulated validation data.

Fig. 3 presents AUROC, AUPRC, F1 score, and Recall across training epochs for MUSE-Net+MGP with different numbers of MB outputs on the validation set. Models with a higher number of MB outputs (MB5, MB7, and MB9) demonstrate faster convergence and superior overall performance. This improvement can be attributed to the enhanced class balance within each branching output. When only one MB output is used (MB1), the class imbalance is severe (approximately 9:1), making it difficult for the model to effectively learn from the minority class. In contrast, increasing the number of MB outputs to 9 results in a more balanced class ratio (1:1) for each branch, facilitating better learning from both classes.

Fig. 3.

Evaluation metrics scores across epochs for MUSE-Net with varying MB outputs on MGP-imputed validation data.

The F1 score and Recall in Fig. 3 further emphasize the differences in learning dynamics. Models with more MB outputs exhibit rapid improvements in these metrics, while those with fewer MB outputs (MB1 and MB3) show delayed growth and lower overall performance, particularly in the early training stages. Overall, increasing the number of MB outputs enhances both convergence speed and classification performance, reinforcing the benefits of a more balanced learning framework. An additional synthetic case study, using data generated from a log-Gaussian Cox process, to further evaluate our MUSE-Net model, is provided in the supplementary material.

B. Experimental Results in Real-world Case Study

We further conduct a case study using the Diabetic Retinopathy (DR) dataset, obtained from the 2018 Cerner Health Facts data warehouse [6], [25], [64], to evaluate our MUSE-Net. The selected variables include 21 routine blood tests, 5 comorbidity indicators, 3 demographic variables, and the duration of diabetes. More details on variable selection can be referred to [25], [65]. Notably, the 21 blood tests are all subject to missing values, and the final dataset consists of records from 23,245 diabetic patients, with a minority of 8.9% diagnosed with DR. We use label “1” and “0” to indicate that a patient diagnosed with and without DR, respectively.

Fig. 4 shows the comparison between MUSE-Net and other benchmarks on the validation set. To ensure a fair comparison, all models are designed to have similar sizes, with the number of parameters ~4,000, as shown in Table I (More discussion on the selection of model capacity can be found in the supplementary material). All models utilize the MGP imputation with missing value masks, 10 MB outputs, and Adam optimizer [63]. According to Fig. 4, MUSE-Net outperforms other benchmarks, achieving the highest AUROC and AUPRC on the validation set. This trend is also evident in the results for the test set in Table II, where MUSE-Net maintains AUROC of 0.949 and AUPRC of 0.883, outperforming TCN, LSTM, T-LSTM, and GRU. Specifically, MUSE-Net has an improvement over TCN by 2.3% in AUROC and 5.3% in AUPRC, and an improvement over LSTM by 3.3% in AUROC and 7.1% in AUPRC, an improvement over T-LSTM by 2.4% in AUROC and 6.8%, and an improvement over GRU 1.9% in AUROC and 4.6% in AUPRC. This performance demonstrates that MUSE-Net is more effective at capturing the critical patterns inherent in irregular longitudinal DR data.

Fig. 4.

AUROC and AUPRC across 15 epochs for our MUSE-Net and other benchmarks on the DR validation set.

TABLE I.

Architectures of Muse-Net and other benchmarks for DR dataset.

	Number of model parameters	Number of layers (blocks)	Number of outputs in MB layer	Optimizer
MUSE-Net	4,010	2	10	Adam with decoupled weight decay (AdamW) [63]
GRU	4,333
LSTM	4,686
T-LSTM	4,660
TCN	4,396

TABLE II.

AUROC, AUPRC, Recall and F1 scores for MUSE-Net and other benchmarks on the DR test set.

	MUSE-Net	TCN	LSTM	T-LSTM	GRU
AUROC	0.952	0.935	0.919	0.916	0.932
AUPRC	0.886	0.848	0.825	0.799	0.844
F1	0.750	0.704	0.580	0.665	0.671
Recall	0.850	0.809	0.836	0.781	0.827

Table III presents the performance comparison in AUROC and AUPRC between our MUSE-Net-10 with MGP imputation and other imputation methods on the DR test set. It is worth noting that the MUSE-Net-10 with MGP+masks outperforms other methods, yielding an AUROC of 0.952 and an AUPRC of 0.886, which represents an improvement over GP+masks by 2.0% in AUROC and 7.1% in AUPRC, and an improvement over mean+masks by 2.5% in AUROC and 1.5% in AUPRC. Moreover, consistent performance improvement is achieved across all imputation methods when missing value masks are applied. Specifically, MGP+masks achieves an improvement over the non-mask MGP by 5.7% in AUROC and 10.6% in AUPRC. The results show that the missing value masks enable the model to effectively account for the missingness patterns and mitigate potential discrepancies between imputed and actual values that may arise during the imputation process.

TABLE III.

AUROC and AUPRC comparison of MUSE-Net-10 with MGP and other imputation methods on the DR test set.

Model: MUSE-Net-10	MGP		GP		Mean
	Mask	w/o	Mask	w/o	Mask	w/o
AUROC	0.952	0.901	0.933	0.889	0.929	0.885
AUPRC	0.886	0.801	0.827	0.791	0.876	0.789

Table IV shows the AUROC and AUPRC for MUSE-Net with MGP imputation across various numbers of MB outputs in the test set. MB 1 corresponds to a model with a single output handling the original dataset with an imbalance ratio of approximately 1:10, and MB 10 is a model with 10 outputs, each of which is trained with one of the 10 balanced sub-datasets with a ratio of about 1:1. This table suggests an overall increasing trend in both AUROC and AUPRC when the number of MB outputs increases. The highest AUROC/AUPRC of 0.952/0.886 are achieved with MB 10, demonstrating the enhanced predictive capability of MUSE-Net in a balanced training environment. The performance gradually declines with a decrease in MB number, reaching its lowest AUROC and AUPRC of 0.940 and 0.864 at MB 1. To further demonstrate robustness, we conduct and report a 5-fold cross-validation (CV) for MUSE-Net and other benchmarks. We report the 5-fold CV results in the supplementary material.

TABLE IV.

AUROC and AUPRC scores for MUSE-Net+MGP with different MB numbers in the DR test set.

	MB 10	MB 8	MB 6	MB 4	MB 2	MB 1
AUROC	0.952	0.947	0.946	0.947	0.945	0.940
AUPRC	0.886	0.876	0.877	0.886	0.869	0.864

C. Intepretability Analysis of MUSE-Net in DR Prediction

We further analyze the attention weights learned by the interpretable multi-head attention mechanism to interpret the model’s decision-making process for DR prediction.

1). Averaged Attention Map Across Layers:

Fig. 5 presents the averaged attention maps over all test samples for the first and second layers of MUSE-Net. Each heatmap represents the attention weight distribution between query (y-axis) and key time steps (x-axis), where higher values indicate stronger attention. These maps help clinicians interpret how the model makes predictions by identifying which time steps play a dominant role in decision-making. In the first layer (Fig. 5(a)), the attention distribution is skewed toward the later time, with the highest attention at the final time step. This suggests that the model places greater importance on more recent time steps when making predictions. Earlier time steps receive lower attention, indicating that they contribute less to the decision-making in the initial layer. Clinically, this aligns with the understanding that recent variable changes carry the most relevant information for assessing DR progression.

Fig. 5.

The averaged attention maps over all test samples for the first and second layers of the MUSE-Net: (a) the first layer; (b) the second layer

In the second layer (Fig. 5(b)), attention weights are more evenly distributed. This indicates that, after the first layer refines feature representations, the second layer integrates a more comprehensive view of temporal dependencies, balancing both recent and earlier observations. From a clinical perspective, this suggests that while initial assessments focus on recent trends, a deeper analysis incorporates long-term history to ensure a comprehensive decision-making process. This hierarchical mechanism mirrors how clinicians evaluate patient histories: first prioritizing the most recent indicators, then considering long-term trends for a more thorough assessment.

2). Attention Patterns:

Fig. 6 shows the column-wise sum of attention weights across all steps, providing insight into how much total attention is allocated to each step for different groups. Fig. 6(a) shows the sum of attention weights in the first layer, revealing a clear difference: DR samples exhibit a more evenly distributed pattern across steps. This suggests that the model considers multiple steps when identifying DR, which is clinically relevant because DR progression is a gradual process influenced by long-term trends of many risk factors. Non-DR samples, on the other hand, show a steady increase in attention allocation over time, meaning that recent observations are more predictive of non-DR. This aligns with clinical practice where the absence of warning signs in recent history reduces the disease likelihood. This divergence in attention distribution provides critical insights: for DR patients, historical data remains important in assessing disease progression, whereas for non-DR patients, absence of recent warning signs is a stronger predictor of continued healthy status.

Fig. 6.

The column-wise sum of attention weights across all time steps in the test dataset: (a) the first layer; (b) the second layer

Fig. 6(b) presents the summed attention weights from the second layer. Here, both DR and non-DR samples follow an increasing trend, but a key observation is that the scale of the $y$-axis is smaller than in the first layer, indicating that attention is more evenly distributed. This suggests that each time step retains its importance, ensuring that no single time step dominates the decision-making process for both DR and Non-DR patients. These findings demonstrate that MUSE-Net not only achieves strong predictive performance but also offers clinically interpretable insights by mimicking how human experts evaluate temporal patient data in DR diagnosis.

V. Discussion

Irregular and Event-Driven Longitudinal Data Beyond Healthcare:

The challenges associated with irregular and sparse time series are not unique to healthcare; they are pervasive across multiple domains. For example, manufacturing systems and chemical plants in the process industry generate event logs and sensor readings that are often recorded only under specific operating conditions or when anomalies occur [66], [67], [68]. A closely related example is predictive maintenance in industrial equipment [69], [70], [71], where machines are continuously monitored through sensors measuring vibration, temperature, and other physical parameters. However, these data streams are frequently incomplete due to downtime, irregular reporting schedules, or asynchronous sensor activation. Moreover, failure events (e.g., machine breakdowns) are rare relative to prolonged periods of normal operation, leading to severe class imbalance between “normal” and “fault” states.

In the literature, various machine learning methods such as SVMs, Random Forest, and Gradient Boosting [72], [73], [74], have been developed for analyzing event-based data that consist of irregular sequences of discrete events [75], [71]. More recently, transformer architectures have been adapted to industrial time-series forecasting and anomaly detection tasks [76], [77]. While these advances demonstrate that deep learning models can handle asynchronous, multivariate event data, they typically treat irregular sampling (e.g., via simple resampling), missingness (e.g., via standalone imputation), and class imbalance (e.g., via synthetic oversampling or undersampling) in separate stages. Such decoupled procedures require extensive manual tuning and risk information loss.

Our MUSE-Net is broadly applicable to analyzing event-based data for process monitoring, explicitly addressing the challenges of irregular sampling, missing data, and class imbalance in event-driven time series beyond healthcare. First, the MGP imputation, augmented with missing-value masks, can be applied directly to sensor measurements, enabling effective reconstruction of sporadic observations while exploiting the informative structure of missingness patterns. Second, the time-aware self-attention encoder does not assume uniform sampling intervals, allowing it to capture temporal dependencies in irregular event sequences (whether the “events” are medical visits, machine sensor alerts, or security log entries). Third, the multi-branching architecture tackles the problem of skewed class distributions (e.g., rare machinery faults, fraud cases) by enhancing representation learning for minority classes. MUSE-Net offers a unified solution that ingests raw, asynchronous and incomplete time series, and addresses all challenges within one integrated model. Moreover, the interpretable multi-head attention in our model further highlights the critical time points driving predictions, a feature particularly valuable in domains such as manufacturing and cybersecurity where explainability is important for user trust.

VI. Conclusions

In this paper, we introduce a novel framework: Missingness-aware mUlti-branching Self-Attention Encoder (MUSE-Net) to model irregular longitudinal EHRs with missingness and imbalanced data issues. First, multi-task Gaussian processes (MGPs) are leveraged for missing value imputation in irregularly sampled longitudinal signals. Second, we propose a time-aware self-attention encoder augmented with a missing value mask and multi-branching architectures to classify irregular longitudinal EHRs. Furthermore, an interpretable multi-head attention mechanism is added to the model to highlight critical time points in disease prediction, offering transparency in decision-making and enabling clinicians to trace model outputs back to influential time points. Finally, we evaluate our proposed framework using both simulation data and real-world EHRs. Experimental results show that our MUSE-Net significantly outperforms existing approaches that are widely used to investigate longitudinal signals. More importantly, this framework can be broadly applicable to model complex longitudinal data with issues of multivariate irregularly spaced time series, incompleteness, and imbalanced class distributions.

VIII. Appendix

A. Parameter Specifications in Synthetic Data Generation

Tables V and VI provide the parameter specification in ARMA models and the feature engineering process.

TABLE V.

Parameter specifications of the three baseline arma models

	${\mathit{p}}_{\mathit{m}},{\mathit{q}}_{\mathit{m}}$	${\mathit{\Omega }}_{\mathit{m}}$
ARMA1, $m=1$	(2, 2)	{−0.75, 0.25, 0.65, 0.35}
ARMA2, $m=2$	(1, 1)	{−0.8, 0.5}
ARMA3, $m=3$	(3, 3)	{−0.65, 0.45, −0.2, 0.70, 0.45, 0.25}

B. Robust Classifier from Multi-branching Network Models

This appendix shows the proof of Theorem I.

Definition I (Bregman Divergence): If $F:𝒳\to \mathbb{R}$ is a convex differentiable function, the Bregman Divergence based on $F$ is a function $D_F:𝒳\times 𝒳\to {\mathbb{R}}_{+}$, defined as

$$D_F\left[x_1\Vert x_2\right]\equiv F\left(x_1\right)-F\left(x_2\right)-⟨\nabla F\left(x_2\right),x_1-x_2⟩,x_1,x_2\in 𝒳$$ (19)

Lemma I (Generalized bias-variance decomposition [78]): Let $F:𝒳\to \mathbb{R}$ be a convex differentiable function, $g\left(Y_p\right)$ is the true function, and $f\left(Y_p;\mathit{\omega }\right)$ is the prediction, where $\mathit{\omega }$ is the model parameter, the generalized bias-variance decomposition based on the Bregman divergence $D_F$ is

$${\mathbb{E}}_{Y_p,\mathit{\omega }}\left[D_F\left(g\left(Y_p\right)\Vert f\left(Y_p;\mathit{\omega }\right)\right)\right]={\mathbb{E}}_{Y_p,\mathit{\omega }}\left[D_F\left(g\left(Y_p\right)\Vert \bar{f}\left(Y_p\right)\right)\right]+{\mathbb{E}}_{Y_p,\mathit{\omega }}\left[D_F\left(\bar{f}\left(Y_p\right)\Vert f\left(Y_p;\mathit{\omega }\right)\right)\right]$$ (20)

where $\bar{f}\left(Y_p\right)=\text{arg}\underset{z}{\text{min}}{\mathbb{E}}_{\mathit{\omega }}\left[D_F\left(z\Vert f\left(Y_p;\mathit{\omega }\right)\right)\right]$.

TABLE VI.

Feature engineering to generate time series for each variable.

$\mathit{m}$	Neg. samples (label = 0)	Pos. samples (label = 1)
1	ARMA1	ARMA11 = ARMA1
2	ARMA2	ARMA12 = ARMA2
3	ARMA3	ARMA13 = ARMA3
4	ARMA4 = 0.8 × ARMA1	ARMA14 = 1.1 × ARMA11
5	ARMA5 = ARMA1 + ARMA2	ARMA15 = ARMA11 + ARMA12
6	ARMA6 = ARMA3 + ARMA4	ARMA16 = ARMA13 + ARMA14
7	ARMA7 = ARMA1 + ARMA2 + ARMA3	ARMA7 = ARMA11 + ARMA12 + ARMA13
8	ARMA8 = ARMA1 × ARMA2 + ARMA3 × ARMA4	ARMA18 = ARMA11 × ARMA12 + ARMA13 × ARMA14
9	ARMA9 = ARMA3 + ARMA4 + ARMA1 × ARMA2	ARMA19 = 1.1 × ARMA13 + 1.1 × ARMA14 + ARMA11 × ARMA12
10	ARMA10 = −ARMA5 + ARMA9	ARMA20 = −ARMA15 + ARMA19

Proof. (Lemma I) From the definition of $\bar{f}\left(Y_p\right)$, we have:

$$\begin{gathered} 0={\left.{\nabla }_z{\mathbb{E}}_{\mathit{\omega }}\left[D_F\left(z\Vert f\left(Y_p;\mathit{\omega }\right)\right)\right]\right|}_{z=\bar{f}\left(Y_p\right)} \\ ={\nabla }_z{\mathbb{E}}_{\mathit{\omega }}\left[F(z)-F\left(f\left(Y_p;\mathit{\omega }\right)\right)\right.{\left.\left.-⟨\nabla F\left(f\left(Y_p;\mathit{\omega }\right)\right),z-f\left(Y_p;\mathit{\omega }\right)⟩\right]\right|}_{z=\bar{f}\left(Y_p\right)} \\ ={\left.\left[\nabla F(z)-{\nabla }_z{\mathbb{E}}_{\mathit{\omega }}⟨\nabla F\left(f\left(Y_p;\mathit{\omega }\right)\right),z⟩\right]\right|}_{z=\bar{f}\left(Y_p\right)} \\ ={\left.\left[\nabla F(z)-{\mathbb{E}}_{\mathit{\omega }}\left[\nabla F\left(f\left(Y_p;\mathit{\omega }\right)\right)\right]\right]\right|}_{z=\bar{f}\left(Y_p\right)} \\ \Rightarrow \nabla F\left(\bar{f}\left(Y_p\right)\right)={\mathbb{E}}_{\mathit{\omega }}\left[\nabla F\left(f\left(Y_p;\mathit{\omega }\right)\right)\right]. \end{gathered}$$ (21)

Then, the right-hand side of Eq. (20) can be rearranged as

$$\begin{gathered} {\mathbb{E}}_{Y_p,\mathit{\omega }}\left[D_F\left(g\left(Y_p\right)\Vert \bar{f}\left(Y_p\right)\right)\right]+{\mathbb{E}}_{Y_p,\mathit{\omega }}\left[D_F\left(\bar{f}\left(Y_p\right)\Vert f\left(Y_p;\mathit{\omega }\right)\right)\right] \\ ={\mathbb{E}}_{Y_p,\mathit{\omega }}\left[F\left(g\left(Y_p\right)\right)-F\left(\bar{f}\left(Y_p\right)\right)\right.\left.-⟨\nabla F\left(\bar{f}\left(Y_p\right)\right),g\left(Y_p\right)-\bar{f}\left(Y_p\right)⟩\right]+{\mathbb{E}}_{Y_p,\mathit{\omega }}\left[F\left(\bar{f}\left(Y_p\right)\right)-F\left(f\left(Y_p;\mathit{\omega }\right)\right)\left.-⟨\nabla F\left(f\left(Y_p;\mathit{\omega }\right)\right),\bar{f}\left(Y_p\right)-f\left(Y_p;\mathit{\omega }\right)⟩\right]\right. \\ ={\mathbb{E}}_{Y_p,\mathit{\omega }}\left[F\left(g\left(Y_p\right)\right)-F\left(f\left(Y_p;\mathit{\omega }\right)\right)\right.-⟨\nabla F\left(\bar{f}\left(Y_p\right)\right),g\left(Y_p\right)-\bar{f}\left(Y_p\right)⟩\left.-⟨\nabla F\left(f\left(Y_p;\mathit{\omega }\right)\right),\bar{f}\left(Y_p\right)-f\left(Y_p;\mathit{\omega }\right)⟩\right] \\ ={\mathbb{E}}_{Y_p,\mathit{\omega }}\left[F\left(g\left(Y_p\right)\right)-F\left(f\left(Y_p;\mathit{\omega }\right)\right)\right.\left.-⟨\nabla F\left(f\left(Y_p;\mathit{\omega }\right)\right),g\left(Y_p\right)-f\left(Y_p;\mathit{\omega }\right)⟩\right] \\ ={\mathbb{E}}_{Y_p,\mathit{\omega }}\left[D_F\left(g\left(Y_p\right)\Vert f\left(Y_p;\mathit{\omega }\right)\right)\right]. \end{gathered}$$

□

The bias-variance decomposition for classification analysis with cross-entropy loss is provided in Theorem II:

Theorem II (Bias-variance decomposition for classification tasks): We denote the true class distribution for input $Y_p$ as $P\left(Y_p\right)$ and the predicted distribution given by the classification model as $\hat{P}\left(Y_p,\mathit{w}\right)$, where $\mathit{w}$ is the model parameter set. We assume there are $J$ classes in total. Then, according to the generalized decomposition for Bregman divergence, the bias-variance decomposition for classification tasks is given as

$${\mathbb{E}}_{Y_p,\mathit{\omega }}\left[D_{KL}\left(P(\mathit{x})\Vert \hat{P}\left(Y_p,\mathit{w}\right)\right)\right]={\mathbb{E}}_{Y_p,\mathit{\omega }}\left[D_{KL}\left(P\left(Y_p\right)\Vert \overline{\hat{P}}\left(Y_p\right)\right)\right]+{\mathbb{E}}_{Y_p,\mathit{\omega }}\left[D_{KL}\left(\overline{\hat{P}}\left(Y_p\right)\Vert \hat{P}\left(Y_p;\mathit{\omega }\right)\right)\right].$$ (23)

where $\overline{\hat{P}}\left(Y_p\right)={\text{arg min}}_{P_{*}}{\mathbb{E}}_{\mathit{\omega }}\left[D_{KL}\left(P_{*}\left(Y_p\right)\Vert \hat{P}\left(Y_p;\mathit{\omega }\right)\right)\right]$ and $D_{KL}(P\Vert \hat{P})=\sum _{j=1}^J{p}_j\text{log}\frac{p_j}{{\hat{p}}_j}$ is the Kullback-Leibler (KL) divergence. The first term in Eq. (23) is the squared bias, and the second term captures the variance of the classifier model.

Proof. (Theorem II) Note that minimizing KL divergence is equivalent to minimizing cross-entropy loss in classification tasks. The cross-entropy loss is defined as $H(P\Vert \hat{P})=-\sum _{j=1}^J{p}_j\text{log}{\hat{p}}_j=D_{KL}(P\Vert \hat{P})-\sum _j\left(p_j\text{log}{p}_j\right)$, where $\sum _j\left(p_j\text{log}{p}_j\right)$ is often considered as a constant. Hence, minimizing $H(P\Vert \hat{P})$ is equivalent to minimizing $D_{KL}(P\Vert \hat{P})$. Additionally, KL divergence is a special case of the Bregman divergence. Specifically, if we define the function in Definition I as $F(P)=\sum _i{p}_i\text{log}{p}_i$, then

$$\begin{gathered} D_{KL}\left(P\Vert \widehat{P}\right) \\ =\sum _i\left(p_i\text{log}{p}_i\right)-\sum _i\left({\widehat{p}}_i\text{log}{\widehat{p}}_i\right)-\sum _i〈\text{log}{\widehat{p}}_i+1,p_i-{\widehat{p}}_i〉 \\ =\sum _i{p}_i\text{log}\frac{p_i}{{\widehat{p}}_i}=D_{KL}\left(P\Vert \widehat{P}\right) \end{gathered}$$ (24)

As such, we can use Lemma I for the Bregman divergence to prove Theorem II by defining $F(P)=\sum _i{p}_i\text{log}{p}_i$, and replacing $g\left(Y_p\right)$ by the true distribution $P\left(Y_P\right)$ and replacing the prediction $f\left(Y_p;\mathit{\omega }\right)$ by $\hat{P}\left(Y_p;\mathit{\omega }\right)$. □

Finally, the following shows the proof of Theorem I:

Proof. (Theorem I) From Theorem II, the variances of SB and MB classifiers are (omitting subscripts $Y_p$ for notation convenience)

$$V_{SB}={\mathbb{E}}_{\mathit{\omega }}\left[D_{KL}\left({\overline{\hat{P}}}_{SB}\Vert {\hat{P}}_{SB}(\mathit{\omega })\right)\right],$$ (25)

$$V_{MB}={\mathbb{E}}_{\mathit{\omega }}\left[D_{KL}\left({\overline{\hat{P}}}_{MB}\Vert {\hat{P}}_{MB}(\mathit{\omega })\right)\right]$$

where ${\hat{P}}_{SB}(\mathit{\omega })$ and ${\hat{P}}_{MB}(\mathit{\omega })$ are the predictions provided by the SB and MB classifiers. In the MB model, the predicted probability for class $j$ is ${\hat{p}}_{MB,j}(\mathit{\omega })=\frac{1}{N_b}\sum _{i=1}^{N_b}{\hat{p}}_j^{(i)}(\mathit{\omega })$, where ${\hat{p}}_j^{(i)}(\mathit{\omega })$ is the predicted probability for class $j$ by branch $i$. Given the assumptions in Theorem I, i.e., ${\overline{\hat{P}}}^{(i)}={\overline{\hat{P}}}_{SB}$ and ${\mathbb{E}}_{\mathit{\omega }}\left[D_{KL}\left({\overline{\hat{P}}}^{(i)}\Vert {\hat{P}}^{(i)}(\mathit{\omega })\right)\right]={\mathbb{E}}_{\mathit{\omega }}\left[D_{KL}\left({\overline{\hat{P}}}_{SB}\Vert {\hat{P}}_{SB}(\mathit{\omega })\right)\right]\left(i\in \left\{1,\dots ,N_b\right\}\right)$ if all the branches and the SB model are sufficiently trained, we have

$$\begin{gathered} V_{SB}=\frac{1}{N_b}\sum _{i=1}^{N_b}{\mathbb{E}}_{\mathit{\omega }}\left[D_{KL}\left({\overline{\hat{P}}}_{SB}\Vert {\hat{P}}_{SB}(\mathit{\omega })\right)\right] \\ =\frac{1}{N_b}\sum _{i=1}^{N_b}{\mathbb{E}}_{\mathit{\omega }}\left[D_{KL}\left({\overline{\hat{P}}}^{(i)}\Vert {\hat{P}}^{(i)}(\mathit{\omega })\right)\right] \\ =\frac{1}{N_b}\sum _{i=1}^{N_b}{\mathbb{E}}_{\mathit{\omega }}\left[\sum _{j=1}^J{\overline{\hat{p}}}_j^{(i)}\text{log}\frac{{\overline{\hat{p}}}_j^{(i)}}{{\hat{p}}_j^{(i)}(\mathit{\omega })}\right] \\ ={\mathbb{E}}_{\mathit{\omega }}\left[\sum _{j=1}^J\left({\overline{\hat{p}}}_{SB,j}\text{log}{\overline{\hat{p}}}_{SB,j}-\sum _{i=1}^{N_b}{\overline{\hat{p}}}_{SB,j}\frac{\text{log}{\hat{p}}_j^{(i)}(\mathit{\omega })}{N_b}\right)\right] \\ ={\mathbb{E}}_{\mathit{\omega }}\left[\sum _{j=1}^J\left({\overline{\hat{p}}}_{SB,j}\text{log}{\overline{\hat{p}}}_{SB,j}-{\overline{\hat{p}}}_{SB,j}\text{log}{\left({\Pi }_{i=1}^{N_b}{\hat{p}}_j^{(i)}(\mathit{\omega })\right)}^{\frac{1}{N_b}}\right)\right] \\ \ge {\mathbb{E}}_{\mathit{\omega }}\left[\sum _{j=1}^J\left({\overline{\hat{p}}}_{SB,j}\text{log}{\overline{\hat{p}}}_{SB,j}-{\overline{\hat{p}}}_{SB,j}\text{log}\left(\frac{1}{N_b}\sum _{i=1}^{N_b}{\hat{p}}_j^{(i)}(\mathit{\omega })\right)\right)\right] \\ ={\mathbb{E}}_{\mathit{\omega }}\left[\sum _{j=1}^J\left({\overline{\hat{p}}}_{SB,j}\text{log}{\overline{\hat{p}}}_{SB,j}-{\overline{\hat{p}}}_{SB,j}\text{log}{\hat{p}}_{MB,j}(\mathit{\omega })\right)\right] \\ ={\mathbb{E}}_{\mathit{\omega }}\left[D_{KL}\left({\overline{\hat{P}}}_{SB}\Vert {\hat{P}}_{MB}(\mathit{\omega })\right)\right]\ge {\mathbb{E}}_{\mathit{\omega }}\left[D_{KL}\left({\overline{\hat{P}}}_{MB}\Vert {\hat{P}}_{MB}(\mathit{\omega })\right)\right] \\ =V_{MB} \end{gathered}$$ (26)

where the first inequality is true due to the fact that the geometric mean of nonnegative variables is always less than or equal to the arithmetic mean, and the second inequality is true due to the definition of ${\overline{\hat{p}}}_{MB}$ as given in Eq. (18). As such, the MB classifier is more robust than the SB classifier, i.e., $V_{MB}\le V_{SB}$, as stated in Theorem I. □