PhysioJEPA: Joint Embedding Representations of Physiological Signals for Real Time Risk Estimation in the Intensive Care Unit

Abstract

Self-supervised learning of multi-modal, high-frequency physiological signals is largely unexplored, despite its potential for critical care applications. We present PhysioJEPA, a Joint Embedding Predictive Architecture (JEPA) designed for multi-modal physiological signals from critical care bedside monitoring devices. PhysioJEPA learns representations from 30-minute segments of physiological signals from three channels: arterial blood pressure, electrocardiography lead II, and photoplethysmography. Trained on over 10.7 million minutes of data from 4,282 intensive care unit stays (N=2,631 patients) in the Medical Information Mart for Intensive Care-III (MIMIC-III) Waveform Database, the learned, frozen representations of PhysioJEPA can be used to estimate 5-minute risk of hypotension (AUROC = 0.83 [Confidence Interval or CI 0.83–0.84]) and shock index (AUROC = 0.95 [0.95–0.96]), with comparable performance to a self-supervised Patch Time Series Transformer framework (AUROC = 0.87 [0.86–0.87] and 0.96 [0.96–0.96]), better performance compared to another JEPA physiological signal model, ECG-JEPA (AUROC = 0.73 [0.72–74] and 0.92 [0.92–0.93]), and better performance compared to a supervised convolutional model (AUROC = 0.78 [0.78–0.78] and 0.95 [0.95–0.95]). Notably, it can generalize to an independent healthcare system (AUROC = 0.78 [0.78–0.78] and 0.92 [0.92–0.93]) better than all comparison models. These results suggest that self-supervised JEPA representation learning is a promising approach for multi-modal bedside monitoring signal data.

1. Introduction

Physiological time series data contain information-rich health data, capturing continuous measurements of vital signs, organ function, and metabolic processes that provide real-time insight (Orphanidou, 2019; Rooney and Clermont, 2023). From electrocardiograms (ECG) and electroencephalograms (EEG) in hospital settings to continuous glucose monitoring and wearable sensor data in ambulatory care, these high-frequency signals contain complex temporal patterns that reflect underlying pathophysiological processes. The ability to effectively model and learn from such time series data is crucial for enabling early detection of adverse events and supporting clinical decision-making (Orphanidou and Wong, 2017; Rooney and Clermont, 2023).

Following the advancements of the attention mechanism (Vaswani et al., 2017) and subsequent work with bidirectional encoder representations from transformers (BERT) (Devlin et al., 2019) and chat generative pretrained transformers (ChatGPT) (Brown et al., 2020), self-supervised representation learning has transformed approaches to modeling complex, high-dimensional data across domains. In healthcare specifically, foundation models trained on large-scale datasets have demonstrated remarkable capabilities in medical imaging (Zhou et al., 2023), cardiac magnetic resonance imaging videos (Kim et al., 2024), clinical notes (Wornow et al., 2023; Huang et al., 2019; Singhal et al., 2022), and multi-modal medical data (Moor et al., 2023a,b). However, physiological time series present unique challenges that distinguish them from other medical data modalities: they are inherently temporal and often multivariate with complex interchannel dependencies, characterized by high sampling frequencies that generate massive datasets.

Recent developments in self-supervised learning have demonstrated that Joint Embedding Predictive Architectures (JEPA) can learn superior representations by predicting in the embedding space rather than reconstructing raw input data, particularly in imaging and video data (Assran et al., 2023; Bardes et al., 2024). Unlike traditional masked autoencoding approaches that reconstruct values or contrastive approaches that compare augmented embeddings, JEPA uses a context encoder to embed random parts of an input (i.e., patches of an image, intervals of a time series). Then, a predictor network uses these context embeddings to estimate other, non-overlapping parts of the input in the embedding space that have been encoded by a separate target encoder. This target encoder, updated via exponential weighted moving average, generates ground-truth embeddings for comparison. This embedding-space prediction framework is hypothesized to avoid noisy pixel reconstruction while learning higher-level representations, leading to more robust and generalizable features (Assran et al., 2023). For physiological time series, this approach is especially compelling as it circumvents the reconstruction of inherently noisy signal data while potentially capturing complex temporal and cross-channel dependencies that are critical for clinically relevant tasks. Thus, we hypothesize that JEPA offers better representation learning for multi-modal physiological signals by modeling predictive relationships within the embedding space and across signal channels, rather than reconstructing inputs or contrasting samples’ embeddings.

In this work, we introduce PhysioJEPA, a JEPA-based model for multi-modal physiological signals from bedside monitoring devices in intensive care units (ICU). PhysioJEPA learns representations from 30-minute segments of three-channel bedside monitoring data (arterial blood pressure [ABP], ECG lead II, and photoplethysmography [PPG]) through self-supervised training. This approach enables accurate risk estimation of critical care outcomes via task-specific non-linear probing (i.e., fitting a supervised classifier on top of frozen representations).

Our work makes the following key contributions:

• We present the first application of JEPA to bedside monitoring data, extending JEPA principles to handle 30-minute segments of three-channel physiological time series (ABP, ECG, PPG) sampled at 125 Hz.

• We develop channel-specific mask token prediction strategies, employ rotary positional embeddings, and use depthwise convolutions for tokenization, enabling the model to learn distinct representations for each physiological signal type over time while capturing cross-channel dependencies.

• We show that PhysioJEPA is effective in critical care risk estimation for 5-minute hypotension risk and 5-minute shock index risk, matching or outperforming supervised and self-supervised comparison models.

• We demonstrate that PhysioJEPA can robustly generalize to an external dataset without retraining, outperforming performance of supervised and self-supervised comparison models and establishing the potential for real-world ICU deployment.

2. Related Work

2.1. Time Series Representation Learning

Self-supervised representation learning for time series has seen significant development, with frameworks ranging from contrastive learning (Yue et al., 2022) to masked autoregressive models like Patch Time Series Transformer (PatchTST) (Nie et al., 2023). In healthcare specifically, current approaches for physiological signals have made important advances but still typically rely on shorter or lower frequency input sequences, single-channel or single-modality analysis, or domain-specific architectures that limit their generalizability across different physiological monitoring scenarios (Xu et al., 2024; Foumani et al., 2024; Kotoge et al., 2024; Chien et al., 2022; Ding et al., 2024; Zhang et al., 2022; Lutsker et al., 2024). This is a limitation, given that many clinically relevant patterns in physiological data manifest over extended time horizons and require integration of information across multiple signal types.

More recent works including SleepFM (Thapa et al., 2024), NormWear (Luo et al., 2024), PFTSleep (Fox et al., 2025), and wearable work by Merrill and Althoff (2022), Narayanswamy et al. (2024), and Xu et al. (2025) build representation models for higher frequency, multi-modal data streams in the sleep or wearable domains; however, none have been developed for or applied to bedside monitoring data.

2.2. Joint Embedding Predictive Architectures

Building on the foundational JEPA framework (Assran et al., 2023), recent work has explored extensions beyond the original image domain applications. Video-JEPA has demonstrated effectiveness for temporal sequence understanding by adapting the embedding-space prediction paradigm to handle temporal dynamics in video data (Bardes et al., 2024). In the physiological signal domain, studies have applied JEPA principles to ECG analysis, showing promising results on 10-second, 250–500Hz, 8 or 12-lead ECG segments for feature prediction and downstream classification tasks (Kim, 2024; Weimann and Conrad, 2025). Further, Signal-JEPA applies dynamic spatial attention to improve generalization across datasets, showing promise for spatially distributed signals such as EEG (Guetschel et al., 2024). These applications highlight JEPA’s particular advantages for noisy signal modalities; however, these applications to physiological data have been limited to short-duration, single-modality scenarios.

2.3. Physiological Signal Analysis for Critical Care

Traditional approaches to ICU risk estimation have relied heavily on aggregated features from electronic health records, signals, or vital signs (Maheshwari et al., 2021; Yoon et al., 2020; Moghadam et al., 2021; Cherifa et al., 2021; Kapral et al., 2024; Hatib et al., 2018; Jian et al., 2025). While these methods achieve reasonable performance on single tasks, they fail to leverage the rich temporal dynamics present in continuous bedside monitoring. Others have taken advantage of the continuous data for supervised classification tasks (Lee et al., 2021; Moon et al., 2024; Jeong et al., 2024; Jo et al., 2022; Sundrani et al., 2023). To our knowledge, our work is the first to apply self-supervised learning to bedside monitoring signal data for multi-outcome critical care risk estimation.

3. Methods

3.1. Data Extraction

We used the Medical Information Mart for Intensive Care-III (MIMIC-III) Waveform Database Matched Subset (Johnson et al., 2015, 2016). Waveform data with the ABP, ECG lead II, and PPG signal channels were extracted and stored in zarr (Miles et al., 2024) format. Also, we extracted adult ICU waveform signal data from an external institution for evaluation provided by the Mount Sinai Health System. The MIMIC-III data was split into 95% training and 5% validation for pretraining.

3.2. Data Segmentation and Normalization

3-channel signal data are segmented into 30-minute, non-overlapping windows and resampled to $f_s=125\text{Hz}$. 30-minute segments are excluded from training if 20% or more of a single channel’s values are constant or null values. Following, null values are linearly interpolated. Each channel $x_c\left(t\right)$ is then normalized with inter-quartile range normalization, as described by Brink-Kjaer et al. (2022).

3.3. PhysioJEPA Architecture

The framework consists of three components:

1. Context encoder: A PatchTST transformer (Nie et al., 2023) processes a masked subset of input patches to produce context encodings ${\mathbf{h}}_c$.

2. Predictor: A smaller PatchTST transformer predicts target encodings of masked tokens ${\stackrel{\mathbf{ˆ}}{\mathbf{h}}}_t$.

3. Target encoder: An exponential weighted moving average copy of the context encoder produces target encodings ${\mathbf{h}}_t$ from the full input, the output of which is subset and compared to that of the Predictor.

Context and target encoders use 3-layer PatchTST encoders with 8 attention heads, dimension 512, and feedforward sizes of 2048. The predictor implements 2 layers with 4 heads and a dimension of 256.

3.3.1. Channel-Specific Mask Tokens

For target token prediction using context encodings, channel-specific learnable mask tokens ${\mathbf{c}}_i$, $i=1,\dots ,C$ are repeated $t$ times (the number of targets to predict), concatenated, and shuffled with context tokens. The predictor then outputs ${\stackrel{\mathbf{ˆ}}{\mathbf{h}}}_{t,i}^{\left(c\right)}$ at each masked position $i$ for each channel $c$. This result is compared to the target encoder output and optimized with a mean squared error loss function.

3.3.2. Patch Embedding with Depthwise Convolutions

Each 30-minute segment is tokenized into 1-second patches via a depthwise convolution per channel, equivalent to PatchTST’s channel independent patching and linear tokenization procedure (Nie et al., 2023):

$$h_{p,c,d}=\sum _{i=0}^{k-1}{w}_{c,d}\left[i\right]x_c[p\cdot k+i]+b_{c,d},d=1,\dots ,D.$$ (1)

where $x_c\in {\mathbb{R}}^{125}$ is the channel $c$ indexed at patch $p$ of size $k$ (the kernel size). Each channel $c$ has $D$ convolutional kernels $w_{c,d}\in {\mathbb{R}}^k,d=1,\dots ,D$ with bias $b_{c,d}$. This helps learn patch-wise features into an embedding vector $Z\in {\mathbb{R}}^{c\times d\times p}$ where $d=512$ and $p$ is the number of tokens. Following this, the batch and channel dimensions are flattened together prior to input into the transformer, creating a tensor of shape $bs*c\times 1800\times 512$, where 1800 is the number of tokenized 1-second patches from the 30-minute segment.

3.3.3. Rotary Positional Embeddings

For temporal encoding, rotary positional embeddings (RoPE) were applied (Su et al., 2021) to each patch token. For token $i$ with embedding dimension $d$, each embedding vector $Z\in {\mathbb{R}}^d$ is split into two subvectors ${\mathbf{h}}_j=\left[h_{2j},h_{2j+1}\right]$, $j=1,\dots ,\frac{d}{2}-1$. Each even-odd pair in token $i$ is then rotated by an angle ${\theta }_j={10000}^{-2j/d}$. This is applied independently to each token ${\varphi }_{p,j}=p\cdot {\theta }_j$ with:

$${\tilde{\mathbf{h}}}_{p,j}=\left[\begin{array}{c}\text{cos}\left({\varphi }_{p,j}\right)-\text{sin}\left({\varphi }_{p,j}\right)\\ \text{sin}\left({\varphi }_{p,j}\right)+\text{cos}\left({\varphi }_{p,j}\right)\end{array}\right]{\mathbf{h}}_{p,j}.$$ (2)

Effectively, this step captures relative and absolute positional embeddings across and within patches.

3.3.4. Masking and Training Parameters

For training, target masks select 10–30% of patches at random from each flattened batch-channel dimension. Context masks select 10%−40% of the remaining patches. The representation framework was trained for 100 epochs with a one-cycle learning rate scheduler and AdamW optimizer. After training, the context encoder inputs tokenized signal data $Z_{\mathit{\text{in}}}\in {\mathbb{R}}^{c\times d\times p}$ and generates representations $Z_{\mathit{\text{out}}}\in {\mathbb{R}}^{c\times d\times r}$, where r is the learned representation for channel $c$ with dimension $d$.

3.4. Non-Linear Probing Architecture

After representation learning, non-linear probing classifiers are trained with learned, frozen representations for each task. The context embedding tensor $Z\in {\mathbb{R}}^{c\times r\times d}$ is passed into an attentive classifier (Assran et al., 2023; Bardes et al., 2024) with flattened batch and channel dimensions. A single learned query vector $q\in {\mathbb{R}}^d$ is utilized to extract relevant encoding information from the context encoders representations $Z$. Queries, keys, and values are calculated $Q=q{W}_q,K=Z{W}_k$, and $V=Z{W}_v$ for each attention head. Following, attention scores and weights are generated:

$${\mathbf{a}}_{\mathbf{\text{score}}}=\frac{Q{K}^T}{\sqrt{d}},{\mathbf{a}}_{\mathbf{\text{weights}}}=\mathit{\text{softmax}}\left(a_{\mathit{\text{score}}}\right).$$ (3)

A final, pooled representation vector is calculated $\tilde{z}=a_{\mathit{\text{weights}}}V\in {\mathbb{R}}^{1\times d}$. The pooled representation $\tilde{\mathbf{z}}$ is reshaped to $bs\times c\cdot d$ and passed through a final linear layer for binary classification.

Notably, the weights of the context encoder from PhysioJEPA are frozen during this process. Attentive classifiers used 4 attention heads. Classifiers are trained for 20 epochs with a one-cycle learning rate scheduler and the AdamW optimizer.

3.4.1. Augmentation Techniques

During non-linear probing, common augmentation techniques were employed to 30-minute input segments, including random noise, channel dropout, and mixup (Zhang et al., 2018) to better handle class imbalance.

3.5. Supervised and Self-Supervised Comparison Models

For comparison to a supervised model, we trained a fully supervised convolutional classifier presented by Wang et al. (2016) for each task. Additionally, for comparison to other representation learning frameworks, we trained an equivalent PatchTST (Nie et al., 2023) encoder via masked autoregression and another JEPA based signal model, ECG-JEPA (Kim, 2024). For implementation details, see Appendix A.1.

3.6. Clinical Tasks

3.6.1. 5-Minute Hypotensive Risk Estimation

For the first task, we estimated risk of having a hypotensive event at a 5-minute forecast horizon. Hypotensive risk was chosen due to its common occurrence in ICU patients (60–75% develop hypotension (Terwindt et al., 2022)) and the critical need for improved proactive monitoring. A hypotension threshold was defined based on mean arterial pressure (MAP) of ≤ 65 mmHg or systolic blood pressure (SBP) ≤ 90 mmHg at each minute.

A positive hypotensive event was defined as five consecutive minutes below the hypotension thresholds. Thus, a hypotensive event at a 5 minute forecast was derived with data from the 5 to 10 minute interval ahead of the end of the input signal. MAP and SAP were calculated using peak detection algorithms adapted from PhysioNet’s wfdb package (Goldberger et al., 2000; Moody et al., 2022). Hypotensive events longer than 5 minutes were treated as a single event, ensuring at least 5 minutes between multiple events for the same patient. Non-hypotensive patients were those with no hypotensive events throughout their entire ICU stay. Data was split into training (80%), validation (10%), and testing (10%) using a proportional, subject-wise data splitter. Additionally, during training a weighted sampler was used to present the model with more positive cases, given the high class imbalance.

3.6.2. 5-Minute Shock Index Risk Estimation

For the second task, we estimated risk of elevated shock index (SI) at a 5-minute forecast horizon. SI was chosen as it serves as an important marker for shock and mortality (Koch et al., 2019). SI was computed as the ratio of heart rate to SBP:

$$\text{SI}=\frac{\text{HR}}{\text{SBP}},\text{Positive}\text{class}\text{if}\text{SI}\ge 0.9$$ (4)

A shock index event was defined as five consecutive minutes above the 0.9 threshold. Heart rate was derived from the ABP signal channel using PhysioNet’s peak detection algorithms (Goldberger et al., 2000; Moody et al., 2022). Normal SI values range from 0.5 to 0.7, and higher values are more predictive of adverse outcomes (Cannon et al., 2009). Data was split into training (80%), validation (10%), and testing (10%) using a proportional label, subject-wise data splitter. Again, a weighted sampler was used during training to account for class imbalance.

3.7. External Validation

To assess generalizability, we conducted external validation on the Mount Sinai Bedmaster Dataset collected from 100 randomly selected patient ICU stays (per task) from 6 separate adult ICUs between 2019 and 2024.

3.8. Evaluation

We evaluated PhysioJEPA, PatchTST, ECG-JEPA, and supervised convolutional models for estimating 5-minute risks of hypotension and shock index using the MIMIC-III test set and external Mount Sinai Bedmaster Dataset. Performance metrics included area under the receiver operating characteristic curve (AUROC), average precision, F1, recall, specificity, and sensitivity at 90% and 95% specificity (Sens@90%Spec, Sens@95%Spec, respectively). The latter two measures could be particularly valuable in clinical settings, which require high specificity to reduce false alarms.

4. Results

Of the 5,660 stays from the MIMIC-III Waveform Database with the required three signal channel data (ABP, ECG lead II, PPG), 1,378 were not used for representation learning due to 20% or more constant or NaN values in a single channel (within every 30-minute segment). We found that the majority of the removed samples had ≥ 75% constant or NaN values (total of 1,050 samples). Thus, samples removed were majority constant or NaN values. Overall, PhysioJEPA was trained with 356,903 30-minute segments (total of 10,707,090 minutes) across 4,282 ICU stays (N=2,631 patients) with 3-channel signal data. Demographics are shown in Appendix C Table 3. Training was stopped after 100 epochs, and the last model was used for non-linear probing. An overview of the architecture is shown in Figures 1 and 2. Compute resources are detailed in Appendix B.

Figure 1:

PhysioJEPA Context Encoder. Bedside monitoring signal channels (ABP, ECG lead II, and PPG) are extracted in 30-minute segments sampled at 125 Hz and tokenized into 1-second patches. Random patches are selected by context and target masks, which are encoded using a Patch Time Series Transformer (PatchTST) encoder (Nie et al., 2023).

Figure 2:

PhysioJEPA Predictive Architecture and Target Encoder. Channel specific mask tokens are added to the context embeddings to indicate which patches the predictor must reconstruct, while the target encoder processes the full segment and subsets embeddings at the target locations. Prediction is optimized by minimizing mean squared error between predicted and target embeddings.

4.1. Risk Estimation Results

Dataset statistics of both hypotension and SI are reported in Table 1.

Table 1:

Hypotension and shock index event statistics for training, validation, testing, and external test sets.

	Hypotension				Shock Index
Data Split	N	N Patients	Positive Events	Negative Events	N	N Patients	Positive Events	Negative Events
MIMIC III Train	3280	2060	44897 (0.04)	1015429 (0.96)	3264	2062	53897 (0.05)	1087321 (0.95)
MIMIC III Validation	381	237	5601 (0.05)	116307 (0.95)	335	216	6882 (0.07)	96229 (0.93)
MIMIC III Test	341	229	4435 (0.04)	106273 (0.96)	413	245	8526 (0.05)	149215 (0.95)
Mt Sinai Bedmaster	99	99	2638 (0.03)	82626 (0.97)	98	98	1952 (0.05)	37089 (0.95)

Performance of Supervised Convolutional Models:

Two fully supervised convolutional classifiers achieved AUROC scores of 0.778 (95% bootstrapped confidence interval [CI]: 0.771–0.784) for 5-minute hypotension (Figure 3A) and 0.950 (95% CI: 0.948–0.952) for 5-minute shock index risk estimation on the held-out test set (Figure 4A).

Figure 3:

Receiver operating characteristic curves for 5-minute hypotension risk estimation for PhysioJEPA, PatchTST, ECG-JEPA, and the fully supervised convolutional model for held-out (A) and external (B) test sets.

Figure 4:

Receiver operating characteristic curves for 5-minute shock index risk estimation for PhysioJEPA, PatchTST, ECG-JEPA, and the fully supervised convolutional model for held-out (A) and external (B) test sets.

Performance of ECG-JEPA Models:

Non-linear probing with frozen ECG-JEPA encoder representations achieved AUROC scores of 0.729 (95% CI: 0.721–0.738) for 5-minute hypotension (Figure 3A) and 0.923 (95% CI: 0.921–0.925) for 5-minute shock index risk estimation on the held-out test set (Figure 4A).

Performance of PatchTST Models:

Non-linear probing with frozen PatchTST encoder representations achieved AUROC scores of 0.867 (95% CI: 0.861–0.871) for 5-minute hypotension (Figure 3A) and 0.956 (95% CI: 0.955–0.958) for 5-minute shock index risk estimation on the held-out test set (Figure 4A).

Performance of PhysioJEPA Models:

Non-linear probing with frozen PhysioJEPA encoder representations was similar to or outperformed comparison models, achieving AUROC scores of 0.833 (95% CI: 0.825–0.838) for 5-minute hypotension risk (Figure 3A) and 0.954 (95% CI: 0.952–0.955) for 5-minute shock index risk estimation (Figure 4A). For details of other performance metrics of all models, see Table 2.

Table 2:

Comprehensive performance comparison of PhysioJEPA, PatchTST, ECG-JEPA, and the fully supervised convolutional model on hypotension and shock index risk estimation tasks across the held-out MIMIC-III test set and external Mount Sinai Bedmaster Dataset. Best performance for each metric is shown in bold (95% bootstrapped confidence intervals).

Condition	Dataset	Model	AUROC	Avg Precision	F1	Recall	Specificity	Sens@90%Spec	Sens@95%Spec
Hypotension	Held-Out	PhysioJEPA	0.833 (0.825–0.838)	0.264 (0.254–0.277)	0.122 (0.119–0.125)	0.924 (0.916–0.931)	0.446 (0.443–0.449)	0.552 (0.538–0.566)	0.407 (0.391–0.421)
		PatchTST	0.867 (0.861–0.871)	0.267 (0.257–0.282)	0.105 (0.102–0.108)	0.979 (0.974–0.984)	0.304 (0.302–0.307)	0.576 (0.565–0.616)	0.429 (0.416–0.442)
		ECG-JEPA	0.729 (0.721–0.738)	0.110 (0.104–0.116)	0.077 (0.075–0.079)	1.000 (1.000–1.000)	0.000 (0.000–0.000)	0.322 (0.310–0.335)	0.146 (0.135–0.157)
		Supervised Conv	0.778 (0.771–0.784)	0.140 (0.133–0.148)	0.082 (0.080–0.085)	0.998 (0.997–0.999)	0.073 (0.071–0.074)	0.422 (0.406–0.435)	0.296 (0.282–0.310)
	Mount Sinai	PhysioJEPA	0.781 (0.773–0.788)	0.098 (0.093–0.104)	0.084 (0.081–0.086)	0.949 (0.939–0.956)	0.340 (0.337–0.343)	0.339 (0.319–0.356)	0.204 (0.191–0.218)
		PatchTST	0.761 (0.753–0.769)	0.085 (0.081–0.090)	0.075 (0.072–0.078)	0.955 (0.947–0.963)	0.251 (0.248–0.254)	0.303 (0.287–0.324)	0.000 (0.000–0.000)
		ECG-JEPA	0.568 (0.560–0.577)	0.039 (0.037–0.041)	0.060 (0.058–0.062)	0.986 (0.981–0.991)	0.018 (0.017–0.019)	0.153 (0.059–0.168)	0.063 (0.056–0.072)
		Supervised Conv	0.695 (0.686–0.705)	0.057 (0.054–0.060)	0.064 (0.062–0.067)	0.917 (0.908–0.927)	0.151 (0.149–0.154)	0.213 (0.197–0.225)	0.069 (0.056–0.074)
Shock Index	Held-Out	PhysioJEPA	0.954 (0.952–0.955)	0.474 (0.465–0.485)	0.308 (0.303–0.313)	0.981 (0.978–0.983)	0.749 (0.747–0.751)	0.881 (0.875–0.889)	0.706 (0.697–0.715)
		PatchTST	0.956 (0.955–0.958)	0.595 (0.584–0.607)	0.391 (0.385–0.396)	0.941 (0.936–0.945)	0.836 (0.834–0.838)	0.873 (0.866–0.880)	0.743 (0.734–0.760)
		ECG-JEPA	0.923 (0.921–0.925)	0.364 (0.355–0.370)	0.217 (0.213–0.221)	0.986 (0.984–0.989)	0.595 (0.593–0.598)	0.726 (0.716–0.735)	0.437 (0.425–0.447)
		Supervised Conv	0.950 (0.948–0.952)	0.555 (0.546–0.565)	0.296 (0.292–0.300)	0.978 (0.974–0.980)	0.735 (0.733–0.738)	0.852 (0.846–0.859)	0.676 (0.665–0.687)
	Mount Sinai	PhysioJEPA	0.923 (0.917–0.927)	0.396 (0.381–0.410)	0.226 (0.219–0.234)	0.969 (0.963–0.976)	0.652 (0.648–0.657)	0.716 (0.699–0.738)	0.356 (0.335–0.374)
		PatchTST	0.873 (0.864–0.882)	0.312 (0.295–0.333)	0.244 (0.235–0.254)	0.862 (0.847–0.877)	0.727 (0.723–0.731)	0.672 (0.642–0.691)	0.501 (0.477–0.520)
		ECG-JEPA	0.888 (0.882–0.894)	0.231 (0.218–0.242)	0.142 (0.137–0.146)	0.988 (0.982–0.992)	0.371 (0.365–0.375)	0.584 (0.565–0.607)	0.000 (0.000–0.000)
		Supervised Conv	0.782 (0.771–0.794)	0.162 (0.152–0.171)	0.167 (0.160–0.174)	0.840 (0.828–0.853)	0.569 (0.564–0.572)	0.522 (0.498–0.548)	0.202 (0.185–0.218)

At clinically relevant specificity thresholds, PhysioJEPA achieved matched or superior performance on the held-out test set (Table 2): 55.2% sensitivity at 90% specificity and 40.7% at 95% specificity for 5-minute hypotension risk estimation, compared to PatchTST performance of 57.6% and 42.9%, ECG-JEPA performance of 32.2% and 14.6%, and the supervised convolutional model performance of 42.2% and 29.6%, respectively. For shock index, PhysioJEPA achieved 88.1% and 70.6% sensitivity at 90% and 95% specificity thresholds, similar to PatchTST results of 87.3% and 74.3%, and outperforming ECG-JEPA results of 72.6% and 43.7% and supervised convolutional classifier results of 85.2% and 67.6%, respectively.

4.2. External Validation Results

External validation on 100 randomly selected patient-ICU stays (for each task) from the Mount Sinai Bedmaster Dataset demonstrated strong cross-site generalizability without retraining. The external test set comprised data from 6 different adult ICUs. Demographics are shown in Appendix C Table 4. 3 samples were excluded due to our missingness constraints as detailed in the Methods. 5-minute hypotension risk estimation using the PhysioJEPA architecture and classifier trained on MIMIC-III data achieved an AUROC of 0.781 (95% CI: 0.773–0.788), compared to 0.761 (95% CI: 0.753–0.769) for PatchTST, 0.568 (95% CI: 0.560–0.577) for ECG-JEPA, and 0.695 (95% CI: 0.686–0.705) for the supervised convolutional classifier (Figure 3B). Similarly, 5-minute shock index risk estimation achieved an AUROC of 0.923 (95% CI: 0.917–0.927), compared to 0.873 (95% CI: 0.864–0.882) for PatchTST, 0.888 (95% CI: 0.882–0.894) for ECG-JEPA, and 0.782 (95% CI: 0.771–0.794) for the supervised convolutional classifier (Figure 4B).

At clinically relevant specificity thresholds, PhysioJEPA achieved consistent high performance across both tested thresholds, 33.9% and 20.4% sensitivity at 90% and 95% specificity for 5-minute hypotension risk estimation, compared to PatchTST performance of 30.3% and 0.00%, ECG-JEPA performance of 15.3% and 6.30%, and the supervised convolutional model performance of 21.3% and 6.90%, respectively. For shock index, PhysioJEPA achieved 71.6% and 35.6% sensitivity at 90% and 95% specificity thresholds, compared to PatchTST results of 67.2% and 50.1%, ECG-JEPA results of 58.4% and 0.00%, and supervised convolutional model results of 52.2% and 20.2%, respectively (Table 2).

5. Discussion and Conclusion

PhysioJEPA demonstrates that JEPA-based self-supervised learning effectively captures complex multi-modal physiological signal relationships for critical care risk estimation. The strong performance on both tasks establishes the potential for real-world ICU deployment and cross-site adaptability. Compared to PatchTST, which has been established as a multi-modal physiological signal representation model (Fox et al., 2025), PhysioJEPA’s primary advantage lies in improved performance on the external test set in terms of AUROC and average precision across both tasks. For more clinically relevant thresholds, PhysioJEPA outperforms PatchTST for hypotension and shock index risk estimation at 90% specificity. At 95% specificity, PhysioJEPA outperforms PatchTST for hypotension risk estimation, but under performs on the shock index outcome. Compared to the supervised convolutional model and ECG-JEPA, which has been established as a single modality JEPA-based physiological signal representation model (Kim, 2024), PhysioJEPA consistently matches or outperforms across both test sets for hypotension and shock index risk estimation. In general, PhysioJEPA maintains consistently high performance across both risk estimation tasks, in comparison to other supervised and self-supervised models. This warrants further investigation into additional risk estimation outcomes to distinguish how performance across these models vary.

Clinical Impact:

The 5-minute risk estimation horizon provides actionable early warning capabilities for preventing adverse events. PhysioJEPA’s performance demonstrates the clinical value of self-supervised representation learning. The model relies solely on bedside monitoring data, enabling deployment across diverse ICU environments without requiring electronic health record integration.

Technical Contributions:

Our work extends JEPA to long-duration, high frequency, multi-modal physiological time series and adds improvements to better handle signals independently (depthwise convolutions, rotary embeddings, per channel masked tokens, and flattened batch-channel dimensions for attentive classification). This adds to other physiological signal JEPA approaches to handle multi-modal signals, instead of only ECG (Kim, 2024) or EEG (Guetschel et al., 2024). Additionally, PhysioJEPA is to our knowledge, the first self-supervised representation learning model to be developed for ICU bedside monitoring data. Our results show that the embedding-space prediction method can learn relevant features of physiological signal data and generalize equally or better to unseen datasets for predictive tasks, compared to PatchTST, ECG-JEPA, and a fully supervised convolutional method. Furthermore, given that the representations are generated without being fine-tuned to specific tasks, they may be applicable to other critical care outcomes, which merits further investigation.

Comparison to Existing Approaches:

Our work adds to the growing literature on hypotensive risk estimation by introducing a self-supervised framework that learns directly from raw bedside signals. Prior approaches using raw bedside signals trained fully supervised models to estimate hypotensive risk (Lee et al., 2021; Moon et al., 2024; Jeong et al., 2024; Jo et al., 2022). Lee et al. (2021), Moon et al. (2024), and Jo et al. (2022) used multi-modal signals to estimate 3-, 5-, 10-, and/or 15-minute hypotensive risk on the VitalDB (https://vitaldb.net/) database, without external validation. Jeong et al. (2024) trained a fully supervised model using VitalDB with ECG, PPG, capnography, bispectral index, and non-invasive ABP to estimate intraoperative hypotension at a 5-minute forecast with good performance on an external test set (AUROC: 0.833 (95% CI, 0.830–0.836)). In contrast, PhysioJEPA learns task-agnostic features by learning representations directly from multi-modal bedside signals via self-supervision and demonstrates competitive performance across internal and external datasets. To our knowledge, we are among the first to predict shock index using a self-supervised representation learning approach for raw multi-modal physiological data.

Limitations:

First, the model currently requires invasive ABP monitoring, which may not be available for all patients or institutions. Second, the evaluation was conducted on two specific critical care outcomes; broader validation across additional clinical endpoints is needed to establish general applicability. Third, the 30-minute input window needs to be investigated further along with the missingness threshold of 20%, as the tradeoff between these two elements could be optimized to reduce noise and better represent longer-term physiological patterns. Fourth, the external validation was conducted on a relatively small subset of patients from a single additional institution, and broader multi-site validation would strengthen generalizability claims. Fifth, optimal masking ratios for context and targets was not explored. Adding more context (like in PatchTST or ECG-JEPA) could improve learned representations and performance on downstream tasks. Finally, an additional transformer-based supervised classifier method could be explored to better understand performance, given both PhysioJEPA, PatchTST, and ECG-JEPA encoded signals with a transformer backbone.

Future Work:

Evaluating non-invasive risk estimation by removing the ABP channel would significantly expand the patient population that could benefit from this approach. Exploring longer forecast horizons (10–15 minutes) and varying input segment lengths could optimize the balance between early warning capability and prediction accuracy. Exploration of alternative preprocessing techniques, integration of additional physiological channels, masking ratios, and adoption of task-specific fine-tuning strategies or different classifier heads could further improve performance. Comparative evaluation against other self-supervised learning paradigms, such as a contrastive learning method, could further validate the effectiveness of joint embedding predictive architectures for physiological time series data. Additionally, applying the learned representations to other critical care outcomes such as sepsis onset, respiratory failure, or cardiac arrest would demonstrate broader utility.

Data and Code Availability

The MIMIC-III Waveform Database Matched Subset is publicly available upon request from https://physionet.org/content/mimic3wdb-matched/1.0/. Code is available at https://github.com/benmfox/PhysioJEPA. The Mount Sinai Bedmaster Dataset is not currently available.