Parsimonious Waveform-derived Features consisting of Pulse Arrival Time and Heart Rate Variability Predicts the Onset of Septic Shock

Abstract

Sepsis is a major public health emergency and one of the leading causes of morbidity and mortality in critically ill patients. For each hour treatment is delayed, shock-related mortality increases, so early diagnosis and intervention is of utmost importance. However, earlier recognition of shock requires active monitoring, which may be delayed due to subclinical manifestations of the disease at the early phase of onset. Machine learning systems can increase timely detection of shock onset by exploiting complex interactions among continuous physiological waveforms. We use a dataset consisting of high-resolution physiological waveforms from intensive care unit (ICU) of a tertiary hospital system. We investigate the use of mean arterial blood pressure (MAP), pulse arrival time (PAT), heart rate variability (HRV), and heart rate (HR) for the early prediction of shock onset. Using only five minutes of the aforementioned vital signals from 239 ICU patients, our developed models can accurately predict septic shock onset 6 to 36 hours prior to clinical recognition with area under the receiver operating characteristic (AUROC) of 0.84 and 0.8 respectively. This work lays foundations for a robust, efficient, accurate and early prediction of septic shock onset which may help clinicians in their decision-making processes. This study introduces machine learning models that provide fast and accurate predictions of septic shock onset times up to 36 hours in advance. BP, PAT and HR dynamics can independently predict septic shock onset with a look-back period of only 5 mins.

1. Introduction

Sepsis and septic shock remain a leading cause of morbidity and mortality, in the hospital environment, with an estimated 11 million deaths reported worldwide in 2017 [1]. In the United States alone, over 1.7 million adults develop sepsis annually, and over 20% of those patients die during their hospitalization or are discharged to hospice care [2]. About a third of all people who die in the hospital had sepsis during their stay. About 13% of sepsis cases or the infection leading to sepsis start in the hospital [2].

In sepsis, the body’s systemic response to an infection (bacterial, viral or fungal) can lead to organ shutdown and multi-organ failure often associated with acute respiratory distress syndrome (ARDS), which can progress rapidly [3, 4] and after which the only intervention known to improve mortality is lung protective ventilation [5]. Sepsis can rapidly lead to multiple organ dysfunction syndrome (MODs), with a high mortality rate [6] with an increase of in mortality rate for each hour of delayed diagnosis [7]. Early and accurate prediction of sepsis and particularly septic shock is of the utmost importance in preventing morbidity and mortality. Patients with severe sepsis receiving usual care have a 28-day mortality of 33.2% [8].

Many diagnostic and prognostic sepsis screening tools have been developed and evaluated for both pre-hospital and in-hospital settings. Diagnostic scoring systems including the systemic inflammatory response syndrome (SIRS) criteria [9], national early warning score (NEWS) [10], and pre-hospital early sepsis detection (PRESEP) score [11], are designed to have relatively fewer criteria, as the goal of these scores is to rule in patients with potential sepsis. Prognostic scoring systems are mainly designed to identify patients with increased mortality and include the sequential organ failure assessment (SOFA) [12], quick sequential organ failure assessment (qSOFA) [13], sepsis patient evaluation in the emergency department (SPEED) [14], mortality in emergency department sepsis (MEDS) [15], modified early warning score (MEWS) [16], and the predisposition, insult, response, organ dysfunction (PIRO) scores [17]. Most of these scoring systems use vital signs (such as respiratory rate, temperature and blood pressure) or blood tests (such as white blood cell count and lactate) or a combination of both to evaluate the patient’s condition. While these scoring systems may demonstrate high sensitivity, they provide suboptimal specificity, take time to calculate as lab test results are not instantaneous, are not specifically designed to predict sepsis development, and do not account for inter-subject variability or the source of infection.

In contrast to traditional methods, machine learning-based sepsis monitoring tools offer the potential for early sepsis risk detection with higher specificity and increased generalizability, facilitating earlier intervention by clinicians while reducing the burden of false alarms. Recently, efforts have focused on developing machine learning tools to predict the onset of hypotension, a crucial component of shock [18, 19, 20, 21, 22, 23, 24]. While these models present a more broadly applicable approach to predicting septic shock onset times, they often rely on electronic medical records, including lab test results, which hinders real-time responsiveness. To address this limitation, the models discussed in this work were constructed exclusively using high-frequency signals like electrocardiogram (ECG), photoplethysmogram (PPG), and arterial blood pressure (ABP). By leveraging commonly used vital sign calculations and eliminating dependency on lab test results, these models enable real-time predictions with minimal delay.

The recent exploration of models relying solely on continuously recorded vital signs to predict septic shock onset time has gained attention in the literature. In studies such as [25, 26], the inclusion of features like heart rate (HR), respiratory rate (RR), temperature, blood oxygen saturation (SpO2), systolic arterial pressure (SAP), and diastolic arterial pressure (DAP) has shown promise. However, these models exhibit a notable drawback with a two-hour look-back period before making predictions, and respective 24- and 48-hour look-forward periods for shock onset. Despite achieving AUROC curves of 0.84 and 0.83 for [25], and 0.85 and 0.81 for [26], the extended data requirement introduces potential delays that may not align with the urgent nature of predicting septic shock. A study by [27] attempts to address this by using features such as SAP, DAP, mean arterial pressure (MAP), heart rate (HR), and pulse arrival time (PAT), with a shorter 45-minute look-back period and a 15-minute look-forward period. Nevertheless, this reduction in the look-back period is balanced by a limited window for timely clinical intervention.

PAT, HRV and MAP emerge as critical physiological parameters in predicting shock onset due to their close associations with cardiovascular function and autonomic nervous system regulation. PAT, influenced by factors like arterial stiffness [28] and changes in blood flow dynamics, serves as an indicator of vascular changes that may precede the onset of shock. HRV, reflecting autonomic nervous system activity, acts as an early warning sign for autonomic imbalance in shock [29], with reduced HRV indicating potential physiological changes. Dysfunction in baroreceptor reflexes may contribute to decreased HRV in the context of shock [30]. MAP, a key determinant of tissue perfusion and oxygen delivery, reflects compromised perfusion pressure, signaling potential issues with vasomotor tone and widespread vasodilation associated with shock [31]. The continuous monitoring and dynamic analysis of PAT, HRV, and MAP provide a comprehensive understanding, enabling early detection and intervention in critical situations by identifying physiological changes indicative of impending shock.

In this work, we develop machine learning models to predict septic shock onset using four high-frequency (continuously recorded) vital signs which are HR, heart rate variability (HRV), PAT and MAP with a look-back period of 5 minutes and a look-forward period of up to 36 hours. Such models eliminate any dependency on lab test results, allow for fast prediction compared to state-of-the-art literature models and provide a look-forward period long enough for clinicians to intervene and take appropriate measures to reduce morbidity and mortality.

2. Materials and Methods

2.1. Cohort Description

The dataset collected from Emory affiliated hospitals (Atlanta, GA), after Emory University institutional review board (IRB) approval with reference number IRB#STUDY00000302, had 61,374 admissions from 33,423 patients. The number of admissions with high resolution waveforms are 10,462 from 6,704 patients. For this analysis, the patient cohort was limited to 239 ICU patients with sufficient data availability, defined as high resolution ECG, PPG and ABP waveforms for their entire length of stay. A summary of the population demographic information is shown in table 1. The waveforms were sampled in the ICU at 240 HZ using the BedMaster system (Excel Medical Electronics, Jupiter FL, USA), which is a third-party software connected to the hospital’s General Electric (GE) monitors for the purpose of electronic data extraction and storage of high-resolution waveforms. The average age of the patients is 60.2 ± 14.9 years, with 47% females and 53% males and average length of stay of 88.5 hours. We adopt the sepsis-3 definition as in [32] in which patients are considered to have sepsis if they were suspected to have an infection and their SOFA score is up by 2 or more points. For this analysis, patients were considered to be in a septic shock state if they experienced a hypotensive event in which their MAP went below 65 mm-Hg for a period of longer than 5 minutes in the presence of vasopressors and lactate ≥ 2 mmol/L. There were 75 patients who met the sepsis-3 definitions in [32], while 164 patients did not meet sepsis-3 and were used as control.

Table 1:

Demographics table

	Sepsis	No Sepsis
SOFA	4.49 ± 3.08	2.29 ± 2.24
Lactate	3.04 ± 3.24	2.58 ± 2.61
Creatinine	1.74 ± 1.72	1.64 ± 1.71
Bilirubin	3.63 ± 7.72	2.15 ± 5.49
White Blood Cell Count	12.49 ± 10.43	11.35 ± 7.27
PaO2	125.74 ± 91.22	143.50 ± 115.63
Platelets	197.55 ± 138.05	228.69 ± 140.66
FiO2	49.84 ± 20.59	44.20 ± 21.42
SpO2	97.37 ± 3.78	97.34 ± 3.72
Systolic BP	121.88 ± 29.38	127.83 ± 31.09
Diastolic BP	62.23 ± 14.39	63.43 ± 14.94
Temperature	36.95 ± 0.99	37.03 ± 0.98
Respiration Rate	21.62 ± 7.18	19.80 ± 6.44
Heart Rate	93.04 ± 19.88	91.58 ± 19.45
Mortality	53.6%	21.2%

2.2. Feature Selection and Extraction

We use automated algorithms to extract fiducial points from ECG, PPG and ABP waveforms. The detailed feature extraction process starting from raw signals and ending with the dataset are shown in Fig. 1. For ECG, we applied a bandpass filter between 0.6–40 HZ to eliminate baseline wander as well as power-line noises [33, 34]. Then, R-peaks were detected using the PanTompkin’s algorithm [35, 36] after which heart rate and heart rate variability were calculated [37]. PPG and ABP signals were filtered between 0.3–10 HZ to eliminate baseline wander as well as high power noise [38]. For PPG, the foot of each wave was detected using the intersecting tangents method as in [39]. The time between an ECG R-peak and the following PPG foot is considered as pulse arrival time (PAT) as shown in Fig. 1.

Figure 1:

Feature extraction pipeline, raw physiological signals (ECG, PPG and ABP) are filtered at the first stage to remove baseline wander and high frequency noise. Then Rpeaks are detected using the ECG signal, from which HR and HRV signals are extracted. The points of maximum gradient are detected from PPG signal as shown in the bottom left figure. Then the PAT signal is calculated. MAP is also generated by calculating the geometrical average of the ABP signal. Finally, data points are generated using a 300 heart beat sliding window with 10% overlap. The labels are generated by checking the shock onset criteria at a given horizon.

For the ABP signal, we detected the peaks and valleys of the signal. The peaks represent systolic pressure ($\mathrm{S}\mathrm{P}$) while the valleys represent diastolic pressure ($\mathrm{D}\mathrm{P}$). The mean arterial pressure ($\mathrm{M}\mathrm{A}\mathrm{P}$) is calculated using the following equation

$$MAP=\frac{SP+2DP}{3}$$ (1)

A moving window consisting of 300 heartbeats (less than 5 minutes) with 10% overlap is used to segment the extracted features (HR, HRV, PAT, MAP) into data points as shown in Fig 1. The corresponding label for each data point was dependent on the shock onset time, or the time between the end of the window and the start of the shock event as shown in Fig. 1. If the patient experienced shock within a time horizon t in the future, the current data point was labeled as one otherwise it was labeled as zero. Different shock onset horizons were considered with t values of 6, 12, 18, 24, 30 and 36 hours.

2.3. Machine Learning models

The extracted features were normalized by removing the median and scaling the data according to the inter-quartile range. This normalization method is more robust to outliers since it doesn’t use extreme data points in the normalization process, unlike subtracting the mean and dividing by the standard deviation which includes outliers in the calculation of mean and standard deviation. Then the dataset was randomly split, based on subjects, to 80% for training and 20% for testing. The training data is further split, based on subjects, to 70% for actual training and 30% for validation. The subject-based splitting guarantees that during validation and testing that machine learning models were not exposed to these subjects before which avoids any data leakage. Thus any conclusions are more likely to be generalizable to new subjects. The reported results are the average of 10 runs of training, validation and testing using different random subject-based splits.

Logistic Regression (LR), Extreme Gradient Boosted (XGB), Support Vector Machines (SVMs) with radial basis function kernel, Multi-layer Perceptron (MLP), Random Forests and K-Nearest Neighbours (KNN) were trained using threefold cross validation (the splits are done by patients) through a grid search optimization of the hyperparameters. The model parameters that achieved the highest validation accuracy were picked for the model. The training was done using 6 hours horizon shock onset data while the testing was done on 6, 12, 18, 24, 30 and 36 hours horizons. In this work, we compare and contrast the performance and explore the feature importance of the six machine learning models.

3. Results

To evaluate the performance of different models, we report the area under the receiver operating characteristic curve (AUC), F1 score, accuracy (ACC), Sensitivity (SEN) and Specificity (SPE) for all the models at different shock onset horizons as shown in Fig. 2. Each bar in the box plots is the test result mean and standard deviation of 10 different runs. The AUC and ACC plots show the average performance of different models, while the F1, SEN and SPE plots show how different models handle class imbalance across different horizons.

Figure 2:

Different classification key performance indices are shown for different classifiers at 6, 12, 18, 24, 30 and 36 hours septic shock onset horizons. a) Area under receiver operating characteristics (AUC) curve, b) F1 score, c) Accuracy, d) Specificity, and e) Sensitivity.

The model’s performance ranked highest to lowest, according to AUC, ACC, and F1, are LR followed by XGB, SVM, Perceptron, RF and finally KNN as shown in Fig. 2 a), b) and c). For LR, the best test mean AUC, F1 and ACC are 0.83 at the 6 hours onset horizon. While for XGB, the AUC is 0.81, F1 and ACC are 0.83 at the same onset horizon. The results show that for different models, the longer the onset horizon in the future, the less AUC, ACC and F1 scores get. This was a consistent finding across different horizons for all models. Fig. 2 d) and e) show the SEN and SPE of different models across different horizons. The LR is the most specific classifier among different models, while SVM is the most sensitive.

We also report the receiver operating characteristic (ROC) curves for different models at the 6 hour onset horizon in Fig. 3 a). The shown curves are the test results mean and standard deviation of 10 different runs. The mean is represented by the solid middle line, while the standard deviation is represented by a band around that line. All the models are contrasted against a random guess (RG) classifier as shown in Fig. 3 a). The LR and XGB ROC curves show that both models are able to distinguish different classes with high accuracy. The area under the curves for LR and XGB are 0.83 and 0.81 respectively. While the false alarm is 0.19 (specificity = 0.81) for LR and 0.24 (specificity = 0.76) for XGB. In the evaluation of shock prediction models over various time horizons (6, 12, 18, 24, 30, 36 hours), as illustrated in Fig. 3b, the comparison between LR and XGB reveals distinctive performance trends. LR consistently outperforms XGB across all horizons, as reflected in the higher Area Under the Curve (AUC) values: 0.83, 0.81, 0.81, 0.80, 0.80, and 0.79 for LR compared to 0.81, 0.79, 0.78, 0.78, 0.77, and 0.77 for XGB. Notably, both models exhibit a diminishing AUC trend with increasing shock horizons, indicating the heightened challenge of predicting shocks over extended time periods.

Figure 3:

More insights on the classification performance of logistic regression (LR) and extreme gradient boosting (XGB) models. a) Receiver operating characteristics (ROC) curve for the models against random guess (RG). b) AUC performance for RL and XGB accross different shock onset horizons. c) and e) SHAP values for LR and XGB respectively. d) and f) Mean absolute SHAP values for LR and XGB respectively.

To obtain insights on the relative importance of different features and how different models make their predictions, we report the test results Shapley additive explanations (SHAP) [40] values for different models at the 6 hour onset horizon as shown in Fig. 3. The exact SHAP values for different models are reported on the left column of Fig. 3, while the mean absolute SHAP values are reported on the right column of the figure. The MAP scores the highest importance among features while the PAT is the second most important feature. The third most important feature for LR, SVM, RF and KNN is HRV while the least important feature is HR.

We finally show an example of a subject case in Fig. 4. The extracted features for the patient are shown in Figs. 4 b), c), d), and e) while the corresponding model output is shown in Fig. 4 a). The shock onset time was set to zero and the hypotension event is highlighted in red as shown in Fig. 4 b). The output of the predictor, LR trained on 6 hrs onset horizon in this case, flagged the hypotension nearly seven hrs before onset for this subject.

Figure 4:

A real example of logistic regression model output during testing a). The patient features are b) MAP, c) PAT, d) HRV, and e) HR

4. Discussion

In [42], the study concludes that HRV can be informative when predicting sepsis progression and mortality. While in [43], the authors found that PAT is one of the most important features (among those in the study) in identifying patients with sepsis. In this study we have shown that septic shock can be predicted in a fast and reliable way using the two aforementioned features along with MAP and HR. This was demonstrated by different classifiers at different onset horizons. As the features are independent of lab test results, this feature combination allows for real-time prediction, which is crucial since early intervention can potentially decrease mortality rates and improve patient outcomes. As was expected, the prediction performance was found to be inversely proportional to how far in the future the onset is. As shown in Fig. 2, AUC, F1, accuracy and specificity all show a decreasing trend as the shock onset increases from 6 hrs to 36 hrs.

The required time to complete the procedures of sepsis protocol has decreased from hour-6 bundle to hour-3 bundle [44] and hour-1 bundle initiation [45]. Early sepsis diagnosis and initiation of treatment protocols has a strong impact on reducing patient mortality and morbidity. Recently, procedures like administering vasopressors, IV broad spectrum antibiotics and lactate/blood culture measurements became more common [46]. The study in [47] concluded that rapid the completion of a 3-hour bundle of sepsis care and rapid administration of antibiotics, are associated with lower riskadjusted in-hospital mortality. Thus, making early prediction and diagnosis more important than ever. Our developed model is aimed at predicting the occurrence of septic shock as fast as possible using features from physiological waveforms. Indeed, this modeling approach uses only few clinical features that are available in most ICU settings. The used features are extracted from the monitored vital waveforms and available at the bedside.

To our knowledge, this is the first study to achieve AUROC of 0.84 predicting septic shock onset at different horizons using only a 5 minute lookback period. In Table 2, we included similar studies that predict septic shock onset using features extracted only from high frequency signals. The lookback periods range from 45 minutes to 8 hrs, while our models use only 5 minutes of data to predict shock onset. The shorter the look back period, the faster the onset is predicted, for newly admitted patients, which provides clinicians with information as early as possible to help with fast intervention measures. This finding was validated across six different machine learning models and across six different onset horizons as shown in Fig. 2.

Table 2:

Comparison between this work and similar literature works according to Endpoints, number of subjects, used features, look back period, look forward period, machine learning models used, and best area under receiver operating characteristics curve (AUROC).

EndRef. point	Number of Patients	Signals or Features	Look back period	Look forward period	ML model	Best testing AROC
Shock [41] onset	600	HR, MAP, RR, Temp.	8 hrs	4 hrs	LR, SVM, MLP	0.88
Shock [25] onset	91,445 UCSF 21,507 MIMIC-3	HR, RR, SAP, DAP, SpO2, Temp.	2 hrs	24 hrs 48 hrs	XGB	0.84 0.83
Shock [27] onset	100 MIMIC-3	SAP, DAP, MAP, PAT, Pulse pressure.	45 mins	15 mins	LR, SVM, Tree, Ensemble Tree	0.93
Shock [26] onset	270,438 Training 13,581 Testing	SAP, DAP, HR, RR, SpO2, age, Temp.	2 hrs	0 hrs 4 hrs 6 hrs 12 hrs 24 hrs 48 hrs	XGB	0.92 0.87 0.86 0.85 0.85 0.82
This Shock workonset	239 Emory	MAP, PAT, HRV, HR.	5 mins	6 hrs 12 hrs 18 hrs 24 hrs 30 hrs 36 hrs	LR, XGB, SVM, MLP, RF, KNN	0.84 0.83 0.83 0.82 0.81 0.80

We have also compared the importance of 4 different features that are commonly used in literature. We found that blood pressure features, specifically MAP in this study, are highly important when predicting sepsis progression, aligning with the findings in [41, 25, 24, 27]. In our prior work, we have shown these characterizations from physiological data to be predictive of sepsis across the age group, from neonates to adults [48, 49, 50, 51]. We also investigated the relative importance of different features in Fig. 3 b), c), d), and e). We found that PAT is another important predictor of sepsis progression as well as HRV. However, PAT is shown to have more impact on the model prediction compared to HRV which might indicate that PAT could potentially be more correlated to sepsis progression than HRV. This could be due to the fact that PAT is related to blood pressure while HRV is related to the patient’s autonomic tone, which changes with various stressors on the body.

This decision support notification system is meant to alert clinicians that there is a prediction of sepsis progression while providing feature level justification for that prediction which will prompt an immediate focused review of a patient’s physiology. The sensitivity and specificity of such alerts can impact its usefulness and trustworthiness in real scenarios. The SVM classifier has the highest sensitivity and hence the lowest false negative rate among different classifiers considered in this work as shown in Fig. 2. This property makes SVM more suited for critical cases where a higher rate of false positive alarms may be more acceptable. On the other hand, LR is the classifier with the highest specificity and hence the lowest false alarm rate with makes it more suited for cases that are less critical and in environment where alert volumes are relatively high. There are no studies framing a discussion around acceptable sensitivity and false alarms rates in sepsis detection systems, and very little literature on clinical decision support systems for the critically ill patient. Formal decision analysis might offer a potential framework. Alternative approaches beyond simple thresholding of risk, such as those based on trajectory modeling, might also address the step of transitioning from a machine learning to a clinically useful decision support system [52].

The proposed approach in this paper is designed around providing fast and relatively accurate predictions while being least disruptive to ICUs setup. The decision support system uses features that are available in almost any ICU setting namely: MAP, PAT, HRV, and HR which makes its implementation and usage relatively easier than other complicated approaches. Another attribute is that this approach uses by far the shortest look back period compared to all other literature approaches so it can provide fast predictions without compromising much the prediction accuracy. Adding to that, the proposed method does not require any lab test results, such as lactate lab tests, which makes our model even faster by eliminating the wait time for lab tests. The development of such a tool to collect, process vital signs and provide informative risk scores paves the way for the development of more sophisticated algorithms that can optimize patient treatment and follow-up.

5. Limitations

One limitation in this study is that it was performed exclusively on ICU data from Emory hospitals, which might limit generalizability of our results to other hospitals and hospital systems. While our models operate using only data that are commonly available in ICUs, the outcomes reported in this study on ICU data do not provide a guarantee of equivalent performance in other settings. A much larger cohort and external validation of results on other datasets are certainly needed to address different stages of sepsis evolution and to develop prediction algorithms that can be used in a diverse ICU scenarios. Additionally, monitoring requirements, presence of high quality waveforms and invasive arterial blood pressure line as part of the patient inclusion criteria, might somehow influence the patients population used for this study. Given the widespread lack of availability of an ABP signal, a promising future direction would be to construct a model either not using ABP, or using a proxy ABP signal reconstructed from pleth waveforms and intermittent non-invasive signals. As for the experimental design, The AUROC, F1, accuracy, sensitivity, and specificity calculated for longer prediction windows were calculated using a smaller patient pool than the shorter prediction window metrics. These results could have been skewed by the smaller patient populations. To account for this uncertainty, we presented confidence intervals alongside the used metrics. Finally, while our model uses much less look back period, which speeds up predictions, the achieved AUROC are slightly less than models that use 2 hrs of look back period as in [26].

6. Conclusions

This study introduces machine learning models that provide fast and accurate predictions of septic shock onset times up to 36 hours in advance. BP, PAT and HR dynamics can independently predict septic shock onset with a look-back period of only 5 mins. The predictive performance is inversely proportional to the time length of onset horizon. Paired with the minimal patient data required for predictions, our models can predict onset times with AUROC of 0.84, without requiring additional data analyses from clinicians. Future studies will validate these findings in a larger cohort, and add include more advanced derivatives of the ECG and BP waveform to test for further improvement in sepsis prediction.

Highlights.

• This is the first study to provide accurate septic shock onset prediction using only 5 minutes of patient’s vital signals which makes the proposed models the fastest to provide insights to clinicians.

• The proposed models rely only on vital signals unlike other literature models which include lab tests and patient history which might take time to collect. Our models eliminate that dependency on such data.