A novel artificial intelligence based intensive care unit monitoring system: using physiological waveforms to identify sepsis

Abstract

A massive amount of multimodal data are continuously collected in the intensive care unit (ICU) along each patient stay, offering a great opportunity for the development of smart monitoring devices based on artificial intelligence (AI). The two main sources of relevant information collected in the ICU are the electronic health records (EHRs) and vital sign waveforms continuously recorded at the bedside. While EHRs are already widely processed by AI algorithms for prompt diagnosis and prognosis, AI-based assessments of the patients’ pathophysiological state using waveforms are less developed, and their use is still limited to real-time monitoring for basic visual vital sign feedback at the bedside. This study uses data from the MIMIC-III database (PhysioNet) to propose a novel AI approach in ICU patient monitoring that incorporates features estimated by a closed-loop cardiovascular model, with the specific goal of identifying sepsis within the first hour of admission. Our top benchmark results (AUROC = 0.92, AUPRC = 0.90) suggest that features derived by cardiovascular control models may play a key role in identifying sepsis, by continuous monitoring performed through advanced multivariate modelling of vital sign waveforms. This work lays foundations for a deeper data integration paradigm which will help clinicians in their decision-making processes.

This article is part of the theme issue ‘Advanced computation in cardiovascular physiology: new challenges and opportunities’.

1. Introduction

In the last 20 years, ICU has been recognized as one of the hospital departments where the use of artificial intelligence (AI) applications could contribute the most in the attempt to improve data exploitation and help clinicians in predicting patients’ critical conditions [1–3].

The hope for the near future is that data can be organized to automatically extract critical information in a context where only 10–20% of clinical decisions are evidence based [4]. Since randomized control trials (the gold standard in medicine to assess validity of procedures and medications) are long and expensive, a thorough retrospective processing and analysis of data collected during each hospitalization might be an effective way to validate specific hypotheses and improve clinical evidence. Unfortunately, most of the time this data iswasted [5,6].

Recently, advanced mathematical algorithms have allowed for retrospective analysis of accumulated stored data, and demonstrated their potential effectiveness in improving patients’ outcomes [7] and analysing ICU strategies [8], thus increasing the potential applications of AI techniques to clinical data.

Vital signs are continuously monitored from patients admitted in the ICU, generating data recorded at higher time resolution than the clinical information stored into the electronic health record (EHR). For example, the electrocardiogram (ECG) and arterial blood pressure (ABP) recordings are two of the most commonly recorded signals from which vital signs are derived. These physiological signals contain important information about the physiological and pathological state of the patient, with the potential for extraction of features whose use for developing predictive algorithms is still low. Some examples of studies that have considered vital waveforms include the detection and reduction of false alarms [9], the identification of atrial fibrillation [10], the prediction of hypotensive events [11] and the estimation of patients’ risk of in-hospital mortality [12].

The aim of this work is to validate a novel framework that uses recorded vital waveforms to infer patient conditions in the ICU independently from data stored in their EHR. In particular, the ultimate goal is to improve both patient monitoring and clinical state assessment through the extraction of meaningful physiological indices able to provide a more informed and continuously updated description of the patients’ state. Our study focuses on the applicability of this framework by introducing a design that exploits information available in the considered ICU setting in order to identify if patients have sepsis at admission.

(a) . Sepsis in the intensive care unit

According to the third international consensus definition of sepsis and septic shock [13], sepsis is a life-threatening organ dysfunction caused by a dysregulated host response to infection. This new definition (sepsis-3) focuses the attention on the life-threatening condition, organ dysfunction and abnormal host response to infection, thus modifying the point of view of previous definitions that relied on systemic inflammatory response syndrome (SIRS) only.

Sepsis occurrence ranges from 13.9% to 39.3% worldwide [14], and it is one of the major causes of mortality in the ICU, and its reported incidence is not decreasing. The final stage of the sepsis syndrome is septic shock, characterized by a strong hypotension despite fluid resuscitation and vasopressor administration, which shows a higher risk of mortality. Indeed, Driessen et al. [15] indicate that 47% of septic patients met the definition of septic shock according to the third definition of sepsis, and show a mortality rate of 38.9%.

Sepsis is also known to have a strong impact on the cardiovascular system, indeed it also induces a marked cardiac dysfunction. The first studies about the mechanisms behind sepsis-induced cardiac dysfunction (SICD) date back to the early 1980s, and they are still not completely understood. The main effects of sepsis on the cardiovascular system are highlighted in a recent review [16], where the author points out the first findings about the reduction of left ventricle ejection fraction (LVEF) and introduces the recent left ventricular peak systolic strain and peak systolic strain rate as more reliable indicators because they are less influenced by afterload than LVEF. Also, left ventricular diastolic dysfunction is indicated as very common in septic patients, thus describing SICD as an intrinsic global (systolic and diastolic) dysfunction of the whole heart. The review points out an impairment in cardiac function modulation induced also by cardiac autonomic dysfunction and not only by myocardial dysfunction. Cardiac autonomic dysfunctions can indeed be observed from an increased heart rate despite a strong reduction in sympathetically and vagally mediated heart rate variability (HRV) as well as a loss in the link between heart rate increase and baroreflex, which also suggests an impairment in the signal transduction pathways [16]. In 2001, Schmidt et al. [17] strengthened the key role of autonomic assessment in a critical care setting, especially focusing on the role of heart rate variability and the different reflex arches (e.g. chemoreflex and baroreflex) in frequently seen disorders such as brain injuries, myocardial infarction, multiple organ dysfunction syndrome (MODS) and sepsis. They showed how MODS and sepsis are coupled with an impairment of the autonomic function.

Nowadays, sepsis does not have a standard diagnostic test that allows for its identification. Therefore, clinical studies have focused the attention on the comparison of the discriminating abilities of commonly used scores like SIRS, quick sequential organ failure assessment (qSOFA) and National Early Warning Score (NEWS) [18–20] or on the comparison between standard clinical scores like SIRS, SOFA, qSOFA, Modified Early Warning Score (MEWS), National Early Warning Score (NEWS) and new scores based on machine learning models like the Risk of Sepsis score (RoS) [21] for the early identification of sepsis in patients at the emergency department.

(b) . Artificial intelligence in the intensive care unit

Despite the strong influence of sepsis on the cardiovascular system and cardiovascular control mechanisms, and a discrete ability of machine learning models in identifying sepsis with clinical data [22], only a few studies have conveyed features extracted from higher resolution signals into Machine Learning (ML) models in order to identify sepsis [23,24] and characterize septic subjects by the identification of different phenotype responses in severe sepsis patients [25]. Within this context, preliminary studies about sepsis identification and septic shock prediction [26,27] have already assessed the potential of continuous waveform monitoring of ICU patients through the application of different ML models.

After ICU admission, two main sources of data are available: the information contained in the EHRs and the data continuously recorded and displayed in the monitors, containing patients’ vitals. While EHRs are already widely processed by AI algorithms for prompt diagnosis and prognosis, AI-based assessments of the patient’s state using waveforms of vital signs are less developed and mainly used for real-time pathophysiological assessment. A proper combination of these two sources within an AI framework is expected to strongly improve patient monitoring by extending the sparse monitoring towards a continuous patient monitoring where the prospective software embedded in the monitoring device may constantly look at the patient’s state of health through the combination of all available information.

An inspirational example of this multimodal approach is given by Xu et al. [28] where the authors propose an attention-based deep learning algorithm which predicts patients’ length of stay and 24-h mortality risk combining high-resolution data (ECG sampled at 125 Hz), low-resolution data (e.g. mean arterial blood pressure sampled at 2 Hz) and uneven and sparse information from EHRs. However, this approach does not fully exploit the full potential of continuously recorded signals, since the measures fed to the neural network algorithm are simply segments of the raw signals and no specific processing techniques based on any physiological assumption on the system generating those signals (i.e. the cardiovascular system) are used to extract more informative and interpretable features. Another relevant example can be found in Barnaby et al. [29] where the authors proposed an ensemble of logistic regression models trained with different feature sets, including different combinations of laboratory measures, clinical variables and heart rate variability (HRV) measures, aimed at predicting future deterioration in patients presenting at emergency departments with sepsis. This approach proposes an interesting use of both clinical variables and vital waveforms, however limiting its waveform feature selection to just a few simple ECG-derived features.

Differently from these attempts where either all information are blindly fed to the developed software or all clinical variables are added to the ML algorithm, our framework proposes a more principled way of merging in the continuously streamed physiological sources. Indeed, the availability of streaming data sources can be used to reduce possible common problems when using only EHRs. This is also pointed out by Schinkel et al. [30], where, after the review of AI applications in sepsis diagnosis, prognosis and treatment assistance, authors analyse three main aspects in the development of predictive tools: the presence of bias and missing values and the applicability of developed models.

2. Material and methods

Our approach relies on the use of a subset of the EHR data for the identification of the target population, whereas the remaining EHR information can be included in predictive tools merging information from both EHRs and waveform data sources. This would imply the use of some of the information contained in the EHRs for the study design process and adding the remaining information with the continuous waveform features for the realization of the desired model.

We take advantage of the fact that signals like ECG and blood pressure, when properly processed, are able to provide additional knowledge on how the different underlying physiological mechanisms, e.g. the patients’ autonomic activity, behave and interact with the others, e.g. the cardiovascular system [31,32]. This additional point of view highlights the advantage of using all the wide spectrum of information contained in the EHRs, where many of the clinical features (e.g. laboratory test, antibiotics prescriptions, microbiology test) are considered as the gold standard or the best available markers for the identification of specific pathological conditions, providing reliable estimates of the patient cohort.

From this perspective, the presented study focuses the use of this original framework on the identification of sepsis. The population of interest, i.e. septic subjects, was identified through the use of measures entirely extracted from patients’ EHRs, while the developed models include features from cardiovascular modelling of time series derived from continuously recorded vital waveforms. This approach allows for the estimation of the patient’s state both when clinical variables are not yet available and its update at high time resolution. Additionally, this framework would reduce the problem of the amount of missing values introduced by Schinkel et al. [30]; indeed, as far as the recordings of interest are correctly acquired, all derived features will be constantly available.

(a) . Cohort and data selection

The data used in this study were extracted from the freely, publicly available, MIMIC-III database [33,34], containing EHRs and 22 317 continuous recordings of physiological signals from 10 282 distinct ICU patients admitted between 2001 and 2012 in the Beth Israel Deaconess Medical Center (BIDMC, Boston, MA, USA) ICU.

In our setting, patients are aligned according to their admission in ICU, thus identifying as septic subjects whoever met the sepsis criteria in a window of 48 h around the admission in ICU and considering waveform data during the first hour of admission. In this way, we mimic prospective real-world conditions where a patient that enters the ICU might be immediately put under the surveillance of the automated continuous monitoring system which requires only basic clinical information and a monitor recording its vital signs.

As stated in the Surviving Sepsis Campaign Guidelines, ‘early identification and appropriate management in the initial hours after sepsis develops improves outcomes’ [35], therefore, we decided to focus the study using data collected as close as possible to the patients’ admission in the ICU and specifically within their first hour of admission.

The availability and usability of 1 h recordings in the first hour of intensive care was defined according to the following characteristics:

• —Availability of clinical information.
• —The presence of both ECG, labelled as either I, II or ‘V’ lead, and ABP recordings.
• —Less than 50% of missing values within each of the first 1 h recording window for each patient’s stay.
• —Less than 50% of noise such as saturation, triggers, motion or sensor calibration loss.
• —Only subjects with $18\le \text{age}\le 89\hspace{0.17em}\text{yr}$ old at the admission in ICU were considered.

The selection criteria described above were applied to each candidate recording through a basic signal quality algorithm and processed with an internally developed signal annotator, which provided 538 annotated waveforms. After manual inspection and proper correction of erroneous annotations the final number of high-quality available waveforms was 144, which dropped to 142 after filtering for the defined age requirements. The resulting 142 waveforms were labelled as ‘septic’ (S) or ‘control’ (C) whether they did or did not belong to a subject that developed sepsis in a 48 h window around the admission in ICU, according to the third definition of sepsis [13], thus resulting in 71 septic patients and 71 controls, whose characteristics are shown in table 1.

Table 1.

Population characteristics: gender, age, length of stay (LOS), 90 d mortality, total number of patients, percentages of comorbidities as congestive heart failure (CHF), diabetes (Diab), renal failure (Renal), liver disease (LD) and coagulopathy (CGPT). Additional clinical information includes percentages of subjects undergoing fluid, vasoactive agent (or only vasopressor), sedative administrations, mechanical ventilation and their admission type (emergency (EMRG) or elective (ELCT)). The last rows show the distributions of sepsis and septic shock onsets in hours from the ICU admission time. ($C=\text{control}$, $S=\text{Septic}$ and $\text{SS}=\text{Septic Shock groups}$).

demographics
	gender (F)	age (yr)	LOS (days)	90d mortality		tot.
$C$	52%	56 (45–66)	1.2 (0.9–2.8)	8%		71
$S$	45%	59 (45–67)	1.9 (1.1–3.9)	14%		71
comorbidities
	CHF	diab.	renal	LD	CGPT	hyp.
$C$	4%	14%	6%	15%	3%	42%
$S$	6%	14%	7%	14%	8%	31%
other clinical information
	fluids	vaosactive (pressors)	sedatives	ventilation	EMRG	ELCT
$C$	65%	18% (11%)	31%	24%	38%	62%
$S$	44%	30% (14%)	41%	45%	56%	44%
sepsis onset distribution
[hr]	$[-24,-12]$	$[-12,-6]$	$[-6,0]$	$[0,6]$	$[6,12]$	$[12,24]$
$S$	3%	6%	69%	6%	10%	7%
septic shock onset distribution
[hr]	$[0,0.5]$	$[0.5,1]$	$[1,6]$	$[6,12]$	$[12,24]$	$[24,30]$
$\text{SS}$	7%	3%	1%	3%	3%	1%

At the base of the third definition of sepsis, there are two key elements that can be used to define a sepsis diagnosis: a change in SOFA score and the simultaneous presence of infection. Consequently, a subject was labelled as septic whether he was subjected to the administration of antibiotics and to sampling of bodily fluids for culture, here used as a markers of suspicion of infection, and showed a change in SOFA scores $\ge$2 [36].

ECG and ABP continuous waveforms allow for the extraction of features linked to the physiological processes related to the autonomic nervous system and cardiovascular control mechanisms, both strongly influenced by sepsis [16]. We annotated $R$-peak time positions from the ECG, and onset, systolic and diastolic time positions and values from the ABP (figure 1). The combination of synchronized $R$-peak occurrences and pressure onset ($O_t$) occurrences (computed as the point with maximum first derivative between the previous diastolic and current systolic points) allowed for the computation of the pulse arrival time (PAT), while through the systolic (SAP) and diastolic (DAP) time series we extracted the pulse arterial pressure (PPress) as follows: $\text{PAT}(i)=O_t(i)-R(i)$ and $\text{PPress}(i)=\text{SAP}(i)-\text{DAP}(i)$.

Figure 1.

(a) Exemplary electrocardiographic (ECG) and arterial blood pressure(ABP) waveforms with the extracted fiducial points highlighted. (b) Zoom on 5 min recording of the extracted time-series, i.e. estimated RR interval (muRR) (blue/continuous line) superimposed to measured RR interval (red/continuous line), and pulse arrival time (PAT) (orange/dashed-dot line) in the top panel, and systolic (SAP) (blue/dashed-dot line), diastolic (DAP) (purple/solid-dot line), mean (MAP) (yellow/dashed line) and pulse (PP) (orange/continuous line) arterial pressure in the bottom panel. (Online version in colour.)

From the RR series, we extracted the heart rate variability (HRV) linear (AVNN, SDNN, SDANN, SDSD, TRI and TINN) and nonlinear features (SD1, SD2, SD1/SD2 called SD_ratio, DFA long-term coefficient, Hurst (H) exponent with the Higuchi’s method and Sample Entropy, largest Lyapunov exponent and Correlation Dimension (CorrDim) with the following input parameters: embedding dimension set to 2, lag equal to 1 and radius set to 0.2*Var(RR) [37]).

To provide a more continuous assessment in time, the RR interval probability of occurrence was characterized by using a point process framework [38,39], a probabilistic approach where the parameters of the instantaneous average RR interval series (${\mu }_{\text{RR}}(t)$) are extracted by maximizing the likelihood of an Inverse Gaussian (IG) probability function ($p(t)$) as follows:

$$p(t)={\left(\frac{\theta }{2\pi {(t-u_k)}^3}\right)}^{1/2}\exp (-\frac{\theta (t-u_k-{\mu }_{\text{RR}}(t))}{2{\mu }_{\text{RR}}{(t)}^2(t-u_k)}),$$ 2.1

where $\theta >0$ is the IG shape parameter and $u_k$ the time of the $R_k$ event and consequently, ${\text{RR}}_k=u_k-u_{k-1}$. The instantaneous average RR interval series ${\mu }_{\text{RR}}(t)$ were modelled according to the following bivariate autoregressive model:

$${\mu }_{\text{RR}}(t)=a_{110}+\sum _{i=1}^p{a}_{11i}(t){\text{RR}}_{t-i}+\sum _{j=1}^q{a}_{12j}(t){\text{SAP}}_{t-j}$$ 2.2

and

$${\mu }_{\text{SAP}}(t)=a_{220}+\sum _{i=1}^r{a}_{21i}(t){\mu }_{\text{RR},(t-i)}+\sum _{j=1}^s{a}_{22j}(t){\text{SAP}}_{t-j},$$ 2.3

where ${\mu }_{\text{RR},(t-i)}$ represents the average RR interval at the time of the systolic event and the order of the AR model $p=q=r=s=7$ was chosen as the lowest order that allowed a good fitting with the IG distribution, assessed looking at the KS-distance and at the number of points out of 95% confidence intervals of the autocorrelation function of the residuals, and $a_{11i}$ and $a_{12j}$ for $i,j=0,1,2,\dots ,p$ represent the AR model parameters estimated within the point process framework by maximum-likelihood estimation for the extraction of the instantaneous average RR interval; similarly, $a_{21i}$ and $a_{22j}$ are the AR coefficients estimated through weighted least squares for the estimation of the average systolic arterial pressure series ${\mu }_{\text{SAP}}$.

Using this mixed modelling approach, we obtained a time varying high resolution estimation and spectral representation of both RR and SAP, their cross-spectrum, and the Baroreflex and Feedforward closed-loop gains as shown in [40].

The proposed solution for feature extraction is applied in order to deal with possible non-stationarities of the derived time series. Classical linear features were extracted from Normal-to-Normal beats, and spectral features were computed on 5-min windows and successively averaged assuming stationarity in smaller segments. Nonlinear features were computed on the whole time series because of the need for long recordings for such indices. In this case, non-stationarities could be considered as cardiovascular responses that patients may show during their stay in ICU, thus reflecting their state (e.g. ‘endogenous’ non-stationarities like sudden pressure drops). In order to account for ‘exogenous’ non-stationarities (e.g. induced by the administration of a specific drug) models were informed through treatment-related confounders.

We further averaged the frequency content of continuously estimated features for each of the following frequency ranges: total power (whole spectrum), VLF (0.003–0.04 Hz), LF (0.04–0.15 Hz) and HF (0.15–0.45 Hz). We then divided the obtained averaged signals of each subject into5-min windows and we computed the averages of 5-min window averages in order to obtain one measure for each frequency range and for each subject. Normalized LF (LFn) and normalized HF (HFn) as well as LF/HF were subjected to the previously presented processing and extracted as follows: RR: $\text{LFn}=\text{LF}/(\text{TOT–VLF})$, $\text{HFn}=\text{HF}/(\text{TOT–VLF})$; SAP (and cross-spectrum): $\text{LFn}=\text{LF/TOT}$, $\text{HFn}=\text{HF/TOT}$, LF/HF not computed.

These 5-min window statistics were computed for ${\mu }_{\text{RR}}$, ${\sigma }_{\text{RR}}$, $\theta$ and for the spectral components (total power (TOTPWR), VLF, LF and HF ranges) of ${\mu }_{\text{RR}}$, DAP, PAT and PPress time series. Also, the slope of the linear regression of the power spectrum in the log–log scale ($\alpha$) and the averages and standard deviations for SAP, DAP, PAT and PPress were computed.

Overall, 68 waveform-related features, age, gender and five boolean features (sedative administration, vasopressor administration, mechanical ventilation, diabetes and hypertension) were considered, resulting in a total of 75 features. The set of 142 observations was divided into training and test sets with a stratified hold-out 80–20% partition in order to keep a subset of observations hidden to test the developed algorithms.

Several training procedures were explored by applying the following steps briefly analysed in the next sections: feature transformation, feature scaling, feature selection and classifier training.

(i) . Feature transformation and scaling

The non-boolean features were subjected to different transformations in order to evaluate how the performance varied when handling outliers according to different transformation methods. The considered transformations were: no transformation (No Transformation), power transformations, a family of functions that transforms data in order to make them more Gaussian-like according to specific power laws [41,42] and named Box-Cox and Yeo-Johnson, and quantile transformations which, according to a quantile function, map the quantiles of the input data with those of a Gaussian or Uniform distribution, respectively, named Quantile Gaussian and Quantile Uniform.

The following scaling procedures were applied:

• —One normalization procedure according to a min–max scaling (named MinMaxScaler), which maps the input data into a predefined range which was set to $[0,1]$ according to the following rule: $\overline{x}=(x-min(x))/(max(x)-min(x)).$
• —Two standardization procedures which centre and normalize the data considering their distributions and according to the following formulations: $\overline{x}=(x-\text{mean}(x))/\text{var}(x)$ following indicated as StandardScaler and $\overline{x}=(x-Q1(x))/(Q3(x)-Q1(x))$ named RobustScaler, where $\text{mean}(x)$ and $\text{var}(x)$ represent the mean and variance of the current feature $x$ and $Q1(x)$ and $Q3(x)$ are the first and third quartiles.

(ii) . Feature selection and classification

We considered seven feature selection procedures in order to investigate whether the classification algorithms may benefit from a reduction of the feature space:

• —No Feature Selection (None) where the whole set of features is directly given to the classification algorithm.
• —$K$ Best Selection, which selects the $K$ features that showed the highest ${\chi }^2$ or Mutual Information scores, respectively, called KBEST_CHI2 and KBEST_MI.
• —Recursive Feature Elimination with number of folds set to 10. The models considered in this approach were Logistic Regression with L2 penalty (i.e. Ridge regression) and support vector machines with linear kernels, RFECV_LR and RFECV_SVM, respectively.
• —Model Selection which selects the best features according to a Logistic Regression model with penalty L1 (i.e. LASSO penalty), so this method was called LASSO.

Logistic Regression (LR), Trees (TREE), Support Vector Machines with linear and radial basis kernels (SVM_lin and SVM_rbf), Extreme Gradient Boosted Trees (XGB), $K$-Nearest Neighbours (KNN) and Multi-layer Perceptron (MLP) binary classifiers were trained with threefold cross validation through a grid search optimization of the hyperparameters.

The most important performance metrics considered were the Area Under the Receiver Operating Characteristic (AUROC) Curve and Area Under the Precision Recall Curve (AUPRC) in order to evaluate the generic ability of the classifiers in separating the two classes and in balancing both positive predicted value and sensitivity. 95% confidence intervals for the introduced measures were also computed with 200 bootstrapped observations from the test set. Recall, Precision, Specificity and F₁ measures were extracted to assess the ability in discriminating the two classes with a threshold set on the training observations.

Explanation of the decision rules applied by the best obtained model in predicting sepsis is shown with partial dependence plots between the most relevant features.

In order to evaluate model decision rules and feature importance, all extracted features were subjected to a statistical analysis for assessing the significance of each extracted variable in predicting sepsis. Correction for confounding factors was also performed with a multivariate logistic regression model, considering the following as major confounders: vasoactive agents and sedatives administration, mechanical ventilation, gender, hypertension, diabetes and age. In order to have comparable results, each feature was subjected to the same transformation and scaling procedures. The analyses were performed with PostgreSQL 11.1, Matlab R2019b and Python 3.8.3.

3. Results

(a) . Model performance

Figure 2 shows the AUROC values obtained when testing all 630 developed algorithms, respectively, divided according to the distinct classifiers. It can be observed how linear models (SVC_lin, LR) and multilayer perceptron (MLP) are those with the overall better performance on the test set, showing median (and 95% CI) AUROCs equal to 0.72 ($\pm 0.13$), 0.73 ($\pm 0.13$), 0.72 ($\pm 0.12$), respectively. These results are in agreement with the median (and 95% CI) 3-fold cross-validation AUROC scores on the 80% training set, which equal to 0.74 ($\pm 0.15$), 0.72 ($\pm 0.17$) and 0.73 ($\pm 0.17$), respectively, for SVC_lin, LR and MLP models. Furthermore, all feature transformation procedures performed similarly and generally better than models trained without applying any transformation as expected.

Figure 2.

AUROC values of different classifiers: $K$-nearest neighbour (KNN), SVM with linear kernel (SVC_lin), SVM with radial kernel (SVC_rbf), Multilayer Perceptron (MLP), Logistic Regression (LR) and Extreme Gradient Boosting (XGB). (Online version in colour.)

Figure 3 expands on the performance results, showing the differences between different feature selection approaches. It can be noted that the full set of features performed better than those obtained extracting only a subset of them.

Figure 3.

AUROC values for different classifiers and feature selection approaches: no feature selection (None), recursive feature elimination with LR and SVM models (RFECV_LR, RFECV_SVM), $K$ best selection with ${\chi }^2$ and Mutual Information criteria (KBEST_CHI2, KBEST_MI) and model selection with LASSO penalty (LASSO). (Online version in colour.)

Tree-based algorithms (TREE, XGB) have very similar results, generally lower than the other models, showing that they are not sensitive to different data transformation and data scaling, whereas differences are mainly related to the starting feature subset.

The 10 best performing algorithms on the test set are shown in table 2 in descending order of AUROC values from top to bottom. The best methods $(\text{AUROC}\ge 0.9)$ are those obtained with Gaussian quantile transformers, without the application of any feature selection algorithm and after a standardization procedure with linear classifiers.

Table 2.

The 10 best results obtained in sepsis identification on the test set with different classification pipelines are shown and presented in descending order of AUROC values from top to bottom.

sepsis identification results
transformer	feat sel	feat scal	classifier	AUROC	AUPRC	threshold	recall	precision	specificity	F1
Quant_Gaus	none	Robust	SVC_lin	0.92 (0.79–0.99)	0.90 (0.73–0.99)	0.47	0.86	0.80	0.80	0.83
Quant_Gaus	none	Robust	LR	0.91 (0.78–1)	0.90 (0.72–1)	0.50	0.79	0.92	0.93	0.85
Quant_Gaus	none	StdScal	SVC_lin	0.91 (0.79–0.99)	0.90 (0.74–0.98)	0.51	0.57	0.89	0.93	0.70
Quant_Gaus	none	StdScal	LR	0.90 (0.76–1)	0.89 (0.70–1)	0.51	0.71	0.91	0.93	0.80
Quant_Gaus	none	Robust	MLP	0.89 (0.73–0.99)	0.84 (0.59–0.99)	0.54	0.71	0.91	0.93	0.80
Quant_Gaus	none	StdScal	MLP	0.86 (0.71–0.98)	0.84 (0.61–0.98)	0.46	0.57	0.80	0.87	0.67
Yeo–John	none	StdScal	LR	0.85 (0.70–0.96)	0.85 (0.67–0.96)	0.49	0.50	0.88	0.93	0.64
Box–Cox	none	StdScal	LR	0.85 (0.69–0.96)	0.85 (0.67–0.96)	0.43	0.50	0.64	0.73	0.56
Yeo–John	none	Robust	SVC_lin	0.85 (0.71–0.96)	0.85 (0.68–0.96)	0.49	0.50	0.78	0.87	0.61
Box–Cox	none	Robust	SVC_lin	0.85 (0.70–0.96)	0.84 (0.62–0.96)	0.49	0.64	0.75	0.8	0.69

In particular, the first two results in table 2, in addition to general discriminating abilities, indicate that good performances, evaluated through AUROC and AUPRC, can be achieved, and demonstrate high compliance with the selected criteria for the estimation of the discriminating threshold. Indeed, the classifier with the highest AUROC (0.92), the SVC_lin classifier (see performance in figure 4) also shows the highest Recall (0.86) and reasonably good Precision (0.80), Specificity (0.80) and F₁ (0.83). The highest Specificity (0.93), Precision (0.92) and F₁ (0.85) scores were obtained with the second-ranked classifier ($\text{AUROC}=0.91$), i.e. the Logistic Regression approach.

Figure 4.

Receiver operating characteristic (left) and precision recall (right) curves for the Support Vector Machine classifier with linear kernel. Both training (dashed-dotted line) and testing (solid line) performances are shown. Reference values, area under the curves and their 95% bootstrapped confidence intervals are indicated in the legend. (Online version in colour.)

(b) . Feature importance

Table 3 shows the first 30 most important features of the best-performing model among which the standard deviations of pulse arrival time (SDPAT), SD ratio from RR series (SD_ratio), high-frequency powers of PAT (PAT_HF) and averages of PAT (AVPAT) show the highest importance, together with the ventilation flag (vent_flag) and the spectral parameters from bivariate point process modelling (avg_srr_hfn, avg_scr_hf, avg_srr_lfn, avg_scr_hfn, avg_sbp_hfn, avg_srr_lf, avg_srr_lfhf) and the average of Baroreflex gain in both LF and HF spectral ranges (avg_gain21_lf and avg_gain21_hf, respectively).

Table 3.

A subset of the 30 most important features in descending order of importance from top to bottom for the SVC_lin model. Pulse arrival time (PAT)-related features are among the most important ones followed by the average Baroreflex gain in the LF and HF frequency ranges (avg_gain21_lf ,avg_gain21_hf).

feature importance
importance	SVC_lin	importance	SVC_lin
1 (highest)	SDPAT	16	avg_scr_hf
2	SD_ratio	17	logRMSSD
3	PAT_HF	18	SDSD
4	AVPAT	19	PAT_$\alpha$
5	vent_flag	20	TRI
6	NN50	21	avg_srr_lfn
7	pNN50	22	avg_Mu
8	AVSAP	23	SDSAP
9	avg_srr_hfn	24	hypertension
10	DAP_VLF	25	PPress_TOTPWR
11	Hurst_exp	26	diabetes
12	avg_gain21_lf	27	avg_scr_hfn
13	SDDAP	28	avg_sbp_hfn
14	avg_gain21_hf	29	avg_srr_lf
15	gender	30 (lower)	avg_srr_lfhf

Note also that SVC_rbf classifiers show the most variable results, often leading to very low AUROC values lower than the random classifier ($\text{AUROC}=0.5$). Because of this, their results were excluded from subsequent analyses.

(c) . Statistical comparison

Results from statistical analysis are presented in table 4 and show significant ($p<0.05$) features in predicting sepsis; moreover, in italics are highlighted those 5 out of 10 of the most important features of the best obtained ML model that are also statistically significant predictors of sepsis when correcting for confoundings. NN50 and pNN50 are the features with the highest odds are equal to 2.959 and 2.743, respectively; pulse arrival time-related features, SDPAT and AVPAT, also show $\text{odds}>1$ (2.370 and 1.787), while SD_ratio odds are equal to 0.581. It is worth mentioning also pressure related indices like fluctuations in very low frequencies of diastolic and pulse arterial pressure (DAP_VLF, PPress_VLF) and systolic and pulse arterial pressure variability (SDSAP, SDPPress) with $\text{odds}>1$.

Table 4.

Features, coefficients, odds, $p$-values and 95% confidence intervals of statistically significant variables after correction for confoundings. In italics are shown those features ranked among the 10 most important in the best developed model.

statistical analysis results
feature	coeff.	odds	$p$-value	coeff. 95% CI
NN50	1.085	2.959	0.0191	0.178–1.992
pNN50	1.009	2.743	0.0290	0.103–1.915
DAP_VLF	0.930	2.534	0.0006	0.396–1.463
PPress_VLF	0.912	2.489	0.0014	0.353–1.471
SDPAT	0.863	2.370	0.0019	0.318–1.408
SDSAP	0.771	2.162	0.0044	0.241–1.301
avg_sbp_vlf	0.722	2.059	0.0051	0.217–1.227
PAT_VLF	0.671	1.956	0.0076	0.179–1.163
PPress_LF	0.647	1.910	0.0125	0.139–1.154
PPress_TOTPWR	0.624	1.867	0.0187	0.104–1.145
SDPPress	0.613	1.846	0.0128	0.131–1.095
AVPAT	0.580	1.787	0.0207	0.088–1.072
avg_scr_tot	0.534	1.706	0.0301	0.051–1.017
avg_sbp_lf	0.515	1.673	0.0373	0.030–1.000
SD_ratio	−0.542	0.582	0.0282	$-1.026\text{ to }-0.058$
PAT_spect_slope	−0.543	0.581	0.0189	$-0.996\text{ to }-0.090$

(d) . Model decision rules

Explainability analysis of the best performing model executed on the test set with partial dependence plots (figure 5) shows how SDPAT and AVPAT are directly correlated with an increase in sepsis probability, whereas SD_ratio and PAT_HF are inversely correlated. Baroreflex (avg_gain_21_lf) and Feedforward (avg_gain_12_lf) show, respectively, how the latter is not highly correlated with an increase in sepsis probability while a low Baroreflex determines an increase in probability. Contemporaneously, high average systolic pressure (AVSAP) and heartbeat (i.e. low ${\mu }_{\text{RR}}$) are directly correlated with sepsis probability. The last row shows how model output probabilities are higher in correspondence with low RR, low Baroreflex and high AVSAP.

Figure 5.

Partial dependence plots of most important (SD_ratio, PAT_HF, AVPAT and SDPAT) features in predicting sepsis and the corresponding role of most relevant cardiovascular indices (AVSAP, $\mu \mathrm{\_}\text{RR}$, Baroreflex and Feedforward). (Online version in colour.)

4. Discussion

Our results mainly suggest that, when dealing with data from patients in a highly uncontrolled environment such as the ICU, any hidden features extracted through mathematical modelling based knowledge of the cardiovascular system are of primary importance. Despite the relatively low number of patients, the two best models show similar high-level performance both in training and in testing stages. Our benchmarking of several models confirms that very high performances can be obtained with features mainly extracted from waveforms.

The considered linear models allow for the possibility to extract feature importance. Indeed, considering the previously listed PAT-derived and spectral features, our modelling effort points at a key role of the interaction between heart activity and pressure response in predicting progression of the infectious state. The results of the model behaviour agree with those obtained in statistical analysis with AVPAT, SDPAT and SD_ratio, among others, which showed statistically significant correlation with the outcome and with odds ratios that point at the same results. Indeed, both approaches suggest a direct correlation between AVPAT and SDPAT and an inverse correlation of SD_ratio with sepsis. Results on the role of cardiovascular indices in the model are also in line with literature, indeed the sepsis risk increases for high values of pressure and heart rate, and low values of Baroreflex sensitivity [16,17], which should instead be responsive for such high pressure in physiological conditions. This results confirm that the model is exploiting cardiovascular control features somewhat in line with clinical evidence and reasoning.

Our feature transformation algorithms allowed for effective outlier handling. However, we did not find strong differences among feature scaling approaches, with the only consideration that RobustScaler often showed a slight improvement in performance than other approaches that may also be attributed to the exclusion of outliers for the extraction of statistical measures.

The presented best results outperform previously available classification efforts. Our classification scores (i.e. AUROC and AUPRC) are greater than or equal to those obtained by other studies (NR=Not Reported): Haydar et al. reported SIRS ($\text{SE}=0.96$, $\text{SP}=0.06$, $\text{AUROC}=0.51$, $\text{AUPRC}=\text{NR}$) and qSOFA ($\text{SE}=0.91$, $\text{SP}=0.46$, $\text{AUROC}=0.68$, $\text{AUPRC}=\text{NR}$) when predicting sepsis-related mortality at the emergency department [18], whereas when considering the identification of severe sepsis and septic shock Usman et al. [19] obtained the following NEWS ($\text{SE}=0.84$, $\text{SP}=0.85$, $\text{AUROC}=0.91$, $\text{AUPRC}=\text{NR}$), SIRS ($\text{SE}=0.86$, $\text{SP}=0.79$, $\text{AUROC}=0.88$, $\text{AUPRC}=\text{NR}$) and qSOFA ($\text{SE}=0.29$, $\text{SP}=0.989$, $\text{AUROC}=0.81$, $\text{AUPRC}=\text{NR}$) scores. In [20] a higher discriminating ability of SIRS ($\text{SE}=0.81$, $\text{SP}=0.33$, $\text{AUROC}=0.65$, $\text{AUPRC}=\text{NR}$) than qSOFA ($\text{SE}=\text{NR}$, $\text{SP}=\text{NR}$, $\text{AUROC}=0.58$, $\text{AUPRC}=\text{NR}$) score was recently reported in a study predicting an ultimate infection diagnosis. Finally, we obtained slightly lower results than the RoS scores reported in [21] considering data collected within a 1 h temporal window ($\text{SE}=0.68$, $\text{SP}=0.96$, $\text{AUROC}=0.93$, $\text{AUPRC}=\text{NR}$). However, this score is obtained with data extracted from EHRs, including lactate as the most important feature (which is not always easily available) and creatinine levels among others, resulting in performance measures at higher risk of being biased. Indeed, the authors identify the target population according to the Rhee clinical surveillance criteria [43] which is based on measures like lactate and creatinine, which are included in the total feature set used to train and test the model. Of note, detection time was set to 60 min for our setup (the required waveform recording duration) compared to SIRS (47.1 min) and qSOFA (84 min) documentation times from arrival in the emergency department [18].

These results confirm the strong ability of the proposed modelling approach in identifying septic patients at admission in ICU. The proposed framework takes advantage of the large amount of information contained in the continuously recorded vital signs, allowing for the extraction of several indices related to cardiovascular and cardiovascular control systems, which are known to be affected by sepsis.

5. Clinical relevance

Early recognition and treatment of sepsis have been proven to have a strong impact on reducing patient mortality. In the last few years, the importance of prompt initiation of several procedures, like fluid and vasopressor administration, lactate and blood culture measurements and early administration of IV broad spectrum antibiotics, has significantly increased [44,45]. The time needed to complete the procedures required by the sepsis protocol has reduced significantly: the Surviving Sepsis Campaign (SSC) frequently updated the guidelines moving from a hour-6 bundle to hour-3 [46] and finally to a hour-1 bundle [47], thus strengthening the importance of more rapid completion (i.e. within 1 h of presentation) of the sepsis protocol.

The developed model is not aimed at predicting the occurrence of sepsis but is focused on the identification of this pathological state with features from physiological waveforms. Overall, our results validate the importance of including continuously recorded vital signs. Indeed, this modelling approach has the advantage of using only a few clinical features that are easily available at the beginning of the ICU stay (patients’ comorbidities, age and undergoing treatments). These features are enriched through additional information about the underlying physiological mechanisms of the patients, extracted from the monitored vital waveforms and easily available at the bedside. Also, they allow for the estimation of inner mechanisms not directly measurable, like the Baroreflex gain, which can be of help in a more rapid identification of sepsis as well as in the monitoring of the physiological response to initiated treatments in the early ICU stay. Our models indeed contain a large set of physiology-based features that convey information about the autonomic regulation of heart activity and blood pressure. Time-varying features, when displayed, might also provide the possibility to assess in real time the patients’ autonomic response to interventions.

A schematic potential representation of a prospective monitoring system is shown in figure 6, where RR, SAP, mean arterial pressure (MAP), sympatho-vagal balance (LF/HF) and Baroreflex gain are shown for a specific patient in addition to other features fed to the model from the EHRs such as age, gender and comorbidities, as well as the current treatment to which the patient is subjected (intravenous administration of crystalloids) and the estimated sepsis risk. Note that the window length has been zoomed for clarity of presentation, whereas the sepsis risk is computed also including the previous times, i.e. extracting features from a 1 h recording window as we have proposed in our analysis. Here, the exemplary bedside monitor scenario is based on actual real data from a patient belonging to the database used in this study. Several trends are observable in the traces within a 1 h window. In the first 500 s of the selected hour a steady increase in RR (decrease in heartrate) can be observed, causing a parallel progressive decrease in blood pressure. At around 2800 s, we can observe an increase in sympathetic activation, as indicated by the LF/HF index, leading to the corresponding localized decreases in the RR values (i.e. rises in heart rate): the system is trying (with no success) to restore blood pressure levels (evidenced by the black empty boxes). In this time window, the continuous decrease of blood pressure is identified by the monitoring system as a dangerous condition and marked with yellow (light grey) circles. The observed responses lead to an unstable state (very low pressure levels) starting from the 2950 s mark, when Baroreflex starts to fail and less sympathetic action can be observed (vasovagal state), confirmed by a sudden increase in RR values despite the very low pressure and a predominant vagal activity with a relatively low Baroreflex gain, a very critical situation. Here AI algorithms can take advantage of the available continuously streaming information in order to provide an alarm, e.g. a colour-coded recommendation going from yellow/light grey (warning) to red/dark grey (alarm), which implements the proposed continuous physiological monitoring paradigm. After intervention by the clinicians, the RR interval decreases sharply (heartbeat increases) and restores both blood pressure and Baroreflex, coded in figure in green/grey.

Figure 6.

Schematic of a prospective continuous monitoring system portraying RR interval series, systolic and mean arterial pressure series (SAP and MAP), Sympatho-vagal balance (RR-LF/HF) and Baroreflex gain as well as clinical information coming from EHR and the estimated sepsis risk. (Online version in colour.)

The development of a tool like the one portrayed, able to collect and process vital waveforms in order to provide useful information to the clinician, paves the way for the development of higher level algorithms that may optimize treatment delivery and continuously assist in the follow-up of the patient. The potential of continuous monitoring through vital waveform recording and processing will be assessed in further studies, exploring the usefulness of the derived sepsis risk score to continuously assess the infectious state during the whole stay of the patient. Indeed, the considered study design relies on the ICU admission as a different aligning time for distinct patients and so it does not refer to any specific time interval with respect to the sepsis onset. This particular set-up may be a good starting point for the continuous and real-time assessment of the developed score, which can also be considered as a guide for evaluating the progression of the infection, intended as a worsening or improvement of the patient’s state of health, as well as a guide for the development of patient-specific treatment strategies.

6. Limitations and future developments

The presented work has several limitations that need to be taken into consideration. First, the proposed study design (i.e. the alignment of patients according to the admission in ICU, and not to sepsis onset time) likely increases uncertainty (variability) in the considered cohort. Indeed, the limited data available is the number one limitation. A much larger population is certainly needed to deal with the different stages of sepsis evolution and to develop prediction algorithms that may be used in a more realistic ICU scenario. Additionally, as the inclusion criteria and the monitoring requirements (i.e. presence of an invasive arterial blood pressure line and high-quality waveforms) might somehow influence the extracted population of patients, we always recommend to adjust for the inclusion criteria before including this study in any meta-analysis. Second, the presented results are aimed at testing the feasibility of sepsis recognition within an hour using exclusively information from waveforms. We therefore focused our quantitative analysis on benchmarking several models on a 20% hold-out partition of data to test the proposed physiological monitoring technique. On the other hand, if we were focusing on the predictive power of our framework, these models could be validated only on a larger population, and possibly from multi-centre databases, in order to reliably estimate the actual discrimination power for clinical use. We aim to address these limitations in future studies by relaxing the inclusion criteria and therefore further increasing the population study.

7. Conclusion

The presented results show that during the first hour of the ICU stay, when features coming from blood gas analysis or other laboratory and clinical microbiology measures are still not available, the potential use of continuous monitoring with vital sign physiological waveforms recorded at the bedside is able to provide essential features to the AI algorithms, giving them enough information to automatically detect sepsis. The proposed approach both addresses the sepsis identification problem and provides a continuous assessment of the patient’s state. The general framework is based on the extraction of additional features from the continuously recorded vital signs, allowing for the computation of features which provides to clinicians an estimate of the inner cardiovascular system functionalities and their interactions.

The main novelty of our framework is that the capacity of our models to extract physiological parameters allows for the quantification of underlying physiological mechanisms that can help the physician characterize the patient’s state, and may even explain specific physiological events associated with sepsis. In the considered ICU scenario, the inclusion criteria were quite influential and limited the application to a population that had to meet stringent requirements (primarily the presence of an arterial blood pressure line and high-quality waveforms). In this regard, we advocate for a near future ICU scenario where a prospective prediction model could be applied to patients that meet the inclusion criteria at a much higher percentage, as recording devices are becoming more and more accurate, far less noisy and less invasive. Even with limitations, our models provide significant evidence that the physiological waveforms contain critical information about sepsis, offering a basis for possible future applications for personalized treatment strategy optimization and the early assessment of patient’s responses to interventions.

Efforts should definitely be directed towards the inclusion of these rich sources of information, as well as to the development of tools that will improve the applicability of vital waveform processing in real time. Advancements in this direction will likely yield important breakthroughs in automatic clinical state assessment, in supporting clinical decision making and in providing an informed and accurate continuous monitoring of patients’ health conditions in the ICU.

Data accessibility

GitHub Repository: https://github.com/MaximilianoMoll/Sepsis-Identification.

Authors' contributions

All authors conceived and designed the study. M.M. performed the data extraction, analysis and wrote the first draft of this manuscript with inputs from L.H.L. and R.B.. L.H.L., R.G.M. and R.B. revised the manuscript. All authors read and approved the manuscript.

Competing interests

We declare we have no competing interests.

Funding

This project was partially supported by ‘Progetto Roberto Rocca, MIT-Italy Program’ and the NIH grant no. 5R01 EB017205.