Automatic Classification of Sedation Levels in ICU Patients using Heart Rate Variability

Abstract

Objective

To explore the potential value of HRV features for automated monitoring of sedation levels in mechanically ventilated ICU patients.

Methods

ECG recordings from 40 mechanically ventilated adult patients receiving sedatives in an ICU setting were utilized to develop and test the proposed automated system. Richmond Agitation-Sedation Scale (RASS) scores were acquired prospectively to assess patient sedation levels, and were used as ground truth. RASS scores were grouped into four levels, denoted “unarousable” (RASS = -5,-4), “sedated” (-3,-2,-1), “awake” (0), “agitated” (+1,+2,+3,+4). A multi-class support vector machine algorithm was used for classification. Classifier training and performance evaluations were carried out using leave-one-out cross validation.

Results

An overall accuracy of 69% was achieved for discriminating between the 4 levels of sedation. The proposed system was able to reliably discriminate (accuracy = 79%) between sedated (RASS <0) and non-sedated states (RASS >0).

Conclusions

With further refinement, the methodology reported herein could lead to a fully automated system for depth of sedation monitoring. By enabling monitoring to be continuous, such technology may help clinical staff to monitor sedation levels more effectively and to reduce complications related to over- and under-sedation.

Introduction

Critically ill mechanically ventilated patients in the intensive care unit (ICU) are often sedated to facilitate ventilation, analgesia, relief from psychological stress, and injury prevention (1). Clinicians must be vigilant to avoid over- or under sedation, both of which can lead to adverse patient outcomes. The current standard of care for assessing sedation is to use clinically validated behavioral assessment scales performed by health care providers (e.g. nurses), such as the RASS score (2). However, behavioral scales are ultimately subjective, relying on experience and clinical observation, and are necessarily performed only intermittently. Augmenting behavioral sedation scales with a physiologically-based monitoring technology, if it existed, could potentially offer greater robustness and reliability and could be performed continuously, enabling more judicious titration of sedatives and reduced sedative-related adverse events.

Several EEG-based indicators of the depth of general anesthesia have been developed, including the BIS (Aspect Medical Systems, Newton, MA) (3) and M-entropy (GE Healthcare, Helsinki, Finland) (4) monitors. However, relatively little is known about their validity for monitoring sedation in ICU patients, and some studies suggest that they may be unreliable at light and deep levels of sedation (5). Moreover, brain monitoring is not yet routinely performed in ICU patients.

By contrast, the electrocardiogram (ECG) is routinely used in ICU care to monitor cardiovascular function but has not been intensively studied for the purpose of monitoring sedation. Several features can be extracted from the ECG related to the pattern of variation in beat-to-beat intervals. Such features are collectively known as measures of heart rate variability (HRV). HRV analysis is a noninvasive method that has been shown to reflect activity of the sympathetic and parasympathetic branches of the autonomic nervous system (ANS) (1). Previous pilot studies have suggested that some HRV features show systematic, drug-specific responses to anesthetic drugs (6, 7). Recently it was shown that sedation reduces heart rate and respiratory variability in patients without severe organ failure (8).

The primary aim of most of the ECG-based studies cited above has been to characterize the effects of drugs or disease on HRV. The problem of inferring the level of sedation from HRV measures has received less attention. One exception is a pilot study by Janz et al., in which an ECG-derived metric was shown to provide some information regarding a patient's level of arousal (9). In particular, the authors found that an increased frequency of non-physiological artifacts in the ECG predicted that the patient's was awake and agitated.

Herein we present progress toward developing an automated system to classify levels of sedation from HRV features derived from the ECG in mechanically ventilated ICU patients. In contrast to most prior studies in this area, we focus not on predicting the effects of sedatives on HRV, but rather on the inverse problem of inferring from HRV features the patient's level of sedation.

Materials and Methods

Dataset

All data collection for this work was performed under a protocol approved by the local IRB. Continuous ECG telemetry recordings were archived from 40 patients (25 males; 15 females) admitted to several ICUs at Massachusetts General Hospital (MGH), Boston, USA. BedMaster (Excel Medical Electronics, Jupiter FL, USA) software was used to capture ECG data from GE bedside patient monitors. The sampling frequency of the ECG recordings was 240 Hz. Patient demographic characteristics are presented in Table 1.

Table 1.

Summary of patient demographics. Values are presented as minimum, maximum, mean ± standard deviation.

Variable	min	max	mean ± SD
Age	25	86	56.3 ± 16.8
Weight (kg)	87	107	97.4 ± 14.3
No. of days in ICU	3	34	13 ± 7.5
No. of drugs	2	18	7.6 ± 3.3

Sedation measurement

Sedation levels were scored using the Richmond Agitation-Sedation Scale (RASS), shown in Table 2 (2). Negative numbers on the scale denote various levels of sedation, ranging from -5 = unarousable to -1 = drowsy; 0 denotes a state of calm alertness; and positive numbers denote increasing levels of agitation, from +1 = restless to +4 = combative. RASS assessments were performed approximately once every two hours by ICU nurses as part of routine care and once daily by trained clinical research staff as part of the research protocol, respectively.

Table 2.

The Richmond Agitation-Sedation Scale (RASS) [7] and their corresponding groupings used in this study.

Score	Term	Description	Group
+4	Combative	Overly combative, violent, immediate danger to staff	A
+3	Very agitated	Pulls or removes tube(s), catheter(s); aggressive
+2	Agitated	Frequent non-purposeful movement, fights ventilator
+1	Restless	Anxious but movements not aggressive vigorous
0	Alert and calm		B
-1	Drowsy	Not fully alert with sustained a wakening to voice	C
-2	Light sedation	Briefly awakens with eye contact to voice
-3	Moderate sedation	Movement or eye opening to voice (but no eye contact)
-4	Deep sedation	No response to voice, responses to physical stimulation	D
-5	Unarousable	No response to voice orphysical stimulation

For this work we grouped the 10 possible RASS scores into four categories, as shown in Table 2. These pre-specified RASS groupings were selected a priori on clinical grounds as the smallest set of sedation states that seem important to discriminate between within the RASS scale.

System architecture

The architecture of the proposed automatic sedation classification system is shown in Figure 1. After obtaining RR intervals from the raw ECG signal and subsequent pre-processing, several features (see below) were extracted from the RR interval time series. For classification, we used the linear support machine (SVM) algorithm implemented in the freely available LIBSVM software (see below) with various HRV features as inputs. The output of the classifier is the predicted state of sedation.

Figure 1.

Architecture of the proposed automatic sedation classification system.

Pre-processing and Feature Extraction

Each ECG file was divided into 5-min epochs (with 50% overlap) according to international guidelines regarding HRV feature extraction (10). We limited analysis to epochs centered at the time of RASS assessments, which were typically performed once every two hours. This resulted in a total of 3713 epochs (A = 490, B = 341, C = 2085 and D = 797).

The Pan-Tompkins algorithm was used to identify RR intervals (RRI) (11). Artifacts in the RR interval data were removed using a thresholding method (12). Due to limited prior knowledge about the best set of HRV features for classification, we calculated 14 different candidate time, frequency, nonlinear and entropy HRV features from the artifact-corrected RR intervals, and selected a maximally informative subset using an automated feature selection method (see below). The 14 features are summarized in Table 3. These HRV features were selected based on previous studies in adults (13, 14). The Lomb-Scargle periodogram, which is able to accommodate non-uniformly sampled data, was used to estimate frequency-domain HRV features (15). All features were normalized using the box-cox transformation (16) to have uniform mean and standard deviation before training and testing the SVM classifier model.

Table 3.

HRV features used in this work for the classification of sedation depth.

Domain	Feature	Description
Time	MHR(bpm)	Mean heart rate (number of beats per minute)
	SDNN(ms)	Standard deviation of the NN interval
	RMSSD(ms)	Root mean of the squares of successive differences between adjacent NN intervals
	SDHR(bpm)	Standard deviation of heart rate
Frequency	PVLF(ms)	Power in very low frequency spectrum (0.003-0.04 Hz)
	PLF(ms)	Power in low frequency spectrum (0.04-0.15Hz)
	PHF(ms)	Power in high frequency spectrum (0.15-0.4 Hz)
	LFHF	PLF/PHF
	PNLF(%)	PLF/PTOT × 100, PTOT is the total power spectrum
	PNHF(%)	PHF/PTOT × 100
Nonlinear	SD1	Poincaré plot
	SD2
Complexity	KC	Kolmogorov complexity
	SE	Sample entropy

An example of an RR interval time series for one patient is shown in Figure 2. Also shown are the corresponding spectrogram and a few selected HRV features, including mean heart rate (MHR) and high frequency spectral power (PHF). Systematic variations in the RRI with different levels of sedation can be easily appreciated. Due to the effect of mechanical ventilation, regular peaks due to residual ECG artifact are visible. When the patient is awake, the RRI is characterized by increased complexity and variability.

Figure 2.

Example of an RRI signal (a) and its corresponding spectrogram (b), mean heart rate (MHR) (c), power in the high frequency component (PHF) (d), and RASS scores (e) for one of the patients included in this study. MHR and PHF measures are seen to decrease with increasing levels of sedation.

Classification using a multiclass support vector machine

A linear support vector machine (SVM) algorithm was used to construct four binary classifiers, which were combined to form a four-state classifier for the level of sedation. In general, an SVM maps data in the input feature space, which may not be linearly separable, into a multidimensional feature space and attempts to distinguish between classes using hyperplanes. We provide a brief description of the SVM algorithm in the supplementary material. For a more detailed explanation, readers are referred to (17, 18).

Several methods for extending SVMs to multiclass problems have been used in the literature. In developing this classification system, we utilized a one-against-one approach (19). In this approach, for a k class classification problem, a total of k(k-1)/2 binary-class SVMs are required. Since in this study there are 4 classes, 6 different binary-class linear SVM classifiers are obtained to classify between each pair of RASS groupings. The LIBSVM toolbox was used in this study for training and testing SVMs (20).

Feature Selection

For automatic feature selection we implemented the forward feature selection method (21). In this method, the single best feature is initially selected and then additional features are added incrementally. With this procedure, cross validation accuracy initially increases with each additional feature added up to a point, then begins to decrease, when adding additional features begins to cause model overfitting. The optimal number of features is selected as that number at which the cross validation classification performance peaks.

Performance Assessment

Leave-one-out cross validation (LOOCV) was used to test the performance of the proposed algorithm. In this method, features from all epochs except one were used for training and the left-out epoch was used for testing. A 10-fold cross-validation on the training data was performed to find the optimal regularization parameter C for the linear SVM.

Unequal numbers of epochs were available in our dataset for each state of sedation (see supplemental material, figure S1). If not dealt with carefully, this problem of “data imbalance” can strongly bias the SVM training algorithm in favor of accurately classifying over-represented classes. One common approach to minimize this effect is to discard data in the over-represented class to create a balanced set. Another approach is to assign more weight in the objective function being optimized to samples in the less prevalent classes. We took this latter approach, applying more weight to the less prevalent class in each training iteration for tuning the parameter C (22). This was performed by using the ‘-w’ option in the LIBSVM toolbox (20).

The optimal value for the C parameter was then used to train the final SVM model on all of the training data. The obtained classifier was then applied to the left-out testing epoch and the classification decision was obtained. This approach resulted in a total of 3713 iterations until each epoch was used once for testing. The procedure for evaluating performance of the proposed system using LOOCV is illustrated in Figure 3. All analyses were performed in MATLAB (version 2015a, The MathWorks Inc., Natick, MA).

Figure 3.

Performance assessment of the proposed automatic sedation assessment system.

We compared the overall classification accuracy (percent correct classifications) to a theoretical maximum accuracy attainable by chance. The accuracy attainable by chance was defined to be the average percentage of correct classifications obtained by guessing each of the four possible classes at random with a probability equal to the class prevalence. That is, to estimate chance-level accuracy, we calculated average accuracy achievable by guessing sedation levels A, B, C, or D with probabilities [p₁, p₂, p₃, p₄]=[490, 341, 2085, 797]/3713 = [0.1320,0.0918, 0.5615, 0.2147], respectively. This calculation yields an estimated chance level accuracy of $p_1^2+p_2^2+p_3^2+p_4^2=0.39$, or 39%.

Results

The results of the proposed HRV-based automatic sedation classification system employing all 10 HRV features are shown in Table 4. The forward selection feature selection procedure performed within each iteration of LOOCV resulted on average in a reduction from 15 to 8 features being included in the final classifier model. The most commonly selected features were SDNN, MHR, PHF, SD1, SD2, KC, which were in fact always selected in every iteration of LOOCV. The overall estimated accuracy of the multistate classifier was 69%, substantially better than chance performance, 39%.

Table 4.

Confusion matrix of the proposed sedation system and the actual sedation score.

Actual Sedation group	System output
	A	B	C	D
A	228	24	38	12
B	11	96	37	11
C	233	210	1862	384
D	18	11	149	388
Accuracy (%)	46.5	28.1	89.2	48.8

It can be seen that the proposed system identifies RASS group C (“light sedation”) efficiently (>85%). Other levels of sedation are classified correctly less often, with accuracy for RASS group B (“alert and calm”) being the lowest. As seen in the confusion matrix (Table 4), when misclassifications occur, the majority represent assignments to neighboring levels of sedation. The distribution of the decisions made by the automated sedation system for individual RASS groups is shown in Figure 4. It can be seen that group C had a major influence on groups A, B and D. However, it reduced influence on groups A and D when compared to group B.

Figure 4.

The distribution of the epochs classified by the proposed automatic sedation system for (a) sedation level A, (b) sedation level B, (c) sedation level C, and (d) sedation level D.

The geometric mean (95% confidence interval) of individual feature across RASS groups and accuracy of the proposed system using each HRV feature in isolation during different sedation levels are given in Table 5. Corresponding box plots are shown in Figure 5. In general, heart rate decreased (mean RRI increased) and most HRV features monotonically decreased with increasing levels of sedation. Almost all HRV features (SDNN, RMSDD, MHR, SDHR, PLF, PHF, LFHF, PNHF, SD1, SD2, KC) provided good discrimination with an accuracy > 60%. Only three features (PVLF, PNLF and SE) were less discriminatory when compared to other features (accuracy < 60%).

Table 5.

The mean values and classification accuracy of individual HRV features between different RASS groups.

Features	Geometric mean (95% CI)				Accuracy (%)
	A	B	C	D
SDNN(ms)	18.57 (14.10 - 23.05)	14.57 (12.87 - 16.27)	28.82 (24.13 - 33.52)	9.29 (7.36 - 11.22)	62.7
RMSDD(ms)	14.06 (9.01 - 19.11)	13.54 (11.71 - 15.36)	27.06 (21.72 - 32.40)	9.37 (6.66 - 12.08)	61.2
MHR(bpm)	93.66 (90.24 - 97.09)	89.92 (88.76 - 91.07)	88.57 (86.20 - 90.94)	84.41 (82.69 - 86.13)	63.9
SDHR(bpm)	2.58 (1.91 - 3.26)	1.96 (1.75 - 2.18)	3.72 (3.09 - 4.35)	1.13 (0.90 - 1.36)	62.2
PVLF(ms2)	102.58 (81.70 - 123.47)	83.02 (76.67 - 89.37)	85.38 (72.11 - 98.64)	71.43 (59.85 - 83.02)	45.3
PLF(ms2)	108.77 (92.17 - 125.37)	90.92 (85.85 - 95.99)	100.48 (90.86 - 110.10)	84.18 (75.79 - 92.57)	59.4
PHF(ms2)	80.52 (60.51 - 100.53)	96.86 (90.27 - 103.46)	133.36 (117.99 - 148.73)	97.77 (87.06 - 108.48)	63.6
LFHF(%)	135.09 (52.66 - 217.51)	93.87 (73.73 - 114.00)	75.34 (18.13 - 132.56)	86.10 (68.47 - 103.74)	62.7
PNLF(%)	29.40 (27.33 - 31.47)	27.20 (26.44 - 27.97)	25.22 (23.76 - 26.69)	26.32 (24.95 - 27.70)	43.8
PNHF(%)	21.77 (17.73 - 25.80)	28.98 (27.53 - 30.43)	33.48 (30.20 - 36.76)	30.57 (27.76 - 33.38)	62.4
SD1	19.73 (15.79 – 23.65)	14.68 (12.63 – 16.72)	23.41 (20.51– 25.32)	19.28 (14.77 – 23.67)	61.4
SD2	24.25 (19.63 – 28.78)	22.09 (19.21 – 24.92)	29.93 (25.57 – 34.31)	30.33 (25.69 – 34.96)	62.2
KC	3.27 (3.09 – 3.45)	3.28 (3.19 – 3.38)	3.35 (3.22 – 3.49)	3.66 (3.52 – 3.81)	61.8
SE	8.41 (8.16 – 8.64)	8.36 (8.26 – 8.46)	8.27 (8.07 – 8.46)	8.07 (7.08 – 8.24)	58.5

Figure 5.

Boxplot of HRV features corresponding to RASS groups used in study.

Discussion

This study explores the potential value of HRV features for automated monitoring of sedation levels in mechanically ventilated ICU patients. Our results suggest that, at the population level, multiple different measures of HRV vary systematically with sedation levels. In particular, SDNN, RMSDD, MHR, PHF, thought to reflect the level of parasympathetic nervous system activity (22), generally decrease with deepening sedation. Among all HRV features studied in this work, the mean heart rate (MHR) and high frequency spectral power (PHF) provided the most discriminatory information. These findings are in general agreement with prior studies (23–26) which have also have shown decreases in HF power at deep levels of sedation (group D). Using machine learning techniques we were able to select weighted combinations of HRV features that achieve moderately good performance in discriminating four different levels of sedation. This performance, measured in term of cross validated classification accuracy, was 30% better than the theoretical floor of chance-level performance (69% vs 39%). Overall, these results argue that the ECG may indeed carry useful information about levels of sedation in ICU patients.

In contrast to the physiologically based sedation assessment strategy explored in this work, sedation monitoring in current practice relies entirely on behavioral assessments, exemplified by the RASS scale. Adherence to strict sedation monitoring protocols using RASS and other scales has been shown to reduce over- and under-sedation (27). However, RASS assessments can only be performed intermittently, and recent studies have shown that compliance with scheduled assessments is often poor (28, 29). Consequently, a reliable system for automated physiologically-based sedation monitoring, if developed, may enhance the quality and safety of ICU patient care. Several studies have explored the promise of EEG-derived indices for tracking the depth of general anesthesia (30–32), though ICU monitoring under conditions of lighter sedation and more severe medical illness has received much less attention. The present work adds to existing literature by exploring the potential value of HRV measures for monitoring sedation.

Though promising, our present results are best viewed as preliminary, with important limitations and room for further system improvement. First, we omitted from the classifier design any explicit information about which drugs or drug doses were used to achieve sedation. This is an important limitation, because HRV measures are known to have drug-specific characteristics (7, 33). Second, our classifier design did not account for specific disease states that might have affected autonomic function. Several studies have shown that conditions such as sepsis, anoxic brain injury, and multiple organ dysfunction syndrome strongly influence autonomic function (34–38). Similarly, previous work has shown that the modulation of HRV by depth of sedation is reduced in the setting of severe organ system dysfunction (8). It is thus likely that the nature and severity of underlying medical illness will need to be accounted for to optimize HRV-based sedation monitoring accuracy. Third, we did not consider the influence of circadian rhythms. Sympathetic nervous system activity decreases and parasympathetic nervous activity increases during sleeping hours (39). Because data from patients in our cohort were collected at different times of the day, circadian rhythms may have had an effect on the performance of our system. Fourth, we did not account for the fact that HRV is affected by respiratory status, whereas the physiological response to PaCo2 is known to be reduced in deep sedation (33, 34, 35). Fifth, the group-level differences we found between patient's HRV features do not necessarily imply that it will be possible to accurately track sedation levels longitudinally over time in individual ICU patients.

Another more technical limitation is our grouping of sedation scores into 4 levels, which is less granular than the 10 levels of the RASS scale. The primary reason for this choice was that our dataset was not large enough to support training a 10-class classifier (Figure S1-supplementary material). Nevertheless, it is desirable and more physiologically realistic for future work to aim for a continuous rather than a categorical measure of sedation depth. An additional minor justification for lumping scores is that, in our experience, the precision with which nurses assign patients into nearby RASS categories is limited, thus lumping together similar states of sedation that are prone to be confused can be expected to reduce the problem of category label noise.

A final technical limitation concerns the method used for model fitting. In the present study, model training and cross validation were done after combining epochs from all patients into a single data set. This method may introduce bias due to statistical correlations between epochs taken from the same patient. The breakdown of epochs by RASS category for patients in our study is shown in Figure S1 (supplemental material). As can be seen, a few patients do contribute more observations to the data set than others. Nevertheless, no single patient or small group contributes the majority of epochs. Thus, while the effect is unlikely to be severe, it is possible that the unequal contributions from different patients introduce some bias into the cross validated performance estimates. Future studies with a larger cohort could avoid these potential biases by replacing the leave-one-epoch-out approach by the leave-one-subject-out method of cross validation in the feature selection and performance evaluation steps of classifier development.

Conclusion

Further research can likely improve on the present work by accounting in the model for HRV-modulatory effects of specific drugs and disease states and the degree of organ dysfunction. A robust solution may also ultimately require joining HRV-derived features with information from other signals such as EEG. It is also possible that accuracy in monitoring individual patients ultimately requires calibration inputs in the form of intermittent manual assessments provided by health care providers. These improvements will require assembling a larger patient cohort dataset to support model development, work currently underway. Nevertheless, the results of this pilot study suggest that HRV play a valuable role in an eventual clinically useful physiologically based sedation monitoring system.