Comparison of Electrophysiologic Monitors With Clinical Assessment of Level of Sedation

Abstract

OBJECTIVE: To assess the correlation between 2 clinical sedation scales and 2 electroencephalographic (EEG)–based monitors used during surgical procedures that required mild to moderate sedation.

PATIENTS AND METHODS: Patients scheduled for elective surgery participated in this institutional review board–approved study from March 2003 to February 2004. Level of sedation was determined both clinically using the Ramsay and the Observer's Assessment of Alertness/Sedation scales and with 2 EEG measures (the Bispectral Index version XP [BIS XP] or the Patient State Analyzer [PSA 4000]). Correlation between these 2 measures of sedation were tested using nonparametric statistical tests.

RESULTS: The BIS XP monitor was used in 26 patients, and the PSA 4000 monitor was used in 24 patients. The Ramsay and Observer's Assessment of Alertness/Sedation scores correlated with each other (r=−0.96; P<.001) and with both the BIS XP (r=−0.89 and r=0.91, respectively; P<.001) and the PSA 4000 (r=−0.80 and r=0.80, respectively; P<.001) values. However, this correlation was strongest only at the extremes. Between the BIS XP and PSA 4000 values of 61 and 80, the clinical sedation scores varied greatly.

CONCLUSION: On the basis of our results, these EEG-based monitors cannot reliably distinguish between light and deep sedation.

Conscious sedation of patients for clinical procedures outside the operating room is frequently performed by nonanesthesiologists. As stated in the American Society of Anesthesiology's Practice Guidelines for Sedation and Analgesia by Non-Anesthesiologists, sedation or analgesia provides 2 types of benefit for patients.¹ First, sedation or analgesia allows patients to tolerate unpleasant procedures by relieving anxiety, discomfort, or pain. Second, sedation or analgesia may expedite the conduct of procedures that are not particularly uncomfortable but that require that the patient not move. This is true especially in children and uncooperative adults.

Sedation scales have been developed to standardize this assessment. All sedation scales incorporate a stimulus followed by a response to determine a patient's level of consciousness. Two such commonly used scales are the Ramsay scale^2,3 and the Observer's Assessment of Alertness/Sedation (OAAS).⁴ The Ramsay scale has been validated in the intensive care unit setting.^2,3,5 However, in each instance, using these scales requires disturbing the patient to elicit a response. In some cases, one has to inflict pain to determine a complete lack of responsiveness. Continual reassessment is also necessary, requiring that a patient be stimulated repeatedly. Although formal assessment scores can be readily performed by nurses trained to evaluate level of consciousness, in most intensive care units they are not.⁶

In fact, monitors based on the electroencephalogram (EEG) have been developed that may be useful in providing a simple measure of sedation without the need to stimulate the patient. A reliable EEG-based monitor that could determine patients' level of sedation would impart additional safety to the delivery of sedative hypnotics. The goal of this study was to assess the correlation between 2 clinical sedation scales and 2 EEG-based monitors of sedation used during surgical procedures that required mild to moderate sedation.

We use the term sedation instead of consciousness or hypnosis, although we acknowledge that to some degree these terms are interchangeable. However, the American Society of Anesthesiology has clearly defined criteria for levels of sedation that along with the 2 sedation scales make this a preferable term.¹

PATIENTS AND METHODS

Patients who were scheduled for elective surgery between March 2003 and February 2004 with moderate intravenous sedation by itself (monitored anesthesia care) or with a regional anesthetic gave informed written consent for this institutional review board–approved study. The day before surgery, 2 operating rooms were selected because surgeons participating in this study were using these rooms to operate on patients who would be under conscious sedation. All patients assigned to undergo surgery in those 2 rooms were invited to participate in this study. Both rooms had patients scheduled for similar surgical procedures; thus, they were anticipated to require a similar level of sedation. An anesthesia technician placed a Bispectral Index version XP (BIS XP) monitor (Aspect Medical, Newton, Mass) and a Patient State Analyzer (PSA 4000) monitor (Physiometrix Inc, North Billerica, Mass) in each room.

The patients were monitored with either the BIS XP or the PSA 4000 depending on the operating room they were assigned to. The 2 groups were defined based on whether a BIS XP monitor or a PSA 4000 monitor was used to assess the level of sedation. No obvious bias existed on which monitor was placed in the specific operating room or on the type of surgical cases assessed with each monitor.

Routine monitors were placed on patients when they arrived in the operating room. In addition, either a PSA 4000 array or a BIS XP sensor (Quatro) was placed on the patient's forehead according to the manufacturer's instructions by the same physician (C.J.C.). Once the monitor completed its self-test, including the calculation of impedance, a number corresponding to level of sedation was displayed. To minimize observer's bias, a towel was placed over the monitor screen to blind the physician evaluating the clinical sedation score to the EEG-based scores. All BIS XP and PSA 4000 values were recorded before the clinical assessment of level of sedation. With the BIS XP monitor, a study participant other than the individual determining the sedation scores recorded the BIS XP values before the clinical sedation scores were determined. With the PSA 4000 monitor, an event marker was activated before the clinical assessment. These Patient State Index values were recalled after the case had ended.

All monitor values were recorded continually, either every 5 seconds with the BIS XP monitor, through the serial port of a laptop running HyperTerminal (Hilgraeve, Monroe, Mich), or every 2.5 seconds with the PSA 4000 monitor, onto an internal IBM microdrive (320 MB). The Signal Quality Index (SQI) values were obtained from the BIS XP monitor every 5 seconds and recorded automatically. We averaged these values for the total recording period. The average SQI value was 83.0, well above the cutoff of 50 below which the BIS values are considered unreliable because of poor signal quality. The PSA monitor generates numbers every 2.5 seconds to evaluate the quality of the electrophysiologic signal. These values represent the percentage of artifact. We averaged these values for the total recording period. The average artifact values were 19.8%, well below the cutoff of 50% above which PSA values are considered unreliable because of poor signal quality. Even though both the BIS XP and the PSA 4000 determine the quality of their recordings differently, the average SQI and average artifact values confirm that our BIS XP and PSA 4000 values were not significantly contaminated by poor-quality electrophysiologic signals.

The first scores of sedation were displayed a minute after the EEG-based monitors were applied. A BIS XP or PSA 4000 number was recorded automatically followed by a written clinical assessment of the level of sedation, first a Ramsay scale score then an OAAS score (this order was followed throughout). The 2 scales are outlined in Tables 1 and 2.

TABLE 1.

Ramsay Scale

Score	Assessment
1	Patient is anxious and agitated or restless or both
2	Patient is cooperative, oriented, and tranquil
3	Patient responds to commands only
4	Patient exhibits brisk response to light glabellar tap or loud auditory stimulus
5	Patient exhibits a sluggish response to light glabellar tap or loud auditory stimulus
6	Patient exhibits no response

TABLE 2.

The Observer's Assessment of Alertness/Sedation Scale^*

Category†	Observation	Score
Responsiveness	Responds readily to name spoken in normal tone	5
	Lethargic response to name spoken in normal tone	4
	Responds only after name is called loudly and/or repeatedly	3
	Responds only after mild prodding or shaking	2
	Does not respond to mild prodding or shaking	1
Speech	Normal	5
	Mild slowing or thickening	4
	Slurring or prominent slowing	3
	Few recognizable words	2
Facial expression	Normal	5
	Mild relaxation	4
	Marked relaxation (slack jaw)	3
Eyes	Clear, no ptosis	5
	Glazed or mild ptosis (less than half of the eye)	4
	Glazed and marked ptosis (half of the eye or more)	3

^*From Chernik et al⁴ and Liu et al.⁷

^†Responsiveness indicates responsiveness to calling the patient's name and/or using physical stimuli; speech, while asking the patient to repeat a standard sentence such as “The quick brown fox jumps over the lazy dog.”; facial expression, degree of facial relaxation; and eyes, ability of subject to focus eyes ptosis.

All clinical evaluations were performed by 5 first- and second-year resident physicians in addition to the the principal investigator (C.J.C.). The principal investigator instructed the resident physicians in performing the clinical evaluations with both the Ramsay and OAAS scales and observed their initial assessments. The reliability of these assessments was evaluated by comparing the correlation of the EEG-based values with the clinical assessments for the principal investigator compared with the first- and second-year anesthesiology residents. No differences in the correlations were found.

The clinical assessment using both scales was obtained at 15-minute intervals. When a clinical sedation score could not be obtained at a particular 15-minute interval, the BIS XP or PSA 4000 score was also omitted.

In all cases, sedation was provided with intravenous propofol, fentanyl, and midazolam. In 87% of the cases, besides bolus drug administration of fentanyl and midazolam, a propofol infusion was used to provide a continuous level of sedation. Anesthetic medications were administered solely at the discretion of the anesthesia team caring for the patient without any limitations.

For statistical analysis, all clinical values of level of sedation and BIS XP and PSA 4000 values were entered into a single Excel table. Partial Mann-Kendall statistical tests were used to characterize the relationship between the time trends in EEG-based monitors of consciousness and the corresponding time trends in clinical sedation scales.^8,9 This nonparametric technique uses the sign of the changes in the measured variables over time and does not require assumptions of normality or independence of the data that comprise the time series. This is in contrast to the requirement for independence among the observed values that underlies the use of the Spearman correlation or analyses using PKMACRO or PKDMACRO.¹⁰ The correlation coefficient obtained from the conditional covariance of the measured time series was subjected to Fisher z transformation, and group comparisons of BIS XP and PSA 4000 were performed using unpaired t tests or analysis of variance. Partial Mann-Kendall statistics and associated covariances and correlations were computed using an Excel macro written by C. Libiseller (www.mai.liu.se/∼cllib/welcome/PMKtest.html). P<.05 was considered statistically significant for all analyses. This study had approximately 80% power to find a significant difference between correlation coefficients of magnitude 0.8 and 0.9 (α=.05).

RESULTS

A sample size of 50 patients was chosen to generate at least 300 pairs of BIS XP scores and PSA 4000 scores. The BIS XP monitor was used in 26 patients, and the PSA 4000 monitor was used in 24 patients. A total of 162 BIS XP scores and 189 PSA 4000 scores were collected, corresponding to 351 Ramsay scale and OAAS scores. For 8 time points of 351 (2.3%), a clinical sedation score could not be obtained at a particular interval, so the BIS XP or PSA 4000 score was also omitted. The 2 groups were similar in their perioperative values: age, sex ratios, height, weight, intraoperative medications, and length of surgery (Table 3). The 2 clinical sedation scales, the Ramsay scale and the OAAS, correlated excellently with each other (r= −0.96; P<.001). In addition, both the BIS XP and the PSA 4000 correlated well with the Ramsay scale (r=−0.89 and r=−0.80, respectively) and the OAAS (r=0.91 and r=0.80, respectively) scores (P<.001 for all). Only the Ramsay scale scores were used in tables and figures because the 2 sedation scales were essentially identical. However, several differences exist in the degree of correlation between the EEG-based monitors, depending on the sedation scale used. Thus, the BIS XP showed a better correlation than the PSA 4000 with the OAAS (P=.02) and the Ramsay scale (P=.001).

TABLE 3.

Demographic Data^*

Demographic	BIS XP and PSA 4000 (N=50)	BIS XP (n=26)	PSA 4000 (n=24)	P value
M/F (%)	44/56	42/58	46/54	>.99
Age (y)	56.5 (15.6)	54.3 (16.7)	58.9 (14.2)	.30
Height (cm)	167.6 (9.6)	165.9 (9.1)	169.1 (10.0)	.26
Weight (kg)	75.8 (14.6)	75.2 (13.8)	76.3 (15.6)	.80
ASA class, No. (%)				.16
1	9 (18)	6 (23)	3 (13)
2	33 (66)	14 (54)	19 (79)
3	8 (16)	6 (23)	2 (8)
Service				.78
General surgery	22 (44)	12 (46)	10 (42)
Orthopedic surgery	28 (56)	14 (54)	14 (58)
Duration of surgery (min)	99.8 (51.1)	92.1 (48.8)	108.1 (53.3)	.27
Anesthesia type				.79
MAC	18 (36)	9 (35)	9 (38)
PNB	19 (38)	11 (42)	8 (33)
Neuroaxial	13 (26)	6 (23)	7 (29)
Fentanyl (μg/kg)	1.4 (0.7)	1.2 (0.6)	0.7 (0.4)	.32
Midazolam (mg/kg)	0.04 (0.02)	0.04 (0.01)	0.02 (0.01)	.64
Propofol (mg/kg)	4.0 (3.3)	4.2 (4.0)	3.7 (2.7)	.62

^*Values are mean (SD) unless indicated otherwise. ASA = American Society of Anesthesiology; BIS XP = Bispectral Index version XP; MAC = monitored anesthesia care; PNB = peripheral nerve block; PSA 4000 = Patient State Analyzer.

The correlation was strongest at the extremes, as is shown for the BIS XP values (Figure 1) and the PSA 4000 values (Figure 2) compared with Ramsay scale score. When the patient was unresponsive (Ramsay scale score, 6), only 56.4% of the time were the BIS XP values and 70.2% of the time were the PSA 4000 values in the range of 21 to 60 (deep sedation). Similarly, when the patient was alert (Ramsay scale score, ≤2), only 85% of the time were the BIS XP values and 65.5% of the time were the PSA 4000 values in the range of 81 to 100 (light sedation). However, when the BIS XP or PSA 4000 scores ranged from 61 to 80, the clinical sedation scores were scattered over the entire range of possible scores (Figures 1 and 2). Under these circumstances, neither the BIS XP nor the PSA 4000 scores correlated significantly with the Ramsay scale (r=−0.24 and r=−0.04, respectively) or the OAAS (r=−0.07 and r=−0.01, respectively) scores.

FIGURE 1.

Bispectral Index version XP (BIS XP) vs Ramsay scale scores. The Ramsay scale scores are plotted on the x-axis against BIS XP values on the y-axis. The points represent patients; 1 point may represent 2 or more patients if both had the same values. The line represents a best-fit regression line. The BIS XP values between 60 and 80 are shaded on the y-axis across all Ramsay scale scores to demonstrate visually the distribution of patient values above and below this range. Patients are considered alert for Ramsay scale scores of 2 or less and unresponsive for Ramsay scale scores equal to 6.

FIGURE 2.

Patient State Analyzer (PSA 4000) vs Ramsay scale scores. The Ramsay scale scores are plotted on the x-axis against PSA 4000 values on the y-axis. The points represent patients; 1 point may represent 2 or more patients if both had the same values. The line represents a best-fit regression line. The Bispectral Index version XP values between 60 and 80 are shaded on the y-axis across all Ramsay scale scores to demonstrate visually the distribution of patient values above and below this range. Patients are considered alert for Ramsay scale scores of 2 or less and unresponsive for Ramsay scale scores equal to 6.

Several differences existed in the degree of correlation between the EEG-based monitors depending on the age of the patient and the type and duration of surgery. Patients older than 55 years showed a significantly weaker correlation between the PSA 4000 and the OAAS (r=0.67) than those 55 years or younger (r=0.86; P=.009). The correlation between the BIS XP and OAAS for patients receiving neuroaxial blocks (r=0.61) was weaker than for patients receiving local anesthesia (r=0.96) or peripheral nerve blocks (r=0.94; P<.001).

Although BIS XP values vs the clinical evaluations based on the Ramsay scale and OAAS scores were no different for first- vs second-year residents, the PSA 4000 values were less strongly correlated with Ramsay scale scores for less experienced anesthesiologists (first-year residents) (r=−0.53) than for more experienced anesthesiologists (second-year residents) (r=−0.79; P=.006). This finding may account for why PSA 4000 values were less strongly correlated with Ramsay scale scores in patients undergoing general surgical procedures (r=−0.54) than in patients undergoing orthopedic procedures (r=−0.84; P<.001), because 92% of the orthopedic procedures were handled by more experienced anesthesiology residents, whereas 80% of the general surgical procedures were handled by less experienced anesthesiology residents.

DISCUSSION

The EEG-based depth of consciousness monitors were first introduced into clinical practice in 1996 with the BIS monitor.^11-14 Since that time several other monitors have been developed that incorporate EEG data received from scalp electrodes.^15,16 We chose to use the PSA 4000 monitor in addition to the BIS XP to provide a more general conclusion regarding the effectiveness of these EEG-based monitors to determine level of sedation. Indeed, the BIS XP and PSA 4000 have been shown to be comparable, with some slight advantages attributable to each.^17,18 These monitors process EEG information and derive a number, which correlates with level of sedation.¹⁹ Since its introduction, more than 400 articles have evaluated the efficacy of the BIS XP monitor. A complete review of this literature would be impractical and is not the focus of this discussion. Most of these articles support the use of these EEG monitors in the practice of anesthesia related to measures of sedation, hypnosis, or consciousness.

Increasingly more nonsurgical procedures, some that may be invasive, are being performed on patients in physicians' offices in an attempt to provide early diagnosis and treatment of illnesses. Sedation and analgesia comprise a continuum of states, ranging from minimal sedation (anxiolysis) through general anesthesia. Patients demand to be made comfortable during these outpatient procedures; therefore, physicians are often pressured to provide more than just minimal sedation. In current practice, anesthesiologists incorporate clinical assessment to titrate anesthetic depth during conscious sedation when procedures are performed with patients under local or regional anesthesia. Several studies^20-23 have looked at the complications of sedation and analgesia even in well-staffed environments. Although rare, these complications include respiratory problems such as dyspnea, desaturation, and laryngospasm, cardiac problems such as hypotension and arrhythmias, and systemic problems such as nausea and vomiting.^20,21,23 These problems must be rapidly recognized and appropriately managed to avoid the risk of hypoxic brain damage, cardiac arrest, or death. In a closed-claims analysis of 900 cases, there were 14 cases of sudden cardiac arrest in healthy young patients receiving spinal anesthesia.²⁴ All were resuscitated, but 6 died because of their neurologic injury. Two patterns were identified, one of which was excessive sedation.²⁴ Because no absolute correlation exists between administered doses of sedatives or analgesics and the level of responsiveness, monitors that could objectively assess the level of sedation would be extremely useful.

Unfortunately, most of these aforementioned publications relate to studies in tracheally intubated patients under general anesthesia in whom an independent measure of level of sedation is impossible. Few studies^25,26 have involved monitoring sedated patients in whom clinical assessment of level of sedation can be measured. For example, Bower et al²⁵ found EEG-based monitors to be helpful and suggested that a BIS XP number of 82 correlated with sedation sufficient for endoscopy. In general, studies^25,26 with local anesthesia were limited to less complicated, short procedures, usually producing little pain.

All the literature reviewed for this article in which the OAAS or Ramsay scale was used showed a good correlation between the EEG-based monitors and the respective scale.^27-33 Struys et al³³ reported that the ability of the BIS XP to detect OAAS and loss of response to the eyelash reflex remained accurate in 45 gynecologic patients receiving remifentanil and propofol infusions using a continuous infusion device to a target effect–site concentration. However, our results differed from theirs possibly because they did not look at the correlation of the BIS XP values, specifically at intermediate BIS XP values, in which we found those values to be inaccurate predictors of sedation scores.

Bruhn et al²⁹ performed a similar study in 20 healthy volunteers. They found a high prediction probability value between the BIS XP values and hypnotic end points. However, a poorer correlation was seen when airway manipulation was attempted on these patients. Airway manipulation was used to provide a stimulus to patients similar to what might occur in “daily clinical practice, where ‘anesthetic depth’ is a fluent continuum….” Had they actually performed a surgical procedure, their results may have been similar to our findings.

Numerous studies have found a strong correlation between these EEG-based monitors and their respective sedation scales; however, none of the studies specifically addressed the intermediate range of sedation values, which were our area of clinical interest.^27-33 In our study, we found an excellent correlation between the sedation scale scores and the BIS XP and PSA 4000 values, when the data were pooled for all values. However, there was a poor correlation within the intermediate range of scores (61-80). The literature cited did not specifically look at this range of values.

We studied patients receiving commonly used medications for sedation over a wide range of ages, during a variety of surgical procedures (general surgery to orthopedic surgery), and of varying duration. Our patient population and types of procedures were consistent with a common anesthesia practice. We also used both the Ramsay and the OAAS scales, which allowed us to confirm the accuracy of our sedation score. Finally, we incorporated a newer sedation monitor, the PSA 4000 monitor, and demonstrated significantly strong correlations for BIS XP and both the Ramsay and OAAS scales.

Both the BIS XP and the PSA 4000 produce a single number, which was significantly correlated with our 2 clinical sedation scales. However, within the range of 61 to 80 for both monitors, the distribution of clinical sedation scores varied from unconscious to alert. These monitors may alert a physician when a patient is awake or unconscious, but they have difficulty distinguishing moderate from deep sedation. Of note, the range of 61 to 80 was chosen on the basis of the most recent information from the Aspect Medical Web site (www.aspectmedical.com/professionals/anesthesia/default.mspx): less than 60, general anesthesia; 61 to 70, moderate sedation; and 71 to 90, light/moderate sedation. Defining an association with a specific number is dangerous, especially knowing that previous investigators found that the BIS XP values were not linearly related to levels of consciousness.³⁴ In fact, Kearse et al,³⁴ using computer-controlled infusion to target specific propofol serum levels, found a “very narrow range in BIS separating those who did and did not follow commands suggest[ing] either that the incremental increases in propofol concentrations were too large to produce graded responses or that there was an abrupt loss of conscious processing at a measured BIS threshold of approximately 65-70.” However, our view is that BIS XP values between 60 and 80 are accepted by many physicians who use this monitor with light to moderate sedation. By extension, we used the same values for the PSA 4000, even though this monitor has not been investigated as extensively.

We used partial Mann-Kendall statistical tests because they allow for determination of correlation among sets of repeated observations within individuals that are not independent of each other.^8,9 Therefore, they avoid the assumptions inherent in the use of Spearman correlation, which assumes independence among observations. Similar assumptions are required for the statistical model using the PKMACRO and PKDMACRO from Smith et al.¹⁰

We consider this study particularly appropriate because these monitors are being used by nonanesthesia personnel to titrate conscious sedation outside the operating room.^25,26 Since it is not common practice to use clinical sedation scales, the idea that a monitor could be used as an objective measure of the level of consciousness, if valid, would improve the quality of conscious sedation. During a procedure, it is not practical to continually disturb a patient to document and standardize his or her level of sedation. For this reason, many patients drift from a lighter plane of sedation to general anesthesia without physicians even becoming aware of it.

Does it matter whether a patient is lightly or profoundly sedated during local or regional anesthesia? Many would say it is irrelevant as long as the physician is able to perform the procedure and the patient is not harmed. However, recent closed-claims data analysis on monitored anesthetic care procedures suggests otherwise.³⁵ There was no significant difference between percentages of claims in local or regional anesthesia and general anesthesia cases based on death, brain damage, and respiratory events. One interpretation is that we have become so safe in delivering general anesthesia that it becomes similar to local or regional anesthesia in its risks. Another interpretation is that many patients sedated as local or regional anesthesia cases actually receive general anesthesia without protecting the airway. On the basis of our clinical assessment of sedation in this study, 24.5% of the time patients had clinical levels of sedation consistent with general anesthesia. If one could reliably determine a patient's level of sedation, unintended general anesthesia could be avoided.

Unfortunately, with the rising popularity of EEG-based monitors, they are being used frequently to determine level of sedation in patients without substantial research to support this application. It is possible that less experienced physicians and nurses will look to a monitor based on the EEG as a sole measure of level of sedation, ignoring some of the clinical signs of oversedation.

CONCLUSION

Even though these EEG-based monitors of sedation showed a correlation with 2 clinical measures of sedation and responsiveness at the extremes of sedation, we did not find a good correlation in the area of clinical interest, namely, at scores between 61 and 80 when one would like to measure light to moderate sedation. Realizing that the derivation of the scale used by EEG-based monitors is in fact not linear, it may not even be justifiable to assume that these monitors can be expected to follow the progression of sedation. This study concludes that the BIS XP and PSA 4000 do not produce a reliable means for assessing intermediate levels of sedation.

Aspect Medical Company supplied BIS XP monitors and sensors. Ms Linda Kovitch and Ms Stacy Glass from Aspect Medical provided technical support. Physiometric Inc supplied the PSA 4000 monitors and sensors. Ms Elizabeth Oei and Mr Rick Ortega from Physiometric Inc provided technical support. Dr Mieczyslaw Finster provided invaluable assistance with editing suggestions, and Raheleh Hatami provided help with preparing the manuscript for submission.

Glossary

BIS XP

Bispectral Index version XP

EEG

electroencephalogram

OAAS

Observer's Assessment of Alertness/Sedation

PSA 4000

Patient State Analyzer

SQI

Signal Quality Index