Prevention of Intraoperative Awareness with Explicit Recall in an Unselected Surgical Population: A Randomized Comparative Effectiveness Trial

Abstract

Background

Intraoperative awareness with explicit recall occurs in approximately 0.15% of all surgical cases. Efficacy trials based on the Bispectral Index^™ (BIS) monitor and anesthetic concentrations have focused on high-risk patients, but there are no effectiveness data applicable to an unselected surgical population.

Methods

We conducted a randomized controlled trial of unselected surgical patients at three hospitals of a tertiary academic medical center. Surgical cases were randomized to alerting algorithms based on either BIS values or anesthetic concentrations. The primary outcome was the incidence of definite intraoperative awareness; prespecified secondary outcomes included postanesthetic recovery variables.

Results

The study was terminated due to futility. At interim analysis the incidence of definite awareness was 0.12% (11/9376) (95% CI 0.07 to 0.21%) in the anesthetic concentration group and 0.08% (8/9460) (95% CI 0.04 to 0.16%) in the BIS group (p = 0.48). There was no significant difference between the two groups in terms of meeting criteria for recovery room discharge or incidence of nausea and vomiting. By post hoc secondary analysis, the BIS protocol was associated with a 4.7-fold reduction in definite or possible awareness events compared to a cohort receiving no intervention (p = 0.001; 95% CI 1.7 to 13.1).

Conclusion

This negative trial could not detect a difference in the incidence of definite awareness or recovery variables between monitoring protocols based on either BIS values or anesthetic concentration. By post hoc analysis, a protocol based on BIS monitoring reduced the incidence of definite or possible intraoperative awareness compared to routine care.

Introduction

Intraoperative awareness with explicit recall of surgical events is a potentially devastating event associated with posttraumatic stress disorder¹ and has an incidence of approximately 0.15% for all risk levels.^2,3 Processed electroencephalographic monitors have been developed to assess anesthetic depth and potentially prevent intraoperative awareness, which is considered a sentinel event by the Joint Commission.⁴ The Bispectral Index^® (BIS) monitor (Covidien, Boulder, CO) processes a frontal electroencephalographic channel to calculate a dimensionless number from 100 (awake) to 0 (no detectable brain activity) in order to provide a measure of the patient’s level of consciousness; a BIS range of 40 to 60 is suggested to be consistent with the state of general anesthesia.^5–7

Past efficacy trials have evaluated the role of protocols based on the BIS monitor^5–7 and anesthetic concentrations^6,7 for the prevention of intraoperative awareness. However, these studies were performed exclusively in patients at high risk for the complication. A large cohort study did find that BIS monitoring decreased the incidence of intraoperative awareness in a broad surgical population compared with historical controls, but was limited by its observational design, changing practice patterns regarding end-tidal anesthetic concentration monitoring, and exclusion of patients not receiving neuromuscular blockers.⁸ As such, there are currently no comparative effectiveness data to guide the decisions of providers or policy makers as they attempt to prevent intraoperative awareness in the >200 million major surgeries performed worldwide each year.⁹

Similarly, there are no effectiveness data supporting the claim that anesthetic consumption is reduced with the use of a BIS monitor, which has been suggested to decrease inhaled anesthetic use by up to 38%.¹⁰ These data are reinforced by meta-analyses of small efficacy trials of both inhaled and intravenous anesthesia.^11,12 It has recently been argued that decreased anesthetic use and the ensuing clinical benefits such as faster recovery or reduced nausea and vomiting make the BIS monitor cost-effective and that it should therefore be routinely incorporated.¹³

Here we describe a comparative effectiveness study with active comparators and a two-sided superiority design. This randomized controlled trial compared alerting protocols based on either anesthetic concentration or BIS values in an unselected surgical population at three hospitals within a tertiary academic medical center. The primary outcome was the incidence of definite awareness; prespecified secondary outcomes included the incidence of definite or possible awareness, as well as anesthetic usage and recovery variables.

Materials and Methods

A detailed description of the experimental protocol for the Michigan Awareness Control Study (ClinicalTrials.gov number NCT00689091) has been previously reported.¹⁴ The conduct of the study and the reporting of results followed the Consolidated Standards of Reporting Trials guidelines.¹⁵

Participants

The study received approval from the Institutional Review Board of the University of Michigan, Ann Arbor, and was deemed to be of minimal risk. A full discussion of the risks and benefits was conducted with each patient approached. Patient consent to interventions and follow-up was electronically documented in our perioperative information system (Centricity^®, General Electric Healthcare, Waukesha, WI). Patients were recruited from three hospitals of the University of Michigan Health System from May 2008 until May 2010. Inclusion criteria were age >18 yr, general anesthesia using inhalational or intravenous technique for any surgical case that did not involve the forehead, and availability for follow-up interviews. Exclusion criteria were intracranial procedures, adhesive allergy, psychosis, or history of traumatic brain injury. All patients enrolled in the study were blinded to group assignment and had the BIS electrode applied to the left side of the forehead by a member of the research staff prior to entering the operating room.

To detect a reduction in the incidence of intraoperative awareness from 0.15% to 0.04%,⁸ we calculated a need for 14,072 per group or a total n = 28,144 with 80% power and a significance level of 5%. We targeted a total recruitment of 30,000 patients, with a prespecified interim analysis after 20,000 patients were recruited (2/3 target sample).¹⁴ A constant likelihood group sequential method with formal futility boundaries was used with a two-sided O’Brien-Fleming stopping rule. There was no contingency for early termination for efficacy. An acceptance region plot (or a futility region plot) was generated using SAS statistical software (SAS version 9.2, Carey, NC). The two-sided futility boundary (for the differences in proportions between the BIS and the anesthetic concentration group) at the planned interim analysis was from −0.0005434 to 0.0005434. The difference between the proportions observed at the interim analysis was 0.0003275422 (11/9376 cases of definite awareness in the anesthetic concentration group minus 8/9460 cases of definite awareness in the BIS group), which is within the stopping boundary for futility.

Study Design

The University of Michigan Health System utilizes the Centricity^R electronic perioperative information system in all of its operating rooms. Using this system, automated real-time analysis of BIS values or minimum alveolar concentration (MAC) was performed every 5 min, with the transmission of provider-specific electronic alphanumeric paging alerts in less than 60 s. Operating rooms were randomized every 3 months based on even- or odd-numbered operating rooms to either (1) electronic alerts in the event of median BIS values >60, or (2) electronic alerts for median age-adjusted MAC level of <0.5. The threshold of age-adjusted MAC <0.5 was chosen based on a retrospective analysis of electronically documented cases with and without awareness that occurred prior to the onset of the study,¹⁶ as well as the high frequency with which thresholds of higher MAC are crossed.⁶ In addition to the age-adjusted MAC of standard inhaled anesthetics, alerting based on anesthetic concentrations also reflected documented intravenous anesthetic infusions and bolus doses.¹⁶ Paging alerts to the clinician electronically signed into and physically present during the case reported either the median BIS value or anesthetic concentration level for the prior 5 min epoch, followed by “Potentially insufficient anesthesia- please check vaporizers and intravenous lines.”

In the BIS-targeted rooms, BIS values appeared on the main monitoring screen and were automatically recorded. In the anesthetic alert-targeted rooms, BIS values neither appeared on the monitor nor were accessible intraoperatively. Other aspects of anesthetic care (e.g., choice of anesthetic agents, benzodiazepines) were not standardized for this study.

Randomization and Blinding

Randomization was performed using a random-number, computer-generated block scheme based on even or odd operating room number. The blocks were defined within a specific year of the study based on the original start date of recruitment. The study year was divided into 4 quarters by calendar month (3 months per quarter). Within a specific study year, the odd-numbered operating rooms and even-numbered operating rooms were randomized to BIS alerting two times and anesthetic concentration alerting two times. If the odd-numbered operating rooms were randomized to one alerting protocol, the even-numbered operating rooms were randomized to the alternative alerting protocol for that quarter of the study year. Patients, postoperative interviewers, and all case reviewers were blinded to group assignment. Practitioners receiving pages regarding BIS or MAC values were not blinded to group assignment. However, practitioners were not made aware of the randomization scheme or dates for randomization change during the study.

Technical Factors

The BIS monitors used in the Michigan awareness control study were not free-standing devices, but modules that interfaced with the Solar 9500 (General Electric^®) anesthetic monitors used in our institution’s operating rooms. During scheduled quality control checks within the first two months of the trial, it became clear that in some instances there was a failure of BIS values to be generated. Technical representatives from both manufacturers confirmed this as a known software interface problem. Since the study was designed as an effectiveness trial, the decision was made to proceed and use the population receiving neither the BIS nor anesthetic concentration protocol as a post hoc “no intervention” group for the purpose of secondary analysis. Failure to generate BIS values was similar in both even (17%) and odd (19%) numbered operating rooms, which was the randomization scheme for alerting protocols.

Main Outcome Measures

Blinded, trained interviewers used the modified Brice interview¹⁷ employed in other studies of intraoperative awareness^2,5–7 to screen patients 28 to 30 days after surgery via telephone. A single interview was performed in contrast to past trials^5–7 due to the high number of patients recruited; the 28- to 30-day interview was chosen because it would likely detect the most clinically significant awareness events. If patients could not be reached by telephone after multiple attempts, a written form of the interview was sent to the patient. Any patients reporting intraoperative awareness during the Brice interview had a more detailed interview by an anesthesiologist committee member blinded to the intervention. All patients reporting intraoperative awareness were offered psychiatric care.

For those patients who reported awareness, three blinded experts independently determined whether the reported event was definite, possible, or no awareness based on the data obtained from the first two interviews (Brice screening and follow-up). These individuals also reviewed awareness events for the BAG-RECALL trial (ClinicalTrials.gov number NCT00682825).^7,18 We compared interrater agreement using Fleiss’s Kappa statistic for the three blinded assessments of awareness, which showed fair agreement (0.25). In the event of a conflict, a fourth blinded expert reviewer from another institution made the final determination; this expert reviews cases for the American Society of Anesthesiologists Anesthesia Awareness Registry. The qualitative aspects of the awareness report were classified using the Michigan Awareness Classification Instrument.¹⁹ Class 1 is defined as isolated auditory perceptions, class 2 is tactile perceptions, class 3 is pain, class 4 is paralysis and class 5 is paralysis and pain. If an event is also associated with distress, the class number is modified with a “D.”

Anesthetic usage, time to meeting recovery room discharge criteria, and incidence of postoperative nausea/vomiting were prespecified secondary outcomes.¹⁴ Postanesthesia care unit discharge criteria include (among other variables): oxygen saturation >92% or preoperative baseline (at appropriate levels of supplemental oxygen), core temperature between 36° and 38° Celsius, normal heart rate and rhythm (or no worse than baseline status), other hemodynamic vital signs within normal physiologic range for age or within 20% of baseline values, normal neurological evaluation, pain score ≤4, postoperative nausea and vomiting ≤2. BIS values, MAC values, and doses of propofol, midazolam, fentanyl and morphine were assessed across all groups.

Statistical Analysis

The primary outcome was the incidence of definite intraoperative awareness in the anesthetic concentration and BIS groups using modified intention-to-treat analysis. Modified intention-to-treat was defined as a patient who was randomized and was interviewed at 30 days. Prespecified secondary analysis was conducted to determine the combined incidence of definite and possible awareness as well as the classification of events. Significance was assessed using a two-tailed Pearson chi-square test. Confidence intervals were calculated using Newcombe’s method without continuity correction.²⁰ The average number of paging alerts generated in the groups was compared with the incidence of definite or possible awareness events using a linear regression R-Squared test.

Patient characteristics, comorbidities, and risk factors for awareness (table 1) were analyzed to determine if there were statistically significant differences between the anesthetic concentration and BIS groups in the modified intention-to-treat analysis. The Kolmogorov-Smirnov statistic was used to determine normality for the two continuous variables (age and body mass index). If the p-value was significant (<0.05), the assumption of normality was violated and nonparametric analyses (e.g., Mann-Whitney U test) were used. Nonparametric data are presented as median and interquartile range [25^th to 75^th percentile]. Parametric data are presented as mean ± standard deviation. For categorical variables, a two-tailed Pearson chi-square test was used, where a p-value of <0.05 was considered statistically significant. All categorical data are presented as number (percentage). For ease of interpretation we have defined cardiovascular disease as having one or more of the following conditions: history of myocardial infarction, congestive heart failure, valvular heart disease, dysrhythmia, endocarditis, peripheral vascular occlusive disease, angina or orthopnea. We have defined lung disease as having one or more of the following conditions: history of pulmonary hypertension, chronic obstructive pulmonary disease or dyspnea. We have defined liver disease as having one or more of the following conditions: history of cirrhosis, acute liver failure, or chronic liver failure. We have defined neuropsychiatric disease as having one or more of the following conditions: history of stroke or transient ischemic attack, seizures, depression, bipolar disorder, anxiety disorder, or posttraumatic stress disorder. We have defined alcohol abuse as having three or more drinks daily and/or high withdrawal potential.

Table 1.

Patient Characteristics, Comorbidities, and Risk Factors for Awareness

	Anesthetic concentration n (%) (N=9,376)	Bispectral Index n (%) (N=9,460)	p-value
Male Sex	4,199 (45)	4,237(45)	0.99
Agein years*	53 [41 to 64]	53 [41 to 64]	0.79
Body Mass Index(kg/m2)*	28[25 to 33]	28[24 to 33]	0.50
Cardiovascular Disease	1,702 (18)	1,723 (18)	0.91
Lung Disease	950 (10)	967 (10)	0.84
Renal Disease	601 (6.4)	612 (6.5)	0.87
Liver Disease	88 (0.9)	58 (0.6)	0.01
Neuropsychiatric Disease	2,003 (21)	2,053 (22)	0.57
History of Awareness	50 (0.5)	59 (0.6)	0.41
History of Difficult Intubation	45 (0.5)	40 (0.4)	0.56
Narcotic Dependency	9 (0.1)	11 (0.1)	0.67
Alcohol Abuse	205 (2.2)	180 (1.9)	0.17
Current Anticonvulsant Therapy	222 (2.4)	202 (2.1)	0.28
Current Benzodiazepine, Barbiturates, or GABA agonist	3,490 (37)	3,438 (36)	0.21

^*Nonparametric data presented as median [25^th to 75^th percentile interquartile] range.

All categorical data elements are presented as number (%)

GABA = gamma aminobutyric acid

Cardiovascular Disease = History of myocardial infarction, congestive heart failure, valvular heart disease, dysrhythmia, endocarditis, peripheral vascular occlusive disease, angina and/or orthopnea. Lung Disease = History of pulmonary hypertension, chronic obstructive pulmonary disease, and/or dyspnea. Liver Disease = History of cirrhosis, acute liver failure and/or chronic liver failure. Neuropsychiatric Disease = History of stroke or transient ischemic attacks, seizures, depression, bipolar disorder, anxiety disorder, and/or posttraumatic stress disorder. Alcohol abuse= 3 or more drinks daily or high withdrawal potential.

For the other key secondary outcomes, all continuous elements were assessed for normality as described. We chose to use the post hoc grouping variable (anesthetic concentration, BIS, and no intervention) to assess the secondary outcomes and therefore post hoc comparison testing was employed for elements in table 2. The median BIS values were compared between the BIS and anesthetic concentration groups using a Mann-Whitney U test; a Kruskal-Wallis test was used to compare median anesthetic dosages and discharge times among the anesthetic concentration, BIS, and no intervention groups. A two-tailed Pearson chi-square test was used to compare the outcomes of nausea or vomiting among the three groups. Bonferroni adjustments were used for the Mann-Whitney U test variables. For the variables that were analyzed using the Kruskal-Wallis test, pair-wise comparisons using a series of Mann-Whitney U tests were performed if the omnibus test was significant. For the Bonferroni adjustment, we started at an alpha level of 0.05. Based on the number of comparisons required, the new alpha level to measure significance was 0.002. We calculated a total of 22 comparisons based on the number of embedded Mann-Whitney U tests that were performed for Kruskal-Wallis tests with significant omnibus tests. Only those pairwise comparisons with a p-value <0.002 were reported in table 2 as statistically significant differences. If there were no statistically significant pairwise comparisons, “NS” (no significance) was reported for ease of interpretation. Statistical software IBM SPSS statistics version 19 (IBM Corp, Somers, NY) was used.

Table 2.

Anesthetic Use and Recovery Variables

	Anesthetic concentration (N=9,376)	Bispectral Index (N=6,076)	No intervention (N=3,384)	p-value
BIS Values*	40[34 to 46]	40 [35 to 44]	N/A	NS
% complete data(n)	41%(3,885)	100%(6,076)	N/A
MAC Values**	0.9 [0.8 to 1.1]	0.9 [0.8 to 1.1]	0.9 [0.8 to 1.1]	<0.001(Pairwise-comparisons of No Intervention to Anesthetic Concentration AND No intervention to BIS)
%complete data(n)	98%(9,170)	99%(5,988)	98%(3,303)
Propofol Intraoperative Bolus (mg)**	170 [130 to 200]	180 [130 to 200]	170 [120 to 200]	<0.001(Pair-wise comparison of No intervention to BIS)
% complete data(n)	100%(9,376)	100%(6,076)	100%(3,384)
Midazolam (mg)**	2 [2to 4]	2 [2to 4]	2 [2to 4]	NS
% complete data(n)	100%(9,376)	100%(6,076)	100%(3,384)
Fentanyl (μg)**	175 [100 to 250]	200 [100 to 250]	150 [100 to 250]	<0.001(All pairwise comparisons)
% complete data(n)	100%(9,376)	100%(6,076)	100%(3,384)
Morphine (mg)**	0 [0 to 5]	0 [0 to 5]	0 [0 to 3]	<0.001(All pairwise comparisons)
% complete data(n)	>99.9%(9,374)	>99.9%(6,074)	>99.9%(3,383)
PACU Discharge Readiness(min)**	98 [66 to 140]	95 [64 to 138]	94 [64 to 133]	0.001(Pairwise comparison No intervention to anesthetic concentration)
% complete data(n)	91%(8,527)	91%(5,521)	90%(3,043)
No Nausea(% of patients)(n)***	92% (6,184)	93% (4,042)	93% (2,286)	NS
% complete data(n)	72%(6,787)	72%(4,403)	74%(2,506)
No Vomiting(% of patients)(n) ***	99% (7,149)	99% (4,617)	99% (2,608)	NS
% complete data(n)	78%(7,329)	77%(4,707)	79%(2,687)

Note: Bonferroni adjustments were made due to post hoc comparisons (α =0.002). All data using the Kruskal-Wallis test also had pair-wise comparisons using a series of Mann-Whitney U tests if the omnibus test was significant. Only p-values < 0.002 in the pairwise comparisons were reported as statistically significant differences for these post hoc tests.

^*Nonparametric data presented as median [25^th to 75^th percentile interquartile] ranges and evaluated using Mann-Whitney U test.

^**Nonparametric data presented as median [25^th to 75^th percentile interquartile] ranges and evaluated using Kruskal-Wallis test.

^***Categorical data evaluated using Pearson chi-square.

BIS=Bispectral Index; MAC=minimum alveolar concentration; PACU=postanesthesia care unit.

Results

Recruitment and patient characteristics

A total of 21,601 patients were enrolled in the study at the time of interim analysis, with a 97% recruitment rate (fig. 1). As described in the methods, the study was terminated due to futility. Of the study cohort, 18,836 or 87% of the patients were available for postoperative interview assessing awareness at one month; 9,460 patients were randomized to the BIS group and 9376 patients were randomized to the anesthetic concentration group (fig. 1). Patient characteristics and comorbidities for the modified intention-to-treat BIS and anesthetic concentration groups are demonstrated in table 1. There were no adverse events related to the study.

Figure 1.

Flow diagram of recruitment and follow-up interviews. BIS=Bispectral Index

Of the 9,460 patients randomized to the BIS intervention and successfully interviewed, 3,384 or 36% did not have BIS data recorded due to technical issues described in Materials and Methods (Technical Factors). This population was used for secondary analysis only as a post hoc control group because it had neither intervention; there were more females (p < 0.001) and more patients with lung disease (p = 0.002) in this group. Neither female sex nor lung disease were shown to be associated with an increased incidence of intraoperative awareness in our recent companion randomized controlled trial.⁷

Incidence of intraoperative awareness events

The overall incidence of definite awareness in the study cohort was 19/18,836 or 0.1%. By modified intention-to-treat analysis, the incidence of definite awareness was 11/9376 or 0.12% (95% CI 0.07 to 0.21%) in the group randomized to the anesthetic concentration protocol and 8/9460 or 0.08% (95% CI 0.04 to 0.16%) in the group that was randomized to receive BIS monitoring (p = 0.48, fig. 2). Using the Michigan Awareness Classification Instrument, no statistical differences in event or distress classes were found between the groups. Post hoc power analysis revealed that 102,951 patients in each group would be required to detect a difference between the two interventions. The 13% of recruited patients who did not complete interviews (e.g., due to death or lack of response) were unlikely to skew the reported incidence of intraoperative awareness. Assuming the same incidence rates found in the modified intention-to-treat groups, 100,000 simulations were run to generate cumulative distribution functions that demonstrate the probability of a significant difference of outcome if the 2,765 patients not interviewed were included. Using a Fisher’s exact test, the likelihood of a significant difference with inclusion of this population was 0.016%.

Figure 2.

Primary outcome of definite awareness in modified intention-to-treat groups, defined as those recruited, randomized to Bispectral Index (BIS)- or anesthetic concentration-guided care, and interviewed at one month for intraoperative awareness. Note that the BIS randomization group includes patients that did not have BIS values generated.

By post hoc analysis, the incidence of definite awareness was 11/9376 or 0.12% in the anesthetic concentration group, 3/6,076 or 0.05% in the group that actually received BIS monitoring, and 5/3,384 or 0.15% in the no intervention group (p = 0.27). Based on the 0.12% awareness incidence in the anesthetic concentration group and the 0.05% awareness incidence in the group that received BIS monitoring, a post hoc power analysis revealed that 29,996 patients in each group would be required to detect a difference between the two interventions. The combined incidence of definite and possible awareness cases was 0.08% in the group that received BIS monitoring, 0.20% in the anesthetic concentration group and 0.38% in the no intervention group (p = 0.006, fig. 3). By post hoc analysis, the cohort receiving no intervention had 4.7 times more definite or possible awareness events compared to the cohort receiving the BIS protocol (p = 0.001; 95% CI 1.7 to 13.1). Of patients with definite or possible awareness receiving BIS monitoring, 50% had no 5-min epoch of BIS values <60 during the case and 50% had at least one 5-min epoch of median BIS value >60.

Figure 3.

Secondary outcomes of definite or possible awareness in post hoc groups.

By secondary analysis using post hoc grouping, the average number of alerts in the no intervention group (0/case), anesthetic concentration group (1/case), and BIS group (2.2/case) varied inversely with the incidence of definite and possible awareness events (r² = 0.951).

BIS Values, Anesthetic Usage, and Recovery

The secondary outcome measures of anesthetic use and recovery times were performed using the post hoc comparison groups of anesthetic concentration, BIS, and no intervention. Since the decision was made to present the data using the three post hoc groups instead of the modified intention-to-treat grouping (BIS or anesthetic concentration), Bonferroni adjustments were performed as described in the Statistical Analysis section of the Materials and Methods. Data are presented in table 2, with only significant pair-wise comparisons reported. There was a statistically significant difference in the median MAC for pairwise comparisons of anesthetic concentration to no intervention groups and also for BIS to no intervention groups. Intraoperative propofol bolus dosing showed a significant pair-wise comparison between the BIS and no intervention groups. The total midazolam dose showed no statistically significant differences. Total fentanyl and total morphine use had statistically significant pair-wise comparisons for all combinations of the three grouping variables. Although statistically significant, the clinical relevance of these differences is unclear.

Median time to meeting recovery room discharge criteria was 98 min (interquartile range 66 to 140) for anesthetic concentration group, 95 min (interquartile range 64 to 138) for the BIS group, and 94 min (interquartile range 64 to 133) for the no intervention group. There was a significant pair-wise comparison between the no intervention and anesthetic concentration groups. There was no evidence for reduced recovery time in patients receiving BIS monitoring compared to no intervention. There was no statistically significant difference among the three groups for reduced nausea or reduced vomiting upon first assessment in the recovery room (table 2).

Discussion

This is the largest prospective randomized controlled trial ever conducted on the prevention of intraoperative awareness and the only such effectiveness trial. This negative study was unable to determine if an alerting protocol based on BIS values or anesthetic concentration was superior in preventing definite intraoperative awareness. Other conclusions of the study are that (1) comparative effectiveness trials with definitive results regarding the prevention of intraoperative awareness in unselected patients will likely not be feasible, (2) post hoc secondary analysis suggests that a protocol based on the BIS monitor probably reduces awareness events compared to routine care without a protocol, (3) increased provider alerting is a possible mechanism for decreasing awareness events when comparing two protocols, (4) the BIS monitoring protocol used in this trial is not associated with a reduction in the use of anesthetic drugs in routine clinical practice, and (5) the BIS monitoring protocol used in this trial is not associated with reduced recovery time or incidence of nausea and vomiting in routine clinical practice.

The B-Aware study demonstrated that a BIS-guided protocol significantly reduced the incidence of intraoperative awareness in a high-risk population compared to no intervention.⁵ Subsequently, the B-Unaware study demonstrated no difference between a BIS-guided and MAC-guided protocol in the high-risk population,⁶ a finding supported by the recent BAG-RECALL trial.⁷ The current study differs from all past trials in that it assessed awareness prevention in an unselected, representative surgical population as opposed to the high-risk population alone. The primary results of our study are consistent with the B-Unaware and BAG-RECALL trials in that no statistically significant difference in the prevention of intraoperative awareness could be demonstrated between anesthetic concentration and BIS monitoring protocols. However, the results of the post hoc secondary analysis are consistent with the B-Aware trial⁵ in that the BIS monitor showed a trend toward reducing the incidence of awareness events compared to a group with no intervention. One methodological similarity of the current trial, the B-Aware trial⁵ and the observational study by Ekman et al⁷ is that anesthetic administration was not restricted to potent inhaled agents alone, as it was in the B-Unaware and BAG-RECALL trials.^6,7 Our study supports the conclusion of a recent Cochrane database review suggesting that the BIS monitor may reduce awareness when compared to assessing clinical signs alone, but not when compared to a protocol based on anesthetic concentration.¹²

The use of the BIS monitor in the current study generated approximately twice as many alerts as that of the anesthetic concentration protocol. Therefore, increased alerting could potentially be a mechanism of decreased definite or possible awareness events, an interpretation supported by the results of our companion trial. In the BAG-RECALL study, the alarm frequency based on anesthetic concentration was approximately 2-fold higher than that based on BIS values; the higher alarm rate with the anesthetic concentration protocol was associated with fewer definite and possible awareness events. The different alerting threshold in BAG-RECALL (0.7MAC) and the current trial (0.5MAC) likely explains the ostensibly disparate outcomes. It is important to note that there was a high incidence of false positive alerting, which mitigates any conclusion regarding alerting protocols as a method of preventing awareness.

Efficacy trials and meta-analyses have suggested that the BIS monitor can significantly reduce consumption of anesthetic drugs, which leads to improved outcomes such as faster recovery or reduced nausea and vomiting. These data have been used to argue that the BIS monitor is cost-effective and should be routinely adopted for every general anesthetic.¹³ The BIS protocol used in the current study was not shown to reduce anesthetic dosing, which is in contrast to the recent Cochrane database review.¹² Furthermore, the BIS protocol used in the current study was not associated with reduced recovery time or reduced incidence of nausea and vomiting compared to routine care. One hypothesis to explain the discrepancy is that conclusions derived from efficacy trials or meta-analyses based on such trials are not sufficiently robust to hold in a test of effectiveness. Another hypothesis to explain the discrepancy is that the difference in BIS-guided protocols between the current and past studies led to disparate outcomes.

Limitations of our study include insufficient numbers to answer with precision whether and to what extent there is a difference in the definite awareness incidence between protocols based on BIS values and anesthetic concentrations. This limitation likely reflects the rarity of intraoperative awareness in an unselected surgical population and is informative regarding the future investigation of protocols to reduce awareness. Another limitation of the trial was the proportion of patients randomized to the BIS protocol who did not receive BIS monitoring. However, this unplanned technical issue has yielded useful secondary findings and is mitigated by the following considerations: (1) even complete compliance would almost certainly not have been sufficient to detect a significant difference in the modified intention-to-treat groups, (2) the population receiving neither intervention yielded useful information regarding the effect of anesthetic protocols compared to routine care, a matter of recent controversy,²¹ (3) the incidence of definite and possible awareness events in the no-intervention group was equivalent to that previously reported,^2,3 which validates the methodology of the trial and suggests that a single interview at 30 days was sufficient to detect clinically relevant intraoperative awareness, and (4) the number of prospectively-studied patients who received BIS monitoring nonetheless exceeds all major efficacy trials combined.^5–7

In conclusion, this effectiveness study could not detect a difference between BIS and anesthetic concentration protocols in reducing the incidence of definite intraoperative awareness with explicit recall. By post hoc analysis, we demonstrated that the BIS monitor may play a role in reducing intraoperative awareness compared to no intervention. These findings are consistent with conclusions of a Cochrane review based on various efficacy studies.¹² In contrast to the Cochrane review, the BIS protocol used in this study was not associated with improved recovery.

What we already know about this topic

• Intraoperative awareness with explicit recall occurs in 0.15% of surgical cases

• Randomized trials of BIS (bispectral index) monitoring have focused on high risk patients but few data are available in an unselected surgical population

What this article tells us that is new

• In this large effectiveness study (n = 21,601) no significant difference in intraoperative awareness with explicit recall was detected between BIS and anesthetic concentration protocols (0.08 vs. 0.12 %, p = 0.48) but, by post hoc analysis, BIS monitoring may decrease intraoperative awareness compared to routine care without a protocol