The Effect of Adherence to ACLS Protocols on Survival of Event in the Setting of In-Hospital Cardiac Arrest

Abstract

Aim

Advanced Cardiac Life Support (ACLS) algorithms are the default standard of care for in-hospital cardiac arrest (IHCA) management. However, adherence to published guidelines is relatively poor. The records of 149 patients who experienced IHCA were examined to begin to understand the association between overall adherence to ACLS protocols and successful return of spontaneous circulation (ROSC).

Methods

A retrospective chart review of medical records and code team worksheets was conducted for 75 patients who had ROSC after an IHCA event (SE group) and 74 who did not survive an IHCA event (DNS group). Protocol adherence was assessed using a detailed checklist based on the 2005 ACLS Update protocols. Several additional patient characteristics and circumstances were also examined as potential predictors of ROSC.

Results

In unadjusted analyses, the percentage of correct steps performed was positively correlated with ROSC from an IHCA (p <0.01), and the number of errors of commission and omission were both negatively correlated with ROSC from an IHCA (p <0.01). In multivariable models, the percentage of correct steps performed and the number of errors of commission and omission remained significantly predictive of ROSC (p<0.01 and p<0.0001, respectively) even after accounting for confounders such as the difference in age and location of the IHCAs.

Conclusions

Our results show that adherence to ACLS protocols throughout an event is correlated with increased ROSC in the setting of cardiac arrest. Furthermore, the results suggest that, in addition to correct actions, both wrong actions and omissions of indicated actions lead to decreased ROSC after IHCA.

Introduction

The American Heart Association (AHA) Advanced Cardiac Life Support (ACLS) algorithms are the standard of care for patients suffering cardiac arrest. Although previous reviews did not demonstrate the expected improvement in survival for cardiac arrests of all types, a recent review of 2000-2009 did show an improvement in both return of spontaneous circulation (ROSC) and survival-to-discharge rates.^1-3 It remains unclear how much of the recent improvement is attributable to improved training and adherence to specific algorithms as compared to other improvements, such as in the recommended ACLS algorithms themselves or earlier detection of cardiac arrests.

The presence of ACLS-trained personnel during cardiac arrest increases the likelihood of ROSC and is correlated with significantly better 1-year survival rates.⁴ Additionally, the presence of an anesthesiologist has been shown to reduce failure to rescue rates in emergency resuscitation situations.⁵ While key personnel are important, adherence to the specific content and timing of guidelines is often poor.⁶ The time to defibrillation during in-hospital cardiac arrest (IHCA) is often delayed, which is correlated with worse outcome.⁷ There is less robust data on the importance of the choice and timing of the medications, and the relationship between overall ACLS algorithm adherence throughout an entire IHCA event and patient outcome has not been documented.^7,8 Accordingly, we compared adherence to the 2005 AHA ACLS algorithms between initial survivors and non-survivors of IHCA. We hypothesized that adherence to the ACLS protocols would be significantly different between the two cohorts.

Methods

After the institutional review board approved this study and waived the requirement for written informed consent, a retrospective chart review of adherence to the 2005 AHA ACLS protocols during IHCAs was performed. Information was gathered on all in-hospital cardiac arrests between 2006 and 2008.

Setting and Patient Population

This study was performed at a 600-bed academic medical center. The personnel notified and responding to an IHCA at our institution include an internal medicine resident, an anesthesiology resident, a pharmacist, a respiratory therapist, an anesthesiology attending, and several nurses including an event recorder. All residents from every department are required to pass the AHA-endorsed ACLS provider course before beginning their training at our institution and to remain AHA ACLS current throughout their training.

For each event, the event recorder fills out a standard IHCA flow sheet, which contains demographic patient information, treated rhythm(s), event outcome, date of event, time of event, location of event, names of responders, and all actions taken during the event. This sheet contains all the data fields described in the Utstein Criteria and those entered into the Get With the Guidelines® database, along with a table for recording each action and the time that it was accomplished (e.g. drug administration, shock, etc). After each event, the physician leader and the event recorder both sign and date the IHCA flow sheet to verify accuracy of the recorded information. The critical interventions manager then maintains the database of these IHCAs, which are also reported to the Get With the Guidelines® database, for internal quality assurance and improvement.

The research staff was given access to the internal cardiac arrest database to allow identification of acceptable patients for enrollment. All adult (≥18 years) patients with a recorded cardiac arrest between December 2005 and June 2008 were screened for eligibility. To be included in the analysis, the patient must have suffered an IHCA, and the flow sheet must have been properly completed. Simple respiratory arrest patients were excluded to focus on the patients with cardiac dysrhythmias that required advanced medical management beyond basic life support and airway management. Proper completion of the event flow sheet included legible writing and completion of all data fields in a continuous timeline such that no gaps in data recording were evident.

Design

Enrollment of 150 patients (75 in each group) was targeted. This proportion was based upon published results of approximately 50% of IHCA patients having ROSC.^3,7 This sample size was based upon an estimate of identifying up to seven independent variables and van Belle's recommendations of needing at least 10 cases (deaths) for each independent variable that was to be included in a multi-variable analysis examining associations with ROSC after IHCA.⁹ The primary association of interest was ROSC after IHCA and adherence to the established ACLS protocols. In addition to adherence to ACLS protocols, as measured by percentage of correct checklist actions; the most likely confounding variables prospectively identified for analysis were age, BMI, location of event, time of event, training level of code team leader, and expected mortality risk index (derived from each patient's Medicare severity diagnosis-related group severity of illness group-based, severity of illness-adjusted).

Patient Selection and Data Collection

There were 1674 in-hospital cardiac and respiratory arrests during the three-year study period. After exclusion for age<18 years, respiratory arrest, and completeness and legibility of the event sheet, a total of 403 patients were identified who experienced an IHCA event in the specified period with a legible record. From those eligible IHCA patients, 75 patients who did not survive (DNS) to ROSC and 75 patients who did survive the event (SE) to ROSC were randomly selected, resulting in 150 total patients undergoing detailed abstraction for analysis. Prior to statistical analysis of the de-identified cases, one patient had to be removed from the DNS group due to unintended corruption of the case entry in the study database, thus decreasing the final number in the DNS group to 74 patients. Figure 1 illustrates the CONSORT diagram for final patient inclusion.

Figure 1.

CONSORT diagram for patient inclusion. Only adult in-hospital cardiac arrest patients with legible and complete charts were eligible for randomization. After meeting inclusion criteria, a random sample of 75 patients surviving to ROSC and a random sample of 75 patients that did not survive to ROSC formed the two groups of patients that underwent detailed abstraction and analysis.

From the handwritten event flow sheet and the electronic medical record, the patient's age, sex, BMI, race, date of the event, time of the event, location in the hospital at the time of the event, total number of rhythms treated, training level of the event leader (either resident or fellow/attending), and all actions performed during the event were recorded. Additionally, the expected mortality risk and the categorical severity of illness scores were extracted from the University Health Consortium database for each patient prior to de-identification of each case. Adherence to the 2005 ACLS protocol was then assessed using a validated grading checklist derived from AHA Guidelines.¹⁰ The number of correct expected steps and omitted expected steps was calculated per event and recorded, with each event often containing several pulseless patient states (e.g. VFIB, pulseless electrical activity, asystole). Additionally, errors of commission were recorded into two categories: actions or drug administrations considered wrong at any time during specific rhythm treatment (e.g. defibrillation during asystole) and otherwise correct actions performed at the wrong time (e.g. repeated defibrillation without intervening CPR). Patient location was categorized as monitored (ICU, step down units, emergency room, or perioperative areas) or unmonitored (general patient care floors). The date of the event was categorized into two epochs: January through June and July through December. The time of the event was categorized into ‘day’ and ‘night’ time epochs, 07:00 through 18:59 and 19:00 through 06:59.

Data Analysis

All continuous variables were compared between survivors and non-survivors using two-tailed (independent sample) t-tests. Categorical variables were analyzed using Fisher exact tests. Adherence to ACLS guidelines was analyzed as a percentage of correct steps performed during each cardiac arrest event. The percentage of correct steps was calculated by determining the number of actions (two-minute CPR intervals, drugs given, shocks administered, etc.) performed correctly divided by the total number of indicated actions according to ACLS protocols per event because the number of steps and the time course varied between events. All variables that were found to be significantly (p < 0.05) associated with patient outcome were then evaluated in the context of multi-variable logistic regression models. Because of a high degree of multi-colinearity between the percentage of correct steps and the total number of wrong actions (Spearman's ρ = -0.77), separate models were constructed to examine the association between these 2 variables and ROSC. For these analyses, conditional logistic regression models were used. In each of the models, age group (≤30, 31-40, 41-50, 51-60, 61-70, 71-80, and >80) was included as a stratification (conditioning) variable, and covariates included the expected mortality and a variable reflecting whether or not the patient arrested within a continuously monitored unit bed. Secondary multivariable analyses examined the influence of each type of incorrect action (wrong action or drug administration, wrong timing of correct action, or omission of action indicated by ACLS Protocols) on ROSC. All analyses were conducted using SAS v9.2 (Cary, NC).

Results

Demographics were not significantly different between groups except for age. There were no differences in sex (p=0.13), BMI (p=0.70), race (p=0.89)), or training level of the team leader (p=0.38) between groups. There was a significant difference in the average age between the two groups. The SE group averaged 7 years older than the DNS group (59.95±1.94 vs. 52.36±2.19, p<0.05). The SE group also had significantly higher expected mortality (0.28±0.23 vs. 0.19±0.22, p<0.05) and a higher proportion with an ‘extreme’ severity of illness classification (60.6% vs 39.4%, p <0.0001) than the DNS group. Further details of the patient characteristics are shown in Table 1.

Table 1.

Demographics of Patients
Variable	SE	DNS	p-value
Age	60.0 ± 1.9	52.4 ± 2.2	0.01
BMI	26.7 ±2.0	27.4 ± 1.9	0.70
Sex (Female/Male)	35/40	26/49	0.13
Race (Caucasian/African American/Other)	40/33/2	40/32/3	0.89
Categorical Severity of Illness (Minor/Moderate/Major/Extreme)	0/1/9/60	2/11/16/39	<0.0001
Expected Mortality Rate	0.28 ± 0.23	0.19 ± 0.22	0.02
Characteristics of IHCA Event Management
Variable	SE	DNS	p-value
Location (Monitored/Unmonitored)	54/21	42/33	0.04
Date (Jan.-Jun./Jul.-Dec.)	39/36	38/37	0.87
Time (19:00-6:59/7:00-18:59)	28/47	36/39	0.19
Training Level of Leader (Resident v. Faculty/Fellow)	37/29	32/34	0.38
Percent Actions Correct per ACLS Protocol	63.9 ± 2.0	50.0 ± 3.0	0.0002
Wrong Action or Drug Administration	0.8 ± 0.2	1.7 ± 0.3	0.005
Wrong Timing of Correct Action	0.8 ± 0.1	2.1 ±0.2	<0.0001
Omission of Action Indicated by ACLS Protocols	0.8 ± 0.1	1.7 ± 0.3	0.0008
Total number of wrong actions, wrong timings, omitted actions	2.4 ± 0.3	5.5 ± 0.5	<0.0001

Patients who had ROSC after the cardiac arrest were managed with a significantly higher adherence to protocols as measured by a higher percentage of correct actions performed during the event (Figure 2, p<0.001). Wrong actions or wrong drug administrations, wrong timing of action, and omission of actions indicated by ACLS protocols (team leader inaction) were significantly higher in the DNS group as compared to the SE group (Figure 3). Bivariate (unadjusted) analysis of location in the hospital suggested that patients have a better outcome if cardiac arrest occurs in a monitored area versus an unmonitored area (p<0.05). Outcome was not significantly associated with the time of year or the time of day in which cardiac arrest events occurred. These code event characteristics are also shown in Table 1.

Figure 2.

Association of adherence to published ACLS guidelines and the outcome of an IHCA event, as measured by the percentage of correct steps performed during an event. The percentage of correct steps was calculated by determining the number of actions (two-minute CPR intervals, drugs given, shocks administered, etc) performed correctly divided by the total number of indicated actions according to ACLS protocols per pulseless rhythm event.

Figure 3.

All different types of errors, defined as divergence from AHA ACLS protocol, were significantly associated with not surviving the event. This included both errors of commission and omission, as well as performing an otherwise correct action at an inappropriate time.

Even after stratifying by patient age group and location, and adjusting for expected mortality, multivariable conditional logistic regression analysis revealed that the odds of ROSC was higher among IHCAs in which a greater percentage of correct actions were performed. On average, results of the multivariable model indicated that each ten-percentage point increase in adherence to protocols (e.g. from 60% correct to 70% correct) was associated with a 29% increased odds of ROSC (odds ratio [OR]: 1.29; 95% confidence interval [CI]: 1.09 to 1.53; p<0.01). Figure 4 illustrates the trend between the likelihood of ROSC (as assessed by the multivariable model) and the percentage of the correct steps performed during IHCA management. After similar covariate stratification and adjustment, the analysis also revealed that the odds of ROSC was lower among attempted resuscitations in which more total numbers of wrong actions occurred. Each additional error was associated with a 30% decreased odds of ROSC (odds ratio [OR]: 0.70, 95% confidence interval [CI]: 0.59 to 0.83, p<0.0001). More than one incorrect action translated into a significantly reduced chance of ROSC (see Figure 5).

Figure 4.

When the 149 total patients are divided into three groups based upon ACLS adherence rather than by ROSC (0-33%, 34-67%, and 68-100% correct actions), there is a significant trend for increasing survival to ROSC with increasing ACLS adherence. Adhering to >70% of the correct ACLS actions, independent of other factors, appears to be associated with nearly doubling the ROSC rate. Error bars reflect 95% confidence intervals.

Figure 5.

When the 149 total patients are divided into three groups based upon the number of divergences from ACLS guidelines, as measured by the total number of incorrect actions performed during IHCA management, there appears to be a significant inflection point for decreased survival to ROSC. Accumulating >1 error during IHCA management appears to be associated with roughly halving the ROSC rate.

In both of the multivariable logistic models, whether the patient's location was monitored was no longer significantly associated with ROSC (p=0.08 and p=0.30, respectively). A secondary multivariable analysis suggested that after covariate stratification and adjustment, the odds of ROSC was significantly lower when an incorrect medication was given (OR: 0.3, 95% CI: 0.1 to 0.6, p<0.01) or if a medication was given at an inappropriate time (OR: 0.3, 95% CI: 0.1 to 0.6, p<0.01), but not when an action indicated by the protocol was omitted by the code team (OR = 1.3, 95% CI: 0.6 to 3.0, p=0.53).

Discussion

The data from this study present several novel findings on outcomes of IHCA. First, adherence to ACLS protocols throughout an event is correlated with increased ROSC in the setting of cardiac arrest. Previous studies have shown the importance of time to first defibrillation as being correlated with outcomes.^7,8 However, to our knowledge, adherence to published ACLS protocols throughout the entire resuscitation event has not been previously reported. Wayne et al. have previously shown that high-fidelity ACLS simulation training improves adherence to guidelines throughout an IHCA event in the clinical setting as compared to traditional training. However, they were unable to show an effect on outcomes.⁶ This difference in results may be due to the fact that different AHA ACLS guidelines were used in grading the team leader performance in our study versus this prior study (we used the 2005 ACLS Update and the Wayne et al. study noted use of the guidelines current in 2003-2004). A major change in the 2005 ACLS Update was that 3 stacked shocks were no longer used in VFIB and CPR was given a major emphasis. Thus, the numerical adherence to guideline rates, while very similar between the Wayne et al. study and ours, may illustrate adherence to different and improved guidelines. This interpretation is congruent with the most recent retrospective review demonstrating an improvement of ROSC from 42.7% to 54.1% for IHCA from 2000 to 2009 as the 2005 ACLS Update concepts were incorporated into practice, while previous retrospective reviews have not been able to demonstrate such improvement.^1-3

The second interesting finding of this study is that in addition to correct actions, both commission of wrong actions and omissions of indicated actions likely lead to decreased ROSC. As mentioned above, numerous studies have reported the importance of the timing of first events in a resuscitation (e.g. time to first shock, time to first vasopressor), as is reported in the Get With the Guidelines® database. However, a detailed recording of deviations from published protocols and the effect of this on outcomes has not previously been reported to our knowledge for complete IHCA events. Thus, the significant negative association that our results illustrate between patient outcome and both the wrong timing of actions and wrong drug administrations adds a new knowledge concerning IHCA management. Training courses have typically focused on making sure that the trainee did all of the right steps. However, it may be just as significant to ensure that they do not do any wrong steps in order to produce maximally trained practitioners.

Finally, these results are consistent with previous reports suggesting that cardiac arrest in an intensive care unit or monitored step-down unit is associated with improved outcomes in cardiac arrest.⁷ This is also consistent with the recent publication that intraoperative arrest (the most monitored setting) is associated with improved outcomes compared to other settings.⁸ However, the results do conflict with prior reports concerning the effects of age on the outcome of an IHCA, as previous reports suggest that older patients have a lower rate of survival.¹¹ Our results showed an inverse association of age and ROSC. It appears our study population may be somewhat different from prior reports, as our average age was approximately a decade younger. Thus, it is possible that our patient population that did not survive, while younger, was also sicker than those that did survive. We attempted to use expected mortality derived from administrative billing data to help ensure that the lack of ROSC in our DSE group was not simply due to an increased severity of illness. Interestingly, we found that our DSE group had a significantly lower expected mortality. This unexpected finding may represent a limitation of using retrospective administrative billing data to calculate expected mortality. However, these unexpected findings that our SE group contained older, and perhaps sicker, patients does suggest that the effect of ACLS protocol adherence was robust enough to overcome the previously expected trend of older, and perhaps sicker, patients being less likely to survive an IHCA; although we can still only make statements of association, not causation. These findings may further strengthen our primary results indicating that once a patient has entered a cardiopulmonary arrest state (which one could alternatively argue makes both groups equally sick/dead without appropriate treatment), proper ACLS protocol adherence appears to be the most significant predictor or successful ROSC.

This study further highlights the widespread deficiency that exists in ACLS performance despite ACLS increased attention in the educational process for ACLS certification. Previous educational research has defined 75% correct steps as a cut-off for demonstrating competent performance.¹² As seen in Figure 4, our data show a possible clinical validation of that previous cut-off for determining competency, as a possible inflection point favoring ROSC exists at >70% adherence to guidelines in our data. Additionally, poor adherence to protocols by ACLS-certified personnel suggests that opportunities exist for research concerning the best pedagogical approach to improve clinical performance, as well as consideration of a shortened re-training or re-testing period to maintain ACLS certification. As a whole, the results of this study provide evidence that supports the usefulness of the ACLS protocols at a level of analysis that has not been previously reported, as greater adherence to the 2005 ACLS Guidelines was associated with improved patient ROSC after IHCA.

There are several limitations to the present study. First, it is retrospective in nature and therefore the study is potentially prone to selection bias and we can only make statements about association rather than causality. Second, the actions were graded under the 2005 guidelines rather than the most recent 2010 update. However, other than the emphasis on improved CPR in the 2010 update, the only specific change in pulseless management that would have affected our grading of the events is the exclusion of atropine from PEA and asystole protocols. Third, it is not nearly as large as other studies that are published from national databases and thus may not be generalizable. However, those databases do not contain the granularity of data that we analyzed in this study. If documentation of IHCAs were done in a standard electronic format that captured more discreet components of the resuscitative attempts, then national databases could have enhanced abilities including elucidating which specific protocol components are the most important to patient survival. Fourth, we did not seek to confirm that improved ACLS guideline adherence also translated into longer term benefits such as increased survival-to-discharge or into improved neurologic status at discharge. Finally, quality of CPR is a very important factor in the overall resuscitation event, but data concerning this was unable to be obtained retrospectively. Perhaps there is an unmeasured bias towards improved quality of CPR in monitored units that might explain the trend towards improved ROSC in those locations. At the time of this study, our institution used colormetric confirmation of endotracheal intubation. Thus, objective data about CPR quality as can be gathered from capnometry was not available. Additionally, diastolic pressures from arterial line tracings were not recorded for those patients that had this monitor in place.

Conclusions

In conclusion, our results demonstrate that adherence to ACLS protocols throughout an event is associated with increased ROSC in the setting of IHCA. Furthermore, the results illustrate that both commissions of wrong actions and omissions of indicated actions are associated with decreased ROSC after such an event. Additionally, poor adherence to protocols by ACLS-certified personnel suggests that significant opportunities still exist for improving retention of knowledge regarding ACLS protocols. Future studies need to address the best pedagogical for improving adherence to guidelines by resuscitation teams.