The System-Wide Effect of Real-Time Audiovisual Feedback and Postevent Debriefing for In-Hospital Cardiac Arrest: The Cardiopulmonary Resuscitation Quality Improvement Initiative*

Supplemental Digital Content is available in the text.

Objective:

To evaluate the effect of implementing real-time audiovisual feedback with and without postevent debriefing on survival and quality of cardiopulmonary resuscitation quality at in-hospital cardiac arrest.

Design:

A two-phase, multicentre prospective cohort study.

Setting:

Three UK hospitals, all part of one National Health Service Acute Trust.

Patients:

One thousand three hundred and ninety-five adult patients who sustained an in-hospital cardiac arrest at the study hospitals and were treated by hospital emergency teams between November 2009 and May 2013.

Interventions:

During phase 1, quality of cardiopulmonary resuscitation and patient outcomes were measured with no intervention implemented. During phase 2, staff at hospital 1 received real-time audiovisual feedback, whereas staff at hospital 2 received real-time audiovisual feedback supplemented by postevent debriefing. No intervention was implemented at hospital 3 during phase 2.

Measurements and Main Results:

The primary outcome was return of spontaneous circulation. Secondary endpoints included other patient-focused outcomes, such as survival to hospital discharge, and process-focused outcomes, such as chest compression depth. Random-effect logistic and linear regression models, adjusted for baseline patient characteristics, were used to analyze the effect of the interventions on study outcomes. In comparison with no intervention, neither real-time audiovisual feedback (adjusted odds ratio, 0.62; 95% CI, 0.31–1.22; p = 0.17) nor real-time audiovisual feedback supplemented by postevent debriefing (adjusted odds ratio, 0.65; 95% CI, 0.35–1.21; p = 0.17) was associated with a statistically significant improvement in return of spontaneous circulation or any process-focused outcome. Despite this, there was evidence of a system-wide improvement in phase 2, leading to improvements in return of spontaneous circulation (adjusted odds ratio, 1.87; 95% CI, 1.06–3.30; p = 0.03) and process-focused outcomes.

Conclusions:

Implementation of real-time audiovisual feedback with or without postevent debriefing did not lead to a measured improvement in patient or process-focused outcomes at individual hospital sites. However, there was an unexplained system-wide improvement in return of spontaneous circulation and process-focused outcomes during the second phase of the study.

The quality of cardiopulmonary resuscitation (CPR) is an important determinant of survival following cardiac arrest (1–5). However, observational data suggest that high-quality CPR is infrequently delivered in practice (6, 7). Approximately 35,000 patients have in-hospital cardiac arrests in the United Kingdom each year, of whom less than 20% survive to leave hospital (8). Strategies that improve adherence to resuscitation guidelines should lead to improved outcome following cardiac arrest (9).

Real-time audiovisual feedback and postevent debriefing have been identified as two such strategies. Studies conducted in a Chicago hospital provided early evidence that these strategies may be associated with improvements in CPR quality (10, 11). In 2010, the International Liaison Committee on Resuscitation identified the need for further research into both interventions (12). The objective of this study was to evaluate the effect of implementing real-time audiovisual feedback with and without postevent debriefing on survival and the quality of CPR at in-hospital cardiac arrest.

MATERIALS AND METHODS

Study Design and Setting

We conducted a two-phase prospective cohort study across the three hospital sites, which are part of Heart of England National Health Service (NHS) Foundation Trust, a large teaching acute NHS Trust with over 1,400 beds. The three hospitals are geographically distinct.

At each hospital site, cardiac arrests are managed by a multidisciplinary emergency team (13). The team is alerted to cardiac arrest events by a beeper system, which is activated by the hospital telephone switchboard on receipt of an emergency call. The team leader is a certified Advanced Life Support provider, whereas other clinical team members hold either Advanced Life Support or Immediate Life Support provider certification. Care is delivered in accordance with Resuscitation Council (UK) guidelines (14).

The study was approved by the Coventry research ethics committee, who waived the requirement to obtain informed consent from study participants.

Study Participants

Study participants were consecutive adults (≥ 18 yr) who had a cardiac arrest at study hospitals between November 2009 and May 2013, and who were treated by the hospital emergency team. Cardiac arrest was defined as the absence of a central pulse that required treatment with chest compressions or defibrillation. Patients were excluded if they had a documented “do not attempt CPR order,” had an out-of-hospital cardiac arrest, or had previously participated in the study.

Data Collection

Cardiac arrest events were identified through the emergency call register maintained by the hospital switchboard. For each cardiac arrest, a core dataset of demographic, CPR quality, and outcome data was collected. Standardized definitions for each variable were used (15, 16).

During the study, cardiac arrest trolleys in areas at high risk of cardiac arrest, such as hospital wards, emergency departments, and ICUs, were equipped with study defibrillators (Phillips MRX QCPR defibrillators; Phillips Healthcare, Andover, MA). Study defibrillators incorporated a small accelerometer puck that was placed on the patient’s chest during CPR, to record CPR quality data. Full details of the device have been reported previously (6, 17).

CPR process data were downloaded from study defibrillator records, and then up to the first five available minutes of data for each CPR variable (chest compression depth, chest compression rate, flow-fraction, chest compression incomplete release, preshock pause, and postshock pause) were extracted. Standardized definitions were used for each variable (16). This was done automatically for accelerometer data by manufacturer software (Event Review Pro 4.2; Phillips Healthcare, or QCPR Review version 2.1.0.0; Laerdal Medical, Stavanger, Norway). Where providers failed to use the puck accelerometer, transthoracic impedance or electrocardiogram data (all variables except chest compression depth and incomplete release) were analyzed manually by study personnel (R.A.F., M.C., K.C., J.Y.).

Study Interventions

The study consisted of two phases. During the first phase (November 2009 to November 2011), the study core dataset was collected for each eligible cardiac arrest event at the three study hospitals with no intervention delivered at any hospital. During phase 2 (November 2011 to May 2013), the real-time audiovisual feedback function was activated on study defibrillators at hospital 1 and at hospital 2. In addition, staff at hospital 2 also received cardiac arrest postevent debriefing. During phase 1 at all hospitals and during phase 2 at hospital three, both audio and visual feedback were disabled on study defibrillators.

Real-time audiovisual feedback was provided by study defibrillators (Phillips MRX QCPR defibrillator; Phillips Healthcare). The defibrillators provided audio and visual prompts to rescuers to enable correction of CPR technique to comply with current resuscitation guidelines (14). Training in the use of devices was delivered through mandatory CPR training and training roadshows that were undertaken within study hospitals. The postevent debriefing intervention at hospital 2 was based on that described by Edelson et al (11). Postevent debriefing meetings, which lasted approximately 45 minutes, were held on a weekly basis during phase 2. All interested clinicians were invited to attend meetings, where lunch was provided. Meetings reviewed recent cardiac arrest events and relevant research, with a particular focus on CPR quality. Defibrillator data downloads, including CPR quality data, were used to forensically reconstruct events. Full details of the study interventions, based on the TIDieR (template for intervention description and replication) framework are provided in the online supplement (Supplemental Digital Content 1, http://links.lww.com/CCM/B363) (18).

Outcome Measures

The primary study outcome was return of spontaneous circulation (ROSC), defined as a return of circulation that persisted for at least 20 minutes. A prestudy 12-month audit showed that the Trust’s baseline ROSC rate was 44%. To demonstrate a 16% absolute improvement in ROSC, as observed in the study of cardiac arrest postevent debriefing by Edelson et al (11), based on a simple comparison of the two phases, a sample size of 152 patients was required in each phase at each hospital site to achieve 80% power at a significance level of 0.05 (19). Simulations were conducted to demonstrate that this power was also achieved for the random-effects logistic regression model described below and to estimate the power using this analysis with the actual sample sizes obtained.

Secondary outcomes consisted of other patient-focused and process-focused outcomes. Patient-focused outcomes included survival to hospital discharge and discharge neurologic status, as measured by the cerebral performance category (CPC) score (8). CPC score was analyzed dichotomously as good (CPC 1 or 2) or poor (CPC 3, 4, or 5) neurologic outcome, as is standard in the resuscitation literature. Process-focused outcomes included chest compression depth, chest compression rate, no-flow fraction, preshock pause, postshock pause, and incidence of chest compression incomplete release.

Statistical Analysis

Baseline patient characteristics were summarized for patients in each study phase. Data for categorical variables were summarized using percentages, with proportions for phases 1 and 2 compared using chi-square tests. Data for continuous variables were summarized using means and sd, with means for phases 1 and 2 compared using t test.

Patient-related outcomes were binary and were analyzed using a random-effects logistic regression model to include a hospital random-effect term. Two sets of models (unadjusted and adjusted) were fitted. Unadjusted models only included terms for intervention and phase and a random hospital effect. To account for the implementation of the 2010 resuscitation guidelines, which took place on December 1, 2010, the variable for phase was recategorized into three categories: phase 1 with 2005 resuscitation guidelines, phase 1 with 2010 resuscitation guidelines, and phase 2. The effect for phase reported in the Results section is the difference between phase 1 with 2010 resuscitation guidelines and phase 2, which coincides with the implementation of interventions in hospitals 1 and 2. The estimated effect of the intervention is thus adjusted for changes between the different phases and between the periods before and after the introduction of 2010 resuscitation guidelines and for differences between the three hospitals. As the experimental design means that the intervention effect is confounded with an interaction between phase and intervention, interpretation of the results assumes that the change in effects from one phase to the next is the same across the three hospitals. Adjusted models additionally adjusted for patient baseline characteristics. Results are presented as unadjusted and adjusted odds ratio (aOR) and 95% CI.

Process-related outcomes were either binary or continuous and were analyzed using random-effects logistic regression models or linear-mixed models, respectively, to include a hospital random-effect term. These models were not adjusted for baseline characteristics because it was considered that patient factors should not affect delivery of high-quality CPR and so the models only included a term for intervention and a term for phase, with phase recategorized as above to adjust for implementation of the 2010 guidelines. Results for binary outcomes are presented as OR (95% CI), whereas results for continuous outcomes are reported as mean difference (95% CI). For continuous variables, outcome variances in the different intervention groups were also compared using the F test, with results presented as F-statistic and p value.

The primary intention-to-treat analysis compared intervention patients with control group patients (all patients in phase 1 plus hospital 3 patients in phase 2). Two secondary analyses, which were not defined a priori, were also undertaken. The first of these was a per-protocol analysis whereby the primary analysis was repeated with only patients for whom accelerometer data were available. The second compared phase 2 patients with phase 1 patients. This was undertaken within hospitals and across all hospitals to analyze for the existence of a system-wide effect. In all analyses, a p value of less than or equal to 0.05 was considered statistically significant. All models were fitted using the gamm4 package (gamm4: Generalized additive mixed models using mgcv and lme4, Simon Wood, 2011, http://CRAN.R-project.org/package=gamm4) in R statistical program (R: A Language and Environment for Statistical Computing, R Development Core Team; R Foundation for Statistical Computing, Vienna, Austria).

RESULTS

During the study period, 1,739 cardiac arrest events were screened for study inclusion, of which 1,395 (761 phase 1 and 634 phase 2) were eligible (Fig. 1). Reasons for exclusion included out-of-hospital cardiac arrest (n = 215), do not attempt CPR order (n = 6), previous study participation (n = 71), and cardiac arrest events in the 3-week transition between study phases (n = 22). Patient outcome data were available for all participants, but process data were available for only 674 cardiac arrest events (48.32%). The median duration of process data included per event was 5 minutes for most process outcomes so sensitivity analyses of event duration were not conducted (online supplement, Supplemental Digital Content 1, http://links.lww.com/CCM/B363).

Figure 1.

Study flow diagram. *Other reasons for exclusion include patient did not sustain a cardiac arrest (n = 16), cardiac arrest not attended by hospital emergency team (n = 12), duplicate event in database (n = 1), technical error where defibrillator gave real-time feedback in study phase 1 (n =1). DNACPR = do not attempt cardiopulmonary resuscitation.

Real-time audiovisual feedback was activated on study defibrillators at hospitals 1 and 2 in November 2011. In total, it was used at 100 cardiac arrest events (58.8%) at hospital 1 and 188 cardiac arrest events (54.8%) at hospital 2. Between November 2011 and May 2013, 74 weekly postevent cardiac arrest debriefing meetings were held at hospital 2. Meetings were attended by a total of 323 clinicians, with a mean attendance per meeting of 12.6 (sd, 4.7).

Demographic data were similar for patients in phases 1 and 2 of the study, both within each hospital and across all hospitals (Table 1). Compared with no intervention, the primary analysis found that neither real-time audiovisual feedback (aOR, 0.62; 95% CI, 0.31–1.22; p = 0.17) nor real-time audiovisual feedback supplemented by postevent debriefing (aOR, 0.65; 95% CI, 0.35–1.21; p = 0.17) had a statistically significant effect on return of spontaneous circulation or any patient-related outcome (Table 2). Similarly, neither intervention had a statistically significant effect on any process-related outcome (Table 3).

TABLE 1.

Demographic Data by Hospital Site

TABLE 2.

Analyses and Patient-Focused Outcomes

TABLE 3.

Analyses of Process-Focused Outcomes

Similar results to the primary analysis were observed in per-protocol analysis, with neither intervention being found to have a statistically significant effect on any patient outcome (Table 2). In this analysis, real-time audiovisual feedback supplemented by postevent debriefing was found to increase the proportion of patients who received a chest compression rate in the range of 100–120 chest compressions per minute (OR, 2.42; 95% CI, 1.17–4.98; p = 0.02). In addition, the provision of real-time feedback alone was actually associated with a decrease in chest compression flow-fraction (mean difference, –4.56; 95% CI, –7.64 to –1.48; p = 0.004). For all other process-focused outcomes, neither intervention was found to affect CPR quality (Table 3).

This lack of effect was attributed to marked improvements in CPR quality that were observed during phase 2 at the control site (Table 4). The analysis that compared phase 2 with those in phase 1 found that patients in phase 2 were more likely to achieve ROSC (aOR, 1.87; 95% CI, 1.06–3.30; p = 0.03). However, this improvement did not translate to statistically significant improvements in survival to discharge or neurologically intact survival (Table 2). Within each hospital site, there was no difference between study phases for any patient outcome (Table 4; and online supplement, Supplemental Digital Content 1, http://links.lww.com/CCM/B363).

TABLE 4.

Patient-Focused and Process-Focused Outcomes by Hospital Site

This analysis also identified system-wide improvements between phases 1 and 2 in process-focused outcomes, both across all hospitals and within hospital sites (Tables 3 and 4; and online supplement, Supplemental Digital Content 1, http://links.lww.com/CCM/B363). In particular, the likelihood of the delivery of guideline-adherent care improved across all hospitals in phase 2 in relation to chest compression rate (OR, 2.90; 95% CI, 1.49–5.66; p = 0.002), chest compression depth (OR, 2.11; 95% CI, 1.00–4.44; p = 0.05), but there was no change in relation to chest compression incomplete release (OR, 0.98; 95% CI, 0.47–2.09; p = 0.98). We also observed an improvement in flow-fraction (mean difference, 5.91; 95% CI, 3.17–8.64; p < 0.001). There was also evidence of reduced outcome variability between phases 1 and 2 in relation to chest compression rate (F = 1.575; p < 0.001) and chest compression depth (F = 1.594; p = 0.001), but not in relation to flow-fraction (F = 1.070; p = 0.27).

DISCUSSION

This prospective cohort study assessed the effect of implementing real-time audiovisual feedback and real-time audiovisual feedback supplemented by postevent debriefing on patient and process-focused outcomes at in-hospital adult cardiac arrest. The interventions were intended to improve clinician adherence to cardiac arrest guidelines.

In the initial analysis, there was no evidence of an association between return of spontaneous circulation and the implementation of either real-time audiovisual feedback (aOR, 0.62; 95% CI, 0.31–1.22; p = 0.17) or real-time audiovisual feedback supplemented by postevent debriefing (aOR, 0.65; 95% CI, 0.35–1.21; p = 0.17). Similarly, the interventions were not associated with a change in survival to discharge, neurologic outcome, or any process-focused outcome. A secondary analysis assessed for evidence of a system-wide impact on patient and process-focused outcomes by comparing study phases 1 and 2. This demonstrated an improvement in return of spontaneous circulation (aOR, 1.87; 95% CI, 1.06–3.30; p = 0.03) across all hospitals and improvements in process-focused outcomes, both across all hospitals and within individual hospitals. Although there was no effect on survival to discharge or neurologic outcome, the study was not powered to detect such differences.

These findings may be explained in a number of ways. First, it is possible that the interventions had no effect on CPR quality, due to either the interventions themselves being ineffective or due to poor implementation. In this case, the results of the secondary analysis, which treats the study as a before-after study, may simply reflect a change over time. However, if this were the case, then it would be reasonable to expect the change in ROSC to mirror that observed in the “Get with the guidelines” in-hospital cardiac arrest registry, where an association has been observed between the length of a hospital’s registry participation and ROSC (20). However, the ROSC improvement in this study (aOR, 1.87; 95% CI, 1.06–3.30) is markedly higher than the improvement associated with each year of registry participation (aOR, 1.02; 95% CI, 1.00–1.04), making it unlikely that the observed improvements in this study simply represent an improvement over time.

A second explanation may be that the observed effect was real, but caused by factors external to the interventions. Implementation of new defibrillator technology required staff training in its usage, both initially and through annual resuscitation updates. As a result, staff were aware of the ongoing study examining CPR quality. This likely led to an increased awareness of the importance of CPR and contributed to a Hawthorne effect. However, as before, the magnitude of the observed improvements makes it unlikely that this explanation fully explains the observed effects, given that implementation of this defibrillator technology in previous before-after studies have produced very modest improvements in CPR quality and had no impact on patient outcome (10, 21).

Another external factor of note was the implementation of the 2010 resuscitation guidelines during phase 1 of the study (14). These guidelines placed increased emphasis on CPR quality and increased the recommended chest compression depth. The implementation of guidelines in clinical practice is complex, such that the most effective guideline implementation strategy remains unclear (9, 22). It may be that interventions improved practice by reinforcing best resuscitation practice, given that improvements in care delivery extended to metrics, such as chest compression rate, which were not changed by the 2010 guidelines. Our statistical models sought to adjust for changes caused by guideline changes in phase 1 although this change may nevertheless represent a potential confounder in our analyses.

A more likely explanation for our findings is that although interventions were effective, there was intersite intervention contamination. In this respect, the greatest strength of this study also represented its greatest weakness. The high likelihood of a learning effect associated with interventions precluded the use of patient randomization. Therefore, we opted for a prospective cohort study design with interventions allocated by hospital site in an attempt to minimize bias through contamination. To avoid the methodological weaknesses associated with before-after studies, no intervention was delivered at hospital 3 for the duration of the study to allow for the estimation of system-wide temporal changes. This approach would also allow an estimation of the effect associated with real-time audiovisual feedback when implemented alone, and when delivered alongside debriefing.

The approach, however, proved to be flawed. During the study, we observed an unanticipatedly high level of staff rotation between sites. Although intersite rotation incorporated all staff groups including doctors, it was particularly marked among critical care outreach staff. Critical care outreach staff are experienced critical care nurses, who have both a clinical and an educational component to their role (23). They form a core component of the hospital emergency team on a 24-hour basis at each site and may act as emergency team leader. Although personnel are predominantly based at one hospital, they are frequently required to work at other hospitals within the organization. This staff rotation likely led to cross-site contamination, which may have contributed to the observed system-wide improvement in CPR quality and ROSC. As a result of this contamination, it was no longer possible to compare the impact of real-time audiovisual feedback versus real-time audiovisual feedback implemented alongside debriefing.

Contamination is frequently cited as a concern during study design, but it is a poorly researched area of study design and analysis (24, 25). In this study, factors such as undertaking the study in a single organization, rotation of clinicians between intervention sites, and inclusion of clinicians from the same team, all likely increased the risk of contamination (25). Cluster randomized controlled trials are frequently cited as a useful trial design when there is a risk of contamination. Access to only three hospital sites meant that cluster randomization was not feasible in this study. Furthermore, even a well-designed cluster randomized study may not fully obviate the risk of contamination. For example, in a cluster randomized trial examining the implementation of medical emergency teams, one possible explanation for the absence of effect was increased awareness of medical emergency teams and patient safety nationally leading to improvements at control hospitals (26).

The study results broadly correlate with findings in other studies. Real-time audiovisual feedback may be provided by a range of devices, ranging from basic metronomes to accelerometer devices, but the impact of these devices may vary significantly (27). A recent meta-analysis of three clinical studies of real-time audiovisual feedback found that the technology was associated with significant improvements in CPR quality, but this did not translate in to an improvement in ROSC (28). Postevent debriefing is a complex intervention, with many connotations described in the literature (29). A meta-analysis of four studies reported that the intervention was associated with an improvement in CPR quality and ROSC (OR, 1.46; 95% CI, 1.01–2.13; p = 0.05) (30).

In practice, real-time audiovisual feedback is often provided alongside debriefing because the technology enables the collection of CPR quality data, which can be used to forensically reconstruct cardiac arrest events for postevent debriefing (29). In a simulation study, this combination of approaches had a cumulative effect on CPR quality (31). Clinical studies have described the implementation of cardiac arrest improvement bundles, where postevent debriefing, real-time audiovisual feedback, or both interventions are combined with other interventions, such as pit-crew models of resuscitation, in situ mock cardiac arrests, and intensive training (32–35). Such approaches have produced impressive results, with studies reporting increased hospital survival in both pediatric in-hospital and adult out-of-hospital cardiac arrest (32, 34).

Published data suggest that both real-time audiovisual feedback and postevent debriefing are associated with improvements in CPR quality. In some studies, this has translated into improvements in ROSC and hospital survival. However, most studies in this area have adopted a before/after intervention study design, making it difficult to establish a causal relationship between intervention and effect (36). The 2010 International Liaison Committee on Resuscitation Consensus on Science document made a somewhat cautious recommendation supporting the use of both interventions (12). More recently, American Heart Association consensus statements have advocated the implementation of both real-time audiovisual feedback and cardiac arrest debriefing (37, 38).

Our study has several strengths and weaknesses. A key strength is that it is the largest study to date of the use of real-time audiovisual feedback and postevent debriefing in the in-hospital setting. We attempted to design a study that incorporated a control group during phase 2 of the study, but, as discussed, intervention contamination may have led to improvements at the control site. This evidence of potential contamination prompted us to analyze for a system-wide effect, using an analysis that we had not defined a priori. Although this post hoc analysis should be treated with some caution, we felt it justified in the circumstances. A further study limitation is that the study was conducted in a single UK NHS Trust although it is noteworthy that demographic and outcome data in this cohort were broadly comparable with those reported in the UK national in-hospital cardiac arrest audit (8). Clinicians often failed to use the accelerometer puck that collected chest compression depth and incomplete release data. This was a particular problem in the phase 1 of the study, where clinicians received no immediate benefit from use of the technology. Because puck users particularly in phase 1 were likely more interested in resuscitation, they may have delivered higher quality CPR thereby inflating CPR quality in phase 1 of the study and reducing observed differences between phases 1 and 2. Finally, process data were only available for 48% cardiac arrest events. Unfortunately, we did not maintain a record of reasons for nonavailability of data. However, some common reasons for this, such as short cardiac arrest events and technically inadequate data, were outside our control. It is accepted that this may have biased our analysis of process data.

CONCLUSION

Implementation of real-time audiovisual feedback with or without postevent debriefing did not lead to a measurable improvement in patient or process-related outcome at individual sites. However, there was an unexplained system-wide improvement in ROSC and CPR quality, which may be attributable to factors such as contamination of intervention effects between sites or the implementation of the 2010 resuscitation guidelines.

ACKNOWLEDGMENTS

CPR Quality Improvement Initiative Collaborators: Naheed Akhtar—Heart of England NHS Foundation Trust, Birmingham, United Kingdom, and Warwick Medical School, University of Warwick, Coventry, United Kingdom; Annalie Baker—Heart of England NHS Foundation Trust, Birmingham, United Kingdom; Father Neil Bayliss—Patient Representative; Michelle Davies—Heart of England NHS Foundation Trust, Birmingham, United Kingdom; Natalie Husselbee—Heart of England NHS Foundation Trust, Birmingham, United Kingdom; John Long—Patient Representative; Teresa Melody—Heart of England NHS Foundation Trust, Birmingham, United Kingdom; and Joyce Yeung—Heart of England NHS Foundation Trust, Birmingham, United Kingdom, and School of Clinical and Experimental Medicine, College of Medical and Dental Science, University of Birmingham, Birmingham, United Kingdom.

Supplementary Material

ccm-43-2321-s001.docx ^{(25.8KB, docx)}