Classification of Cardiopulmonary Resuscitation Chest Compression Patterns: Manual Versus Automated Approaches

Abstract

Objectives

New chest compression detection technology allows for the recording and graphical depiction of clinical cardiopulmonary resuscitation (CPR) chest compressions. The authors sought to determine the inter-rater reliability of chest compression pattern classifications by human raters. Agreement with automated chest compression classification was also evaluated by computer analysis.

Methods

This was an analysis of chest compression patterns from cardiac arrest patients enrolled in the ongoing Resuscitation Outcomes Consortium (ROC) Continuous Chest Compressions Trial. Thirty CPR process files from patients in the trial were selected. Using written guidelines, research coordinators from each of eight participating ROC sites classified each chest compression pattern as 30:2 chest compressions, continuous chest compressions (CCC), or indeterminate. A computer algorithm for automated chest compression classification was also developed for each case. Inter-rater agreement between manual classifications was tested using Fleiss’s kappa. The criterion standard was defined as the classification assigned by the majority of manual raters. Agreement between the automated classification and the criterion standard manual classifications was also tested.

Results

The majority of the eight raters classified 12 chest compression patterns as 30:2, 12 as CCC, and six as indeterminate. Inter-rater agreement between manual classifications of chest compression patterns was κ = 0.62 (95% confidence interval [CI] = 0.49 to 0.74). The automated computer algorithm classified chest compression patterns as 30:2 (n = 15), CCC (n = 12), and indeterminate (n = 3). Agreement between automated and criterion standard manual classifications was κ = 0.84 (95% CI = 0.59 to 0.95).

Conclusions

In this study, good inter-rater agreement in the manual classification of CPR chest compression patterns was observed. Automated classification showed strong agreement with human ratings. These observations support the consistency of manual CPR pattern classification as well as the use of automated approaches to chest compression pattern analysis.

The advent of cardiopulmonary resuscitation (CPR) chest compression detection technology is one of the most important advances in resuscitation science and practice. Using accelerometer or electrical impedance sensors, this technology has enabled characterization of CPR chest compression delivery during clinical resuscitation efforts. Prior studies have used CPR process data to describe interruptions in chest compressions, as well as the associations between chest compression fraction and out-of-hospital cardiac arrest outcomes.^1-6

Recent studies have promoted novel strategies for CPR using continuous chest compressions (CCC) with few or no pauses for ventilation.^7-9 However, these prior studies relied on rescuer self-reports to characterize the patterns of delivered chest compressions, without the use of CPR measurement technology. The classification of observed chest compression patterns (e.g., CCC vs. 30:2 vs. other) requires manual interpretation of CPR process data, a process that is arduous, is time-consuming, and has unknown inter-rater agreement. Automated computer analysis could potentially improve the efficiency of CPR pattern classification, but no studies have described this technique nor compared its accuracy with manual CPR patterns classifications.

In this study of CPR delivered in the Resuscitation Outcomes Consortium (ROC) CCC Trial, we determined the inter-rater reliability of CPR chest compression patterns classified by manual data review. We also compared manual and automated computer approaches to chest compression pattern classification.

METHODS

Study Design

We conducted an analysis of chest compression patterns from cardiac arrest patients enrolled in the ongoing ROC CCC Trial. The ROC CCC Trial (www.clinicaltrials.gov NCT01372748) is conducted under US regulations for exception from informed consent for emergency research (21 CFR 50.24) and the Canadian Tri-Council Policy Statement: Ethical Conduct for Research Involving Humans. Additional reviews and approvals were obtained from the Office of Human Research Protection and Health Canada, as well as the institutional review boards and research ethics boards in the communities where the research was conducted.

Study Setting and Population

The ROC is a North American multicenter clinical trial network designed to conduct out-of-hospital interventional and clinical research in the areas of cardiac arrest and traumatic injury.^10,11 Of the 264 emergency medical services (EMS) agencies in ROC, 101 from eight ROC regional sites (Alabama; Dallas, Texas; King County, Washington; Milwaukee, Wisconsin; Pittsburgh, Pennsylvania; British Columbia, Canada; and Ottawa and Toronto, Ontario, Canada) are participating in the ROC CCC Trial.^12-14

The aim of the ROC CCC Trial is to compare survival to hospital discharge between adult out-of-hospital cardiac arrests randomized to a strategy of 30:2 CPR chest compressions versus a strategy of CCC. The protocol entails three consecutive 2-minute bouts of chest compressions. The 30:2 chest compressions arm consists of 30 chest compressions alternating with a full pause for the delivery of two ventilations by bag-valve-mask device. CCC consists of continuously delivered chest compressions with a single, brief ventilation after every 10th compression, without chest compression interruptions.

Study Protocol

Procurement of CPR Chest Compression Process Data

All ROC EMS agencies record CPR chest compressions using state-of-the art portable cardiac monitors with chest compression detection technology, including the Zoll M and X series (Zoll, Inc., Chelmsford, MA), Philips MRX (Philips, Inc., Amsterdam, The Netherlands), and Physio-Control LifePak 12 and 15 (Physio-Control, Inc., Redmond, WA). The Philips device measures chest compressions through an accelerometer-based sternal detector. The Zoll device similarly uses a sternal detector, but the sensor is physically fixed between the two cardiac defibrillation pads. The Physio-Control device detects chest compressions through changes in electrical impedance between chest electrodes; there is no additional hardware required. Independent studies have verified the accuracy of these detection technologies.¹⁵ While minimal formal training is required for operation of the CPR detection technology, technical points reinforced during CPR training typically include (where applicable) proper midsternal placement of the detector, application of chest compressions over the detector, and strategies for maintaining proper detector placement.

All manufacturers provided commercially available processing software for translating the chest compression data into analyzable format. The software identified the timing of each individual chest compression, from which one can determine CPR performance parameters such as compression rate and the segment durations of chest compressions application and interruptions. Because of their use of a sternal accelerometer, only the Philips and Zoll devices are able to indicate chest compression depth. The Physio-Control device uses thoracic impedance changes and cannot ascertain chest compression depth.

Selection of CPR Chest Compression Process Cases

For this analysis we assembled a test set of 30 CPR chest compression process files selected from cardiac arrest cases enrolled in the trial. Based on a projected kappa of 0.6 and at least six raters, we estimated needing CPR pattern ratings on at least 30 cases to achieve a lower one-sided 95% confidence interval (CI) no lower than 0.48.^16,17 Although we planned to solicit ratings from eight raters, we designed the study for six raters to allow for potential subject dropout.

The study team selected candidate chest compression files to encompass the spectrum of chest compression patterns observed during the trial. The final selection of images included examples from both study intervention arms (CCC and 30:2) and seven of the eight ROC sites (Alabama, five; Dallas, five; Milwaukee, three; Pittsburgh, six; British Columbia, five; Ottawa, three; and Toronto, three). The candidate chest compression image set also included examples from the three brands of cardiac monitors used by participating EMS agencies (Zoll, 11; Philips, six; and Physio-Control, 13). We did not systematically sample images by site or manufacturer.

We provided CPR process images for the first 8 minutes of resuscitation, defined from the time of first reported EMS chest compressions and reflecting the 8-minute duration of the trial protocol (Figure 1 and Data Supplement S1, available as supporting information in the online version of this paper). For the Physio-Control and Zoll files, we provided per-minute summary statistics available from the respective software processing programs. We placed the chest compression images in random order without repeats. To minimize subject burden, we did not test intra-rater agreement, which would have required repeat assessment of 10 to 15 additional CPR process files.

Figure 1.

Sample chest compression process file and summary report from Physio-Control cardiac monitor. (Top) Compression process file; (bottom) summary report: Additional sample chest compression process files for Philips and Zoll monitors are provided in Data Supplement S1.

Manual Classification of Chest Compression Patterns

A trained research coordinator from each of the eight participating ROC regional coordinating centers reviewed the sample CPR process images. All raters possessed extensive CPR process analysis experience from prior ROC studies. Most raters had prior out-of-hospital or in-hospital clinical experience as doctors, nurses, or paramedics. Using structured guidelines, raters classified the chest compression pattern as CCC, 30:2, or indeterminate (Table 1). The guidelines for manual classification were developed by the lead study investigators and approved by the cardiac workgroup of the Consortium.

Table 1.

Guidelines for Manual Classification of Chest Compression Patterns

30:2 Chest Compressions

Continuous Chest Compressions

-For the first 8 minutes of resuscitation. -For at least 60% of the available and analyzable CPR chest compression image. -There are chest compression periods of at least 20 seconds duration interspersed with regular chest compression pauses (presumptively for two ventilations). -The chest compression pauses occur in a periodic fashion. -There are at least three chest compression pauses per cycle of CPR. A "cycle" of CPR refers to an approximately 2-minute period of compressions performed preceding a rhythm analysis.

-For the first 8 minutes of resuscitation. -For at least 60% of the available and analyzable CPR chest compression image. -There are chest compressions periods of at least 90 seconds duration without regular chest compression pauses. -Any observed chest compression pauses during a CPR cycle are brief and do not occur in a periodic fashion. -There are fewer than three chest compression pauses per cycle of CPR, where cycle of CPR is defined as above.

Patterns fulfilling neither (or both) criteria were classified as indeterminate.

CPR = cardiopulmonary resuscitation.

Pilot efforts suggested that CCC may be characterized by: 1) the presence of approximately 2-minute periods of uninterrupted chest compressions without pauses; 2) the presence of only brief, nonperiodic chest compression pauses; and 3) the presence of no more than two chest compression pauses per 2-minute cycle of CPR. Similarly, 30:2 appeared to be characterized by: 1) the presence of approximately 20- to 30-second segments of chest compression interspersed with regular deliberate pauses and 2) the presence of at least three chest compression pauses per 2-minute cycle of chest compressions. Preliminary efforts also confirmed that the chest compression pattern in select cases may be indeterminate; for example, if an EMS crew started with 30:2 chest compressions but switched to CCC.

Automated Classification of Chest Compression Patterns

Research coordinators identified the start and end clock times for each chest compression segment appearing on the CPR process file. (Data Supplement S2, available as supporting information in the online version of this paper) Using these parameters, one of the authors (RHS) developed automated computer algorithms identifying mean chest compression fraction, median chest compression time segment length, and mean number of chest compression interruptions (Table 2). The algorithms defined chest compression fraction as the number of seconds with chest compressions, divided by the number of seconds of analyzable data. The program defined chest compression time segment length as the duration of each section of uninterrupted chest compressions, with interruptions defined as chest compression pause greater than 2 seconds.

Table 2.

Criteria Used for Automated Classification of Chest Compression Patterns

Measure	Criteria for Continuous Chest Compressions	Criteria for 30:2 Chest Compressions
Mean chest compression fraction	>0.80	0.60–0.80
Median chest compression segment length (seconds)	60–150	<20
Mean chest compression pauses (n per minute)	<1	2–4

Chest compression patterns were classified as continuous or 30:2 chest compressions it the pattern fulfilled two of the three criteria. Patterns fulfilling neither (or both) criteria were classified as indeterminate.

The automated analysis included available CPR data for the first 8 minutes of resuscitation, excluding time epochs with missing or unanalyzable data. The automated analysis required fulfillment of two of three criteria for classification as 30:2 or CCC (Table 2). If neither criteria were fulfilled, the program classified the case as “indeterminate.” (A detailed description of and rationale for the automated classification criteria are provided in Data Supplement S3, available as supporting information in the online version of this paper.).

Data Analysis

We determined inter-rater agreement between manual classifications using Fleiss’s kappa for multiple raters.¹⁸ Using the Stata module “kapci,” we applied bootstrapping with 100 repetitions to determine the 95% CI for the kappa value. Because of their varying graphical representations of CPR process, on a post hoc basis we repeated the comparisons stratified by cardiac monitor manufacturer.

To determine the agreement between automated and manual CPR patterns classifications, we first defined the criterion standard manual classification as the chest compression pattern provided by the majority (at least five of eight) of human ratings. We determined inter-rater agreement between automated and criterion standard manual classifications using Cohen’s kappa. We similarly determined the 95% CIs by using bootstrapping with 100 repetitions. We also repeated the comparisons stratified by cardiac monitor manufacturer.

The use of three classification categories (CCC, 30:2, indeterminate) may have biased the analysis toward lower kappa values. Therefore, in a sensitivity analysis, we evaluated the potential range of kappa values with indeterminate manual ratings reclassified as CCC or 30:2. All analyses were conducted using Stata v.12.2 and R v.2.15.1.

RESULTS

Based on the majority of classifications for each case, the eight raters classified 12 CPR patterns as 30:2, 12 as CCC, and six as indeterminate. CPR pattern classification was unanimous for 15 cases, including 30:2 (n = 7) and CCC (n = 8; Table 3). Inter-rater agreement between manual classifications of chest compression patterns was κ = 0.62 (95% CI = 0.49 to 0.74; Table 4). Inter-rater agreement was similar when stratified by cardiac monitor manufacturer.

Table 3.

Summary of Chest Compression Pattern Classifications

		Manual (Human) Classification
Case	Cardiac Monitor Manufacturer	Rater 1	Rater 2	Rater 3	Rater 4	Rater 5	Rater 6	Rater 7	Rater 8	Unanimous Agreement	Criterion Standard (Majority Agreement)	Automated Classification
1	Zoll	CCC	CCC	CCC	CCC	CCC	CCC	CCC	CCC	CCC	CCC	CCC
2	Philips	CCC	CCC	CCC	CCC	CCC	CCC	CCC	Indet		CCC	CCC
3	Physio-Control	30:2	30:2	30:2	30:2	30:2	30:2	30:2	30:2	30:2	30:2	30:2
4	Physio-Control	CCC	CCC	CCC	CCC	CCC	CCC	CCC	CCC	CCC	CCC	CCC
5	Physio-Control	30:2	30:2	30:2	30:2	30:2	30:2	30:2	30:2	30:2	30:2	30:2
6	Physio-Control	Indet	Indet	CCC	Indet	Indet	Indet	Indet	30:2		Indet	Indet
7	Zoll	30:2	30:2	30:2	Indet	30:2	30:2	30:2	30:2		30:2	30:2
8	Physio-Control	Indet	CCC	CCC	CCC	CCC	CCC	CCC	CCC		CCC	CCC
9	Physio-Control	Indet	CCC	CCC	Indet	Indet	30:2	30:2	CCC		CCC	CCC
10	Philips	30:2	30:2	30:2	30:2	30:2	30:2	30:2	30:2	30:2	30:2	30:2
11	Zoll	CCC	CCC	CCC	CCC	CCC	CCC	CCC	CCC	CCC	CCC	CCC
12	Physio-Control	Indet	Indet	Indet	Indet	Indet	30:2	30:2	30:2		Indet	30:2
13	Zoll	CCC	CCC	CCC	CCC	CCC	CCC	CCC	CCC	CCC	CCC	CCC
14	Philips	30:2	30:2	30:2	Indet	30:2	30:2	Indet	30:2		30:2	30:2
15	Zoll	Indet	Indet	CCC	Indet	Indet	CCC	Indet	CCC		Indet	Indet
16	Physio-Control	CCC	CCC	CCC	CCC	CCC	CCC	CCC	CCC	CCC	CCC	CCC
17	Zoll	30:2	30:2	CCC	Indet	30:2	30:2	30:2	30:2		30:2	30:2
18	Physio-Control	Indet	CCC	CCC	CCC	CCC	CCC	CCC	CCC		CCC	CCC
19	Philips	CCC	Indet	Indet	Indet	Indet	30:2	Indet	30:2		Indet	30:2
20	Physio-Control	30:2	30:2	30:2	30:2	30:2	30:2	30:2	30:2	30:2	30:2	30:2
21	Zoll	CCC	CCC	CCC	CCC	CCC	CCC	CCC	CCC	CCC	CCClo	CCC
22	Physio-Control	Indet	CCC	CCC	Indet	Indet	Indet	Indet	30:2		Indet	Indet
23	Zoll	Indet	30:2	Indet	Indet	Indet	CCC	Indet	CCC		Indet	30:2
24	Philips	CCC	CCC	CCC	CCC	CCC	CCC	CCC	CCC	CCC	CCC	CCC
25	Physio-Control	30:2	30:2	30:2	30:2	30:2	30:2	30:2	30:2	30:2	30:2	30:2
26	Zoll	CCC	CCC	CCC	CCC	CCC	CCC	CCC	CCC	CCC	CCC	CCC
27	Physio-Control	30:2	30:2	30:2	Indet	Indet	30:2	30:2	30:2		30:2	30:2
28	Philips	30:2	30:2	30:2	30:2	CCC	30:2	30:2	30:2		30:2	30:2
29	Zoll	Indet	30:2	30:2	Indet	30:2	30:2	Indet	30:2		30:2	30:2
30	Zoll	30:2	30:2	30:2	30:2	30:2	30:2	30:2	30:2	30:2	30:2	30:2

Each chest compression case was classified as CCC, 30:2 chest compressions (30:2), or indeterminate (Indet). Manual (human) classifications were provided by raters from each of eight regional coordinating centers. Guidelines for manual classification are listed in Table 1. Unanimous agreement reflects complete agreement among the manual classifications. The criterion standard rating reflects majority agreement among the manual classifications. Automated chest compression pattern classifications were determined by computer analysis of chest compression process files using rules summarized in Table 1.

CCC = continuous chest compressions.

Table 4.

Inter-rater Agreement of Chest Compression Pattern Classifications

	Cardiac Monitor Manufacturer
Inter-rater Agreement	Physio-Control (n = 13)	Philips (n = 6)	Zoll (n = 11)	All Cases
Agreement between manual classifications, kappa  (95% CI)	0.61 (0.36–0.78)	0.59 (0.29–0.91)	0.64 (0.44–0.83)	0.62 (0.49–0.74)
Agreement between [criterion standard manual  classification] and [automated classification],  kappa (95% CI)	0.88 (0.65–1.00)	0.70 (0.00–1.00)*	0.85 (0.60–1.00)	0.84 (0.59–0.95)

CIs determined by bootstrapping with 100 repetitions.

^*The bootstrapped 95% CI is wide because of small sample size despite near complete concordance (only one disagreement among six cases) between criterion standard manual and automated classification.

The automated algorithm classified 15 chest compression patterns as 30:2, 12 as CCC, and three as indeterminate (Table 3). Classification agreement between automated and criterion standard manual classifications was κ = 0.84 (95% CI = 0.59 to 0.95). There was disagreement in three of 30 cases; the pattern of discordance in all three cases was [criterion standard manual = indeterminate vs. automated = 30:2]. The kappa measure of agreement between automated and criterion standard manual ratings was lowest for Philips CPR process files, despite near-perfect concordance in ratings (there was disagreement in only one of six cases), possibly due to the small sample size (Table 4).

In a sensitivity analysis, when reclassifying all indeterminate ratings as CCC, the kappa measure of inter-rater agreement between manual classifications was 0.69 (95% CI = 0.56 to 0.81). When reclassifying indeterminate ratings as 30:2, kappa was 0.75 (95% CI = 0.59 to 0.86).

DISCUSSION

In this study we observed good inter-rater agreement in the manual classification of CPR chest compression patterns, a finding that sets the foundation for use of CPR process data to characterize chest compression strategies.¹⁹ More importantly, we also observed strong agreement between manual and automated classifications, a finding that supports the viability of automated approaches to chest compression pattern identification and categorization.

Our findings have important implications for CPR research. The use of CPR detection technology provides a more rigorous approach to identifying and characterizing chest compressions delivered during clinical care. Prior studies have relied on self-reports for characterizing different CPR chest compression strategies. For example, in the comparison of minimally interrupted with traditional 30:2 CPR by Bobrow et al.,⁷ the type of CPR delivered by EMS personnel was indicated by paramedic reports, without supporting CPR process data. No prior studies have independently confirmed the application of nor adherence to an intended CPR pattern strategy. The identification of CPR patterns is particularly important in an interventional trial (such as the ROC CCC Trial) where it is essential to determine both the intended and the actual CPR treatment received by a subject.

While inter-rater agreement was good, the raters did not show unanimous agreement in 15 cases; in the majority of these cases, the disagreement pattern was either [CCC vs. indeterminate] or [30:2 vs. indeterminate], and in many instances there was only one dissenting rating. Furthermore, the majority of raters classified the CPR pattern as “indeterminate” in 20%; raters indicated that in most of these cases the intended chest compression patterns were simply not discernable. These observations are not unexpected given the complexity of the CPR process images and the natural chest compression variations that may occur from personnel changes, efforts to move the patient, manipulations of the airway, or rescuer fatigue.⁴ The inclusion of an “indeterminate” chest compression category in this analysis was both necessary and appropriate, as we expected natural variation in CPR delivery. While we provided structured guidelines for chest compression pattern classification, the reviewers affirmed that their exact interpretation likely varied.

Perhaps the most important finding of this study is the agreement of automated with manual chest compression classifications. Manual CPR pattern review and classification is time-intensive and arduous; raters in this study reported spending between 5 to 60 minutes to assess and assign a rating to each case. An automated approach may lend efficiencies to this process, bringing clear benefits to scientific and clinical applications. While our automated analysis was based on information that was manually extracted (the time and duration of each CPR segment), we surmise that computer analysis could also automate the latter task. We suspect that the feasibility of computer classification analysis likely depends on the complexity of a given chest compression strategy, and thus independent validation would be appropriate prior to application to a new CPR strategy.

In post hoc analyses, we found that agreement between manual and automated classifications was lower for the Philips CPR process files, an observation that was likely due to the small number of Philips CPR files in the series. However, the raters did comment on graphical distinctions between the CPR process reports that may have influenced chest compression pattern classifications. For example, the Physio-Control device uses bar graphs to depict each discrete compression, while the Zoll and Philips devices use line graphs depicting the vertical displacement of the chest compression sensor (Figure 1 and Data Supplement S1). The time resolution of the CPR reports also varies, with each line of the Physio-Control and Zoll graphical output spanning approximately 15 seconds and each line of the Philips output spanning 2 minutes. Unlike the Philips software, the Physio-Control and Zoll software also offer per-minute summaries of chest compression metrics. Additional study must identify the graphical characteristics most conducive to CPR pattern analysis.

Our study also highlights unexplored strategies for characterizing chest compressions. Prior efforts have focused on individual dimensions of CPR process such as the number and duration of chest compression interruptions or chest compression depth or rate.^2-5,20,21 In the automated algorithm, we classified chest compression patterns using combinations of chest compression fraction, segment length, and number of interruptions. There may be other individual or combinations of metrics that better characterize the overall pattern of chest compressions and correlate with clinical outcomes. These measures might also improve our understanding of the physiologic mechanisms linking CPR strategy to improved outcomes. Additional study must explore these important unanswered questions.

LIMITATIONS

We used as a convenience sample of CPR process images by study team consensus rather than by random selection by site or cardiac monitor manufacturer, and thus selection bias may have influenced the observed results. To minimize subject burden, we did not test intra-rater agreement, which would have required repeat ratings of an additional 10 to 15 chest compression cases. We did not evaluate inter-rater agreement of other chest compression metrics such as compression depth, rate, or chest compression fraction. As suggested in the sensitivity analysis, a smaller number of indeterminate cases would have increased the observed kappa. We did not use audio recordings of the encounters.

In the evaluation of the automated chest compression algorithm, we designated criterion standard classifications by using the ratings assigned by the majority of manual ratings, which may have resulted in misclassification in a small number of cases. Given the complexity of classifying chest compression patterns, we opted not to designate criterion standard by study team consensus. Although we observed good inter-rater agreement estimates, the bootstrapped CIs suggest that a portion of the estimates may have been lower than observed.

Given the absence of prior data, we developed definitions for manual and automated classifications based on study team consensus. The development of different metrics and approaches to chest compression pattern classification is possible, but was outside the scope of this analysis. Additional study must explore and identify additional strategies for chest compression pattern classification.

This analysis used chest compression data from out-of-hospital cardiac arrests treated by expert EMS agencies specifically trained in the ROC research protocols. The clarity of chest compression patterns may differ with other EMS agencies or practitioners. Also, the raters in this study possessed extensive experience with CPR process analysis. We evaluated classification agreement for two specific treatment algorithms; the potential inter-rater agreement for alternate chest compression strategies is unclear.

CONCLUSIONS

In this study human raters showed good inter-rater agreement in the classification of cardiopulmonary resuscitation chest compression patterns. Automated classification showed strong agreement with human ratings. These observations support the consistency of manual cardiopulmonary resuscitation pattern classification as well as the use of automated approaches to chest compression pattern analysis.