Diagnostic performance of point-of-use ultrasound of resuscitation outcomes: A systematic review and meta-analysis of 3265 patients

Abstract

Background

Echocardiography in the setting of resuscitation can provide information as to the cause of the cardiac arrest, as well as indicators of futility. This systematic review and meta-analysis were performed to determine the value of point-of-care ultrasonography (PoCUS) in the assessment of survival for adult patients with cardiac arrest.

Methods

This meta-analysis was performed in adherence to the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines. PubMed, EMBASE, Web of Science, Cochrane have been searched from databases inception until March 2^nd 2021. The search was limited to adult patients with cardiac arrest and without publication dates or country restrictions. Papers were chosen if they met the required criteria relating to the sensitivity, specificity, accuracy, positive predictive value, and negative predictive value of this diagnostic technique concerning resuscitation outcomes.

Results

This systematic review identified 20 studies. Overall, for survival to hospital discharge, PoCUS was 6.2% sensitivity (95% confidence interval [CI] 4.7–8.0%) and 2.1% specific (95% CI 0.8–4.2%). PoCUS sensitivity and specificity for return of spontaneous circulation were 23.8% (95% CI 21.4–26.4%) and 50.7% (95% CI 45.8–55.7%) respectively, and for survival to admission 13.8% (95% CI 12.2–15.5%) and 20.1% (95% CI 16.2–24.3%), respectively.

Conclusions

The results do not allow unambiguous recommendation of PoCUS as a predictor of resuscitation outcomes and further studies based on a large number of patients with full standardization of operators, their training and procedures performed were necessary.

Introduction

Echocardiography in the setting of cardiopulmonary resuscitation (CPR) can provide information as to the cause of the sudden cardiac arrest (SCA), as well as indicators of futility [1–3]. In the first application, echocardiography can identify potentially reversible causes of arrest. This procedure is performed by use of the subcostal, apical and parasternal views to identify cardiac tamponade, findings suggestive of pulmonary embolism, and a pleural view to identify pneumothorax [4–6]. The diagnosis of pulmonary embolism may be challenging, as findings of isolated right ventricular dilation must be considered with caution. Right ventricular dilation occurs within minutes of cardiac arrest, as blood shifts from the systemic circulation to the right heart along its pressure gradient [7, 8], which has been uniformly reported in porcine models of cardiac arrest, regardless of causes that included hypovolemia, hyperkalemia, and primary arrhythmia [9, 10].

The second most useful application of echocardiography during CPR is that of determining the probability of a successful resuscitation. In this situation echocardiography is used to identify spontaneous cardiac movement [11, 12]. The meta-analysis conducted by Tsou et al. [11] has revealed that spontaneous cardiac movement possesses a sensitivity and specificity of 95% and 80% (95% confidence interval [CI] 0.72–0.99, and 0.63–0.91, respectively) for predicting return of spontaneous circulation (ROSC) during cardiac arrest, with a positive and negative likelihood ratio of 4.8 and 0.06 (95% CI 2.5–9.4, and 0.01–0.39, respectively). Despite these findings, the 2020 International Consensus on Cardiopulmonary Resuscitation and Emergency Cardiovascular Care Science with Treatment Recommendations [13] suggested against the use of point-of-care ultrasound (PoCUS) for prognostication during CPR. This occurred because the overall certainty of evidence was rated as very low for all outcomes, primarily due to the risk of bias, inconsistency, and/or imprecision.

Finally, despite the suggested somewhat controversial value of echocardiography in the course of resuscitation, clinicians may have reservations to adopt because of interference with cardiac compressions leading to ineffectual CPR. To address this, some [14] have suggested protocols to perform echocardiography simultaneously with the rhythm check, thus minimizing interruptions to chest compressions. Whether or not a practitioner decides to use echocardiography during a resuscitation, ultimately its use is subject to availability of equipment and skilled operators. Although rapidly becoming more available, the pre-hospital application of PoCUS is still commonly limited.

Because of these concerns and challenges, we sought to evaluate the current status of PoCUS as an aid to diagnosis and a determinate of prognosis during resuscitation. The purpose was to review and appraise the diagnostic accuracy of echocardiography in CPR. A priori subgroup analyses of the primary outcome were planned for location: out-of-hospital cardiac arrest (OHCA) versus in-hospital cardiac arrest (IHCA).

Methods

This systematic review and meta-analysis were performed in accordance with the Preferred Reporting Items for Systematic reviews and Meta-Analysis (PRISMA) statement [15].

Data sources and searches

Two authors independently (M.D. and H.K.) conducted a search of PubMed, Scopus, EMBASE, Web of Science, and the Cochrane Central Register and Controlled Trials (CENTRAL) database from inception to March 2^nd, 2021, with the following search strategy: “out-of-hospital cardiac arrest” OR “OHCA” OR “in-hospital cardiac arrest” OR “IHCA” OR “cardiac arrest” OR “heart arrest” OR “heart attack” OR “advanced life support” OR “resuscitation” OR “CPR” AND “cardiac ultrasound” OR “cardiac ultrasonography” OR “heart ultrasound” OR “heart ultrasonography” OR “echocardiography” OR “ultrasound imaging” OR “USG” OR “US” OR “ultrasound” OR “ultrasonographic” OR “echocardiogram” OR “point-of-care ultrasonography” OR “PoCUS” OR point-of-care echocardiography”. Only studies published in English were included in the meta-analysis. However, the search was limited to human studies without publication date, or country restrictions. Gray literature repositories such as Google Scholar were also searched. Finally, further reviewed references to echocardiography in eligible articles and systematic reviews were manually retrieved.

Selection criteria

Two investigators (M.D. and H.K.) evaluated independently all relevant studies for eligibility criteria and pooled analysis. Disagreements between the authors regarding values or analysis assignments were resolved through discussion with a third researcher (L.S.), and the decision was taken by the majority of the researchers. Raw data were extracted by using a standardized, premade form. Care was taken to avoid inclusion of data from duplicate publications. In any case of suspected data discrepancies, the relevant author was contacted directly.

Studies included in this meta-analysis fulfilled the following criteria (PICOS): (P) Population: adults with cardiac arrest; (I) Intervention: point-of-care echocardiography during CPR; (C) Comparator: absence of that finding or a different finding on point-of-care echocardiography during resuscitation procedure; (O) Outcome: prognosticate clinical outcome: ROSC, survival to hospital admission or survival to hospital discharge; (S) Study design: randomized controlled trials (RCTs) or non-RCTs.

Studies that enrolled children and animal studies were excluded. Case reports, case series, guidelines, review articles; consensus statements, editorials, letters, conference abstracts, studies not pertaining to the field of inters or studies with insufficient data for reconstruction 2 × 2 table were also excluded.

Data extraction

Data were independently extracted by two reviewers (M.D. and L.S.) and were verified by a third reviewer (L.S.). The following data categories were extracted from the included studies: study population characteristics, operator type, ultrasonography window type; initial rhythm, resuscitation outcomes including ROSC, survival to hospital admission or survival to hospital discharge. True positive, false positive, false negative, and true negative numbers of echocardiography were obtained. The results were summarized in 2-by-2 contingency tables.

Quality assessment

Two investigators (A.G. and H.K.) independently extracted individual study data and evaluated studies for risk of bias. Any disagreements were discussed and resolved in a consensus meeting with the third reviewer (M.M.). The ROBINS-I tool (tool to assess risk of bias in non-randomized studies of interventions) was used to assess the quality of non-randomized trials [16] and the RoB 2 tool (revised tool for risk of bias in randomized trials) was used to assess the quality of randomized studies [17]. The Robvis application was used to visualize risk of bias assessments [18]. The scale has seven main domains (confounding, participant selection, classification of interventions, deviation from interventions, missing data, outcome measurement, and selection of reported results) and assigns one point for each of the following four judgements: critical, serious, moderate, and low. The review authors’ judgments about each risk of bias items are provided in Supplementary Figures S1 and S2.

Outcomes

The primary outcome of the current meta-analysis was survival to hospital discharge or 30-day survival. The secondary outcomes were return of spontaneous circulation or survival to hospital admission in case of OHCA.

Data analysis

All statistical analyses were performed using STATA and Review Manager Software 5.4 (The Cochrane Collaboration, Oxford, Copenhagen, Denmark). Sensitivity, specificity, positive likelihood ratio (+LR) and negative LR (−LR), and the log diagnostic odds ratio, including the 95% CI, were calculated. The primary objective was to estimate pooled measurements of diagnostic accuracy: pooled sensitivity and specificity using the Mantel-Haenszel odds ratios [19], and pooled positive and negative LR using the DerSimonian-Laird method [20]. An overall area under the receiver-operating-characteristic (ROC) curve was also calculated. Heterogeneity was assessed using the Cochran Q-statistic (p less than 0.05 indicated the presence of heterogeneity) and the inconsistency (I²) test [21]. According to Higgins a p-value of < 0.10 or I² statistic of > 50% indicated substantial statistical heterogeneity [21]. Subgroup sensitivity analyses were also conducted to determine the robustness of findings. Forest plots were prepared and performed with Review Manager Software 5.4 (The Cochrane Collaboration, Oxford, Copenhagen, Denmark).

Results

Search results

Figure 1 shows a flow diagram summarizing the literature search. A total of 6217 studies were identified during the initial search. After removing 1115 duplicates, 5102 titles and abstracts were reviewed and then 5060 studies were excluded. After reviewing the full text of 47 eligible articles, finally 20 original research articles [22–41] including a total of 3265 patients were included in this meta-analysis. Study types included prospective cohort design, retrospective and case-control studies. The publication dates of these studies ranged from 2001 to 2019. Six studies were conducted in the United States [25, 30, 37–40], 3 in Turkey [27, 36, 41], 2 in Canada [22, 24], and 1 in each of the following countries: Austria [23], Germany [26], Iran [28], Singapore [29], Brazil [31], United Kingdom [33], Republic of Korea [34], and Taiwan [35]. One study was also conducted as a multi-country trial.

Figure 1.

Meta-analysis flow chart of included and excluded studies.

Characteristics and quality assessment of included studies

Table 1 lists the study and population characteristics. The number of patients in the study ranged from 20 to 793. Mean age ranged from 48.6 to 71.1 years. The study’s ultrasonographic characteristics, as well as methodology characteristics, are summarized in Supplementary Table S1 and S2. Results of the quality assessment of studies are summarized in Supplementary Figures S1 and S2.

Table 1.

Characteristics of the included studies.

Study	Country	Study desing	Cardiac arrest setting	No. of patients	Age (mean ± SD)	Inithial rhythm
Aichinger et al. 2012	Austria	Prospective observational study	OHCA	42	70.3 ± 2.4	VF/pVT: 11 AS: 20 PEA: 11
Atkinson et al. 2019	Canada	Retrospective study	IHCA	223	65.3 ± 4.2	NS
Backett et al. 2019	Canada	Retrospective study	OHCA	180	65.27 ± 15.02	AS: 135 PEA: 45
Blaivas et al. 2001	USA	Prospective observational study	OHCA	173	71.1 ± 2.7	VF/pVT: 66 AS: 65 PEA: 38
Breitkreutz et al. 2010	Germany	Prospective observational study	OHCA	100	65 ± 19	VF/pVT: 24 AS:38 PEA: 22
Cebicci et al. 2014	Turkey	Retrospective study	IHCA and OHCA	410	63.2 ± 20.7	VF/pVT: 45 AS: 290 PEA: 75
Chardoli et al. 2012	Iran	Prospective interventional study	IHCA	100	58 ± 6.1	PEA: 100
Chua et al. 2017	Singapore	Prospective study	OHCA	104	69.3 ± 7.2	VF/pVT: 17 AS: 47 PEA: 33
Cureton et al. 2012	USA	Retrospective study	OHCA	318	NS	NS
Flato et al. 2015	Brazil	Prospective, observational cohort study	IHCA	49	60.0 ± 17.6	AS: 17 PEA: 32
Gaspari et al. 2016	USA/Canada	Non-randomized, prospective, protocol-driven observational study	IHCA and OHCA	793	64.2 ± 17.4	AS: 379 PEA: 414
Hayhurst et al. 2011	UK	Retrospective study	IHCA and OHCA	50	NS	VF/pVT: 6 AS: 20 PEA: 23
Kim et al. 2016	Republic of Korea	Prospective clinical observational study	OHCA	48	63.9 ± 14.5	NS
Lien et al. 2018	Taiwan	Prospective observational study	OHCA	177	70.9 ± 14.8	VF/pVT: 31 AS: 82 PEA: 64
Ozen et al. 2016	Turkey	Prospective, single center study	IHCA and OHCA	129	68.96 ± 16.44	VF/pVT: 30 PEA/AS: 20
Salen et al. 2001	USA	Prospective clinical observation study	IHCA and OHCA	102	NS	VF/pVT: 11 AS: 36 PEA: 55
Salen et al. 2005	USA	Prospective clinical observation study	IHCA and OHCA	70	NS	AS: 36 PEA: 34
Schuster et al. 2009	USA	Retrospective study	IHCA and OHCA	28	48.6 ± 20.2	PEA: 28
Tayal et al. 2003	USA	Observational, prospective series	OHCA	20	57 ± 15	NS
Tomruk et al. 2012	Turkey	Prospective follow-up study	IHCA and OHCA	149	61.6 ± 17.9	VF/pVT: 8 AS: 77 PEA: 64

AS — asystole; IHCA — in-hospital cardiac arrest; NS — not specified; OHCA — out-of-hospital cardiac arrest; PEA — pulseless electrical activity; pVT — pulseless ventricular tachycardia; SD — standard deviation; VF — ventricular fibrillation

Primary outcome

Survival to hospital discharge was reported in 6 studies [22–24, 31, 32, 36], in which sensitivity values ranged from 0.01 to 0.46 and specificity values ranged from 0.00 to 0.05 (Fig. 2). The summary sensitivity and specificity values were 0.062 (0.047, 0.078) and 0.021 (0.008, 0.042), respectively. The Q test revealed significant heterogeneity (Q = 24.54; p < 0.001), with substantial heterogeneity detected for sensitivity (I² = 97.5%; p < 0.001) and specificity (I² = 30.9%; p = 0.215). The area under the hierarchic summary ROC curve was 0.0112.

Figure 2.

Forrest plot of the overall sensitivity and specificity of echocardiography for predicting survival to hospital discharge after cardiac arrest; CI — confidence interval; TP — true positive; FP — false positive; FN — false negative; TN — true negative.

Subgroup analysis showed that sensitivity and specificity of echocardiographic test related to survival to hospital discharge in the OHCA group was: 0.041 (95% CI 0.018–0.080) and 0.024 (95% CI 0.001–0.129), respectively (Suppl. Table S3).

Secondary outcomes

The 9 studies reported return of spontaneous circulation [22, 24, 28, 31, 32, 34, 36, 41], in which pooled results of sensitivity and specificity were 0.238 (95% CI 0.214–0.264) and 0.507 (95% CI 0.458–0.557), respectively (Fig. 3). The Q test revealed significant heterogeneity (Q = 35.779; p < 0.001), with substantial heterogeneity detected for sensitivity (I² = 85.5%; p < 0.001) and specificity (I² = 93.8%; p < 0.001). The area under the hierarchic summary ROC curve was 0.3296.

Figure 3.

Forrest plot of the overall sensitivity and specificity of echocardiography for predicting the return of spontaneous circulation after cardiac arrest; CI — confidence interval; TP — true positive; FP — false positive; FN — false negative; TN — true negative.

Use of PoCUS in cases of OHCA arrest was associated with a study sensitivity of 0.194 (95% CI 0.139–0.260) and specificity 0.735 (95% CI 0.589–0.851). Sensitivity and specificity of PoCUS in the case of IHCA was 0.166 (95% CI 0.118–0.233) and −0.554 (95% CI 0.425–0.677), respectively (Suppl. Table S3).

Fourteen studies reported survival to hospital admission [22–27, 29, 30, 32, 33, 37–40], in which sensitivity values ranged from 0.04 to 0.53 and specificity values that ranged from 0.00 to 0.44 (Fig. 4). The summary sensitivity and specificity values were 0.138 (95% CI 0.122–0.155) and 0.201 (95% CI 0.162–0.243), respectively. The Q test revealed significant heterogeneity (Q = 17.74; p < 0.001), with substantial heterogeneity detected for sensitivity (I² = 91.2%; p < 0.001) and specificity (I² = 63.5%; p = 0.003). The area under the ROC curve indicated low accuracy 0.1031. PoCUS sensitivity and specificity for survival to admission after OHCA were 0.121 (95% CI 0.093–0.154) and 0.261 (95% CI 0.173–0.366), respectively.

Figure 4.

Forrest plot of the overall sensitivity and specificity of echocardiography for survival to hospital admission after cardiac arrest; CI — confidence interval; TP — true positive; FP — false positive; FN — false negative; TN — true negative.

Discussion

The present research analysis included studies that met the inclusion criteria. These studies included both OHCA and IHCA, and the patient population was varied. The current analysis included both retrospective studies, prospective clinical observational studies as well as observational and prospective series. Electrocardiographic findings in the analyzed cases were predominantly pulseless electrical activity and asystole.

Cardiac arrest, despite the development of therapeutic methods, is still a condition associated with a very high mortality rate [42–47]. One of the basic tasks during CPR is to shorten the resuscitation period and provide the fastest possible ROSC [48]. International guidelines recommend identifying and treating potentially reversible causes of SCA as soon as possible. With the lapse of time, the chances of victim survival decreases, whereas, with prolonged resuscitation, deterioration of a patient’s neurological prognosis and fatigue of the medical personnel performing resuscitation, with possible deterioration in the quality of basic parameters related to chest compression are important factors [49, 50].

Ultrasonography has been used in intensive care for many years. Over the decades there has been an expansion in the use of ultrasound, the use of this method not only in diagnostic rooms but directly on the ward and even in the pre-hospital settings. Increasingly more physicians, as well as other medical staff, are trained and get experience in diagnosing life-threatening pathologies and the use of ultrasound equipment, which is widely available.

Ultrasound has many potential applications in CPR, ranging from the technical enhancement of resuscitation (correcting the correct position of the rescuer’s hands) [51] to facilitating the identification of the correct cause of cardiac arrest [52], which is critical for further management. There are doubts about the prognostic potential of this technique for CPR; the results of studies in this aspect remain inconclusive. Ultrasonography can identify some potentially reversible causes of SCA, as well as assess myocardial contractility or the absence of any systolic activity of the heart. The complete absence of cardiac systolic activity is a poor prognostic factor for SCA.

The diagnostic aspect of ultrasonography in CPR is particularly relevant to selected special situations such as cardiac tamponade, pulmonary embolism, myocardial infarction, aortic dissection or rupture, hypovolemia, tension pneumothorax, papillary muscle rupture, it also enables the distinction of true asystole from ventricular fibrillation when there is doubt concerning the assessment of heart rhythm (e.g. artifacts, electrocardiogram muscle tremor) together with the assessment of myocardial contractile activity [32, 53]. Studies on the use of ultrasonography indicate that 10–35% of patients with asystole demonstrate myocardial contractile activity.

Ultrasonography also allows the identification of arterial flow when there is doubt about the presence of a pulse on large arteries according to international guidelines. However, it is important to note that the presence of a pulse on large arteries is a prerequisite for the clinical exclusion of SCA — confirmation of the presence of a pulse providing minimal perfusion for survival with a good neurological prognosis [54].

A unique feature of this technique that is of great practical importance is the ability to recognize pseudo-pulseless electrical activity. PoCUS is of particular importance in cases of reversible causes of cardiac arrest; in such cases, it is also an ideal tool for performing emergency procedures under ultrasound guidance (e.g. cardiac tamponade decompression). Due to the advantages of the method, some authors consider the introduction of training also e.g. internal medicine physicians. An interesting development of the PoCUS technique is the use of both transthoracic (TTE) and transesophageal (TEE) echocardiograms. The advantage of this technique is that it can be used in real-time, continuously, and without interfering with ongoing resuscitation (TEE) or minimally interfering with ongoing CPR (TTE) [55].

A problem with the use of ultrasonography in SCA can be a lack of experience of the person performing the assessment and the interruption of resuscitation. With good training and coordination, this time can be reduced to a minimum [56]. Papers were published on the use of transesophageal ultrasonography, which allows continuous ultrasonographic assessment without the need for interruption of resuscitation efforts, although the accuracy of this assessment without interruptions in chest compressions has been questioned [57].

Limitations of the study

There are several limitations in this study. First, the total number of studies in our analyses was small; however, this may be offset by the moderate-to-large number of included patients (n = 3265). Fifth, included studies did not assess all lung regions, as some patients were bedridden and posterior zones were difficult to be assessed. Moderate heterogeneity was found among the included studies, which was a result of differences in the study setting. Another limitation is the variety of staff preparation for ultrasound testing, as well as different medical staff performing PoCUS, starting with paramedics, through emergency physicians, emergency residents, and ending with the surgeons. Another limitation was the fact that not all studies reported onour primary outcome, which was survival to hospital discharge.

Conclusions

The results do not allow unambiguous recommendation of PoCUS as a predictor of resuscitation outcomes and further studies based on a large number of patients with full standardization of operators, their training and procedures performed are necessary.