Termination of Resuscitation Rules and Survival Among Patients With Out-of-Hospital Cardiac Arrest: A Systematic Review and Meta-Analysis

Michael A Smyth; Imogen Gunson; Alison Coppola; Samantha Johnson; Robert Greif; Kasper G Lauridsen; Sian Taylor-Philips; Gavin D Perkins

doi:10.1001/jamanetworkopen.2024.20040

. 2024 Jul 3;7(7):e2420040. doi: 10.1001/jamanetworkopen.2024.20040

Termination of Resuscitation Rules and Survival Among Patients With Out-of-Hospital Cardiac Arrest

A Systematic Review and Meta-Analysis

Michael A Smyth ^1,^2,^✉, Imogen Gunson ³, Alison Coppola ⁴, Samantha Johnson ¹, Robert Greif ^5,⁶, Kasper G Lauridsen ^7,^8,⁹, Sian Taylor-Philips ¹, Gavin D Perkins ^1,¹⁰

¹Medical School, University of Warwick, Coventry, England

²University Hospital Coventry and Warwickshire NHS Trust, Coventry, England

³West Midlands Ambulance Service University NHS Foundation Trust, Brierly Hill, England

⁴University Hospitals Plymouth NHS Trust, Plymouth, England

⁵Department of Anesthesiology and Pain Therapy, Bern University Hospital, Inselspital, University of Bern, Bern, Switzerland

⁶School of Medicine, Sigmund Freud University, Vienna, Austria

⁷Research Center for Emergency Medicine, Aarhus University Hospital, Aarhus, Denmark

⁸Department of Medicine, Randers Regional Hospital, Randers, Denmark

⁹Department of Anesthesiology and Critical Care Medicine, Children’s Hospital of Philadelphia, Philadelphia, Pennsylvania

¹⁰University Hospitals Birmingham NHS Foundation Trust, Birmingham, England

Accepted for Publication: May 2, 2024.

Published: July 3, 2024. doi:10.1001/jamanetworkopen.2024.20040

^✉

Corresponding Author: Michael A. Smyth, PhD, Medical School, University of Warwick, Gibbett Hill Rd, Coventry CV37 8NY, UK (m.a.smyth@warwick.ac.uk).

Author Contributions: Dr Smyth had full access to all of the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis.

Concept and design: Smyth, Lauridsen, Taylor-Philips, Perkins.

Acquisition, analysis, or interpretation of data: All authors.

Drafting of the manuscript: Smyth, Perkins.

Critical review of the manuscript for important intellectual content: All authors.

Statistical analysis: Smyth.

Obtained funding: Smyth, Perkins.

Administrative, technical, or material support: Smyth, Coppola, Johnson, Perkins.

Supervision: Greif, Lauridsen, Perkins.

Conflict of Interest Disclosures: Dr Smyth reported receiving grants from the National Institute for Health and Care Research (NIHR) during the conduct of the study. In addition, Dr Smyth reported serving as a member of the International Liaison Committee on Resuscitation (Basic Life Support Task Force) and the European Resuscitation Committee (Basic Life Support Science and Education Committee) and receiving travel reimbursement from the European Resuscitation Committee outside the submitted work. Dr Perkins reported receiving grants from the NIHR during the conduct of the study. In addition, Dr Perkins reported serving as co-chair of the International Liaison Committee on Resuscitation; receiving funding and travel reimbursement from the Resuscitation Council UK, the European Resuscitation Council, and the British Heart Foundation; and receiving personal or consulting fees from Elsevier and the West Midlands Ambulance NHS University Foundation Trust outside the submitted work. No other disclosures were reported.

Funding/Support: This work was funded by award 17/99/34 from the NIHR Health Service Delivery Research Programme.

Role of the Funder/Sponsor: The funder had no role in the design and conduct of the study; collection, management, analysis, and interpretation of the data; preparation, review, or approval of the manuscript; and decision to submit the manuscript for publication.

Disclaimer: The views expressed are those of the authors and not necessarily those of the NIHR or the Department of Health and Social Care.

Data Sharing Statement: See Supplement 2.

Additional Contributions: We acknowledge the International Liaison Committee on Resuscitation Education, Implementation and Teams Task Force members as collaborators on this systematic review.

^✉

Corresponding author.

PMCID: PMC11222995 PMID: 38958975

Key Points

Question

Can termination of resuscitation (TOR) rules accurately identify patients who will not survive an out-of-hospital cardiac arrest?

Findings

This systematic review and meta-analysis identified 43 studies describing the performance of TOR rules, but evidence concerning the ability of TOR rules to discriminate between those patients who will die and those who will survive was lacking. The available studies provided low-certainty evidence suggesting that the universal termination of resuscitation (UTOR) rule has the best performance; however, even the UTOR rule may not be suitable for use in systems in which transport rates are low and the survival rate is higher than 8%.

Meaning

These findings suggest that there is insufficient robust evidence to support widespread implementation of TOR rules in clinical practice.

This systematic review and meta-analysis examines the use of termination of resuscitation rules to estimate patient survival of out-of-hospital cardiac arrest.

Abstract

Importance

Termination of resuscitation (TOR) rules may help guide prehospital decisions to stop resuscitation, with potential effects on patient outcomes and health resource use. Rules with high sensitivity risk increasing inappropriate transport of nonsurvivors, while rules without excellent specificity risk missed survivors. Further examination of the performance of TOR rules in estimating survival of out-of-hospital cardiac arrest (OHCA) is needed.

Objective

To determine whether TOR rules can accurately identify patients who will not survive an OHCA.

Data Sources

For this systematic review and meta-analysis, the MEDLINE, Embase, CINAHL, Cochrane Library, and Web of Science databases were searched from database inception up to January 11, 2024. There were no restrictions on language, publication date, or time frame of the study.

Study Selection

Two reviewers independently screened records, first by title and abstract and then by full text. Randomized clinical trials, case-control studies, cohort studies, cross-sectional studies, retrospective analyses, and modeling studies were included. Systematic reviews and meta-analyses were reviewed to identify primary studies. Studies predicting outcomes other than death, in-hospital studies, animal studies, and non–peer-reviewed studies were excluded.

Data Extraction and Synthesis

Data were extracted by one reviewer and checked by a second. Two reviewers assessed risk of bias using the Revised Quality Assessment Tool for Diagnostic Accuracy Studies. Cochrane Screening and Diagnostic Tests Methods Group recommendations were followed when conducting a bivariate random-effects meta-analysis. This review followed the Preferred Reporting Items for a Systematic Review and Meta-Analysis of Diagnostic Test Accuracy Studies (PRISMA-DTA) statement and is registered with the International Prospective Register of Systematic Reviews (CRD42019131010).

Main Outcomes and Measures

Sensitivity and specificity tables with 95% CIs and bivariate summary receiver operating characteristic (SROC) curves were produced. Estimates of effects at different prevalence levels were calculated. These estimates were used to evaluate the practical implications of TOR rule use at different prevalence levels.

Results

This review included 43 nonrandomized studies published between 1993 and 2023, addressing 29 TOR rules and involving 1 125 587 cases. Fifteen studies reported the derivation of 20 TOR rules. Thirty-three studies reported external data validations of 17 TOR rules. Seven TOR rules had data to facilitate meta-analysis. One clinical study was identified. The universal termination of resuscitation rule had the best performance, with pooled sensitivity of 0.62 (95% CI, 0.54-0.71), pooled specificity of 0.88 (95% CI, 0.82-0.94), and a diagnostic odds ratio of 20.45 (95% CI, 13.15-31.83).

Conclusions and Relevance

In this review, there was insufficient robust evidence to support widespread implementation of TOR rules in clinical practice. These findings suggest that adoption of TOR rules may lead to missed survivors and increased resource utilization.

Introduction

The incidence of out-of-hospital cardiac arrest (OHCA) in Europe is 67 to 170 per 100 000 inhabitants.¹ Emergency medical services (EMS) personnel attempt resuscitation in 50% to 60% of cases and survival to discharge is 8% (range, 0%-18%).¹ The decision to discontinue resuscitation is challenging. Influencing factors include decisional conflict,^2,3 cardiac arrest location,^2,4 medicolegal concerns,² psychological comfort,⁴ experience,⁴ knowledge of survival outcomes,⁴ and education.⁵ Termination of resuscitation (TOR) rules have been developed to inform decision-making.^6,7,8 These rules have the potential to affect patient outcomes and health resource use. Rules with poor specificity risk premature discontinuation of resuscitation. Rules with high sensitivity increase the number of futile transports and consume valuable health resources.

The International Liaison Committee on Resuscitation identified the evaluation of TOR rules for OHCA as a high priority.⁹ A review of in-hospital TOR rules was published previously.¹⁰ Therefore, this study sought to evaluate the performance of TOR rules in OHCA.

Methods

This systematic review and meta-analysis followed the Preferred Reporting Items for a Systematic Review and Meta-Analysis of Diagnostic Test Accuracy Studies (PRISMA-DTA) statement. The protocol is registered with the International Prospective Register of Systematic Reviews (CRD42019131010).

We followed best practice recommendations for analyzing systematic reviews of diagnostic tests advocated by the Cochrane Screening and Diagnostic Tests Methods Group.¹¹ We utilized test evaluation methods, rather than prognosis analysis methods, because these are easier to understand and there is precedent for using this approach.^12,13,14 The reference standard (died or survived) is a dichotomous outcome that occurs soon after the index test (TOR rule prediction) is applied. Consequently, follow-up time for TOR rules is minimal. Unlike test evaluation methods, prognosis analysis methods include consideration of follow-up time and are therefore less appropriate for analysis of TOR rules.

Eligibility Criteria

We included systematic reviews, meta-analyses, randomized clinical trials, case-control studies, cohort studies, cross-sectional studies, retrospective analyses, and modeling studies. Systematic reviews and meta-analyses were reviewed to identify primary studies. We excluded studies that predicted outcomes other than death or included only post–return of spontaneous circulation (ROSC) populations, non–peer-reviewed studies, in-hospital studies, and animal studies.

Literature Search

We searched the MEDLINE, Embase, CINAHL, Cochrane Library, and Web of Science databases from inception up to January 11, 2024 (eTables 1-6 in Supplement 1). Reference lists were scrutinized, and subject area experts were contacted to identify missed studies. There were no restrictions on language, publication date, or time frame of the study.

Study Selection

Duplicate records were removed. Screening occurred in 2 stages. First, 2 reviewers (I.G. and A.C.) independently reviewed each title and abstract and rated them as “include” or “exclude.” Any record rated as include by either reviewer was considered in stage 2. All other records were rejected as irrelevant. In stage 2, the reviewers (I.G. and A.C.) assessed the full text of the remaining records. Records rated as include by both reviewers were included, whereas those excluded by both reviewers were rejected as irrelevant. Where reviewers disagreed, this was resolved by consulting a third reviewer (M.A.S.).

Data Extraction

Data were extracted using a predefined, piloted data extraction form by 1 reviewer (M.A.S.) and were checked by a second reviewer (either I.G. or A.C.). The data extraction form included study characteristics and contingency tables. If contingency data were not reported, they were calculated from sensitivity, specificity, and prevalence.

Quality Assessment

Risk of bias was independently assessed by 2 reviewers (I.G. and A.C.) using either the ROBINS-I (Risk of Bias in Nonrandomised Studies of Interventions) tool¹⁵ or the Cochrane RoB 2 (Revised Risk of Bias for Randomized Trials) tool¹⁶ and the QUADAS-2 (Revised Quality Assessment Tool for Diagnostic Accuracy Studies) risk of bias and applicability concerns checklist.¹⁷ The Deeks funnel plot asymmetry test was used to identify publication bias. We did not calculate an I² statistic because this is not recommended for systematic reviews of test accuracy.¹⁸ To assess heterogeneity, we assessed the symmetry of the summary receiver operating characteristic (SROC) curve and calculated the correlation coefficient.¹⁹ We adopted the Grading of Recommendations, Assessment, Development and Evaluation (GRADE) Working Group methodology to determine certainty of evidence.²⁰

Outcomes

For outcomes, we adopted Morrison’s¹² recommendation to code death as the true positive. A true positive indicates stop resuscitation and the patient dies, a false positive indicates stop resuscitation but the patient survives (missed survivors), a true negative indicates continue resuscitation and the patient survives, and a false negative indicates continue resuscitation but the patient dies (futile resuscitations).

Statistical Analysis

We analyzed derivation, external validation, and clinical studies separately. Derivation studies use regression methods and develop rules “trained” to the available dataset. External data validation studies evaluate TOR rules in a different dataset to assess generalizability, modeling ideal performance and avoiding the complexities introduced by clinician interaction. Clinical studies describe TOR rule performance in routine clinical practice. Meta-analysis of derivation, validation, and clinical studies together has the potential to bias estimates of TOR rule performance in clinical practice.

Statistical analysis was performed in R Studio, version 1.2.5042 (R Project for Statistical Computing),²¹ using several packages. Univariate analysis required contingency table data to calculate summary estimates using epiR, version 2.0.65.²² We used Meta, version 6.2-1,²³ to generate the Deeks funnel plot. We conducted a bivariate random-effects meta-analysis using a generalized linear mixed model as advocated by Reitsma et al.²⁴ We calculated the area under the curve (AUC) and produced bivariate SROC curves using Metafor, version 4.0-0.¹⁹ We calculated pooled sensitivity, specificity, and diagnostic odds ratios (DORs) using Meta, version 6.2-1.²³ Estimates of effects at different prevalence levels were calculated using the GRADEPro Guideline Development Tool²⁵ and were subsequently used to estimate the effects of TOR rules by calculating terminate and transport rates, miss rates, miss frequency, survivor rates, survivor frequency, and futile transport rates. Numbers of cases (in lieu of patients) are reported, because several studies used the same database, which meant patients could be counted more than once. Statistical significance was established at P < .05 (2-tailed).

Results

The database searches yielded 10 399 records. No additional studies were identified by searching reference lists or contacting subject experts. After deduplication, 7266 records remained. First-pass title and abstract screening yielded 131 potentially eligible studies. After the second-pass full-text screening, 43 studies^{26,27,28,29,30,31,32,33,34,35,36,37,38,39,40,41,42,43,44,45,46,47,48,49,50,51,52,53,54,55,56,57,58,59,60,61,62,63,64,65,66,67,68} remained (Figure 1).

Figure 1. — ROSC indicates return of spontaneous circulation populations.

Study Characteristics

No randomized clinical trials were identified. This review included 43 nonrandomized studies involving 1 125 587 cases from 11 different countries. Publication dates spanned 1993 to 2023. There was substantial variation in study populations and prevalence across the included studies (eTables 7-9 in Supplement 1). Most studies reported death prior to hospital discharge; however, 1 study utilized death within 1 month.³⁶ Three studies collected data prospectively,^30,37,55 whereas the remainder utilized retrospective datasets. A summary of each study and a description of each TOR rule is provided in eTables 10 and 11, respectively, in Supplement 1.

Several studies have incorrectly reported efficacy of the basic life support (BLS) TOR rule. The BLS and UTOR rules use the same variables, but the BLS TOR rule captures use by BLS responders only.⁶⁵ When the BLS TOR rule is applied within systems deploying clinicians operating above the BLS level, it should be reported as the UTOR rule.⁵⁴ Multiple studies reporting BLS TOR rule performance included responders who operated above the BLS level.^{27,33,36,41,42,46,47,58,60,64,68} However, clinicians providing BLS cannot administer adrenaline or undertake advanced airway interventions. They therefore experience lower rates of ventricular fibrillation and return of spontaneous circulation (ROSC)—both variables of the BLS and UTOR rules. To accurately describe the performance of the BLS TOR and UTOR rules, we reclassified the aforementioned studies to facilitate accurate meta-analysis.

Risk of Bias

Summary risk of bias and individual study risk of bias are reported in eFigures 1 and 2, respectively, in Supplement 1. There was substantial concern for patient selection, as many studies excluded subsections of the cardiac arrest population. Low risk of bias for the reference standard was expected due to the unambiguous nature of the outcome being assessed—the patient either lives or dies. These data are commonly reliable and readily confirmed. High risk of bias in flow and timing relates to concerns for verification bias; several studies were conducted in EMS systems practicing TOR, creating a self-fulfilling prophecy.

Publication Bias

We generated a Deeks funnel plot for each TOR rule with 4 or more studies (the minimum required for computation).⁶⁹ Asymmetry in the plot indicates potential publication bias.⁶⁹ The generated plots (eFigure 3 in Supplement 1) suggested no concern for publication bias for the advanced life support (ALS) and UTOR rules. Publication bias may be a concern for the BLS rule; however, this was uncertain because there were few included studies. Insufficient data prevented generation of Deeks plots for the Marsden, Petrie, Goto 1, and Shibahashi 1 TOR rules.

Derivation Studies

We identified very low-certainty evidence (downgraded for risk of bias, inconsistency, imprecision, and indirectness) from 15 observational studies^{26,29,34,36,40,41,44,49,51,53,59,64,65,68,70} involving 198 442 cases. These studies reported the derivation of 20 unique TOR rules. eTable 7 in Supplement 1 presents their sensitivity and specificity with 95% CIs. Because each derivation rule is distinct, meta-analysis is not appropriate and no heterogeneity assessment was undertaken.

External Validation Studies

We identified very low-certainty evidence (downgraded for risk of bias, inconsistency, imprecision, indirectness, and publication bias) from 33 observational studies^{27,28,30,31,32,33,35,36,37,38,39,41,42,43,45,46,47,48,49,50,52,53,54,56,57,58,60,62,63,64,66,67,68} involving 927 534 cases. These studies reported external data validations of 17 TOR rules. eTable 8 in Supplement 1 presents their sensitivity and specificity with 95% CIs. Within these 33 studies, we identified 7 TOR rules (BLS, ALS, UTOR, Marsden, Petrie, Shibahashi 1, and Goto 1) for which meta-analysis was possible.

Bivariate SROC curves are shown in Figure 2 for the BLS, ALS, and UTOR rules and in eFigure 4 in Supplement 1 for the remaining rules. Correlation coefficients for the BLS, ALS, and UTOR rules were −0.77, −0.65, and −0.72, respectively, suggesting that heterogeneity between studies was not a notable concern. It was not possible to reliably estimate correlation coefficients for the Marsden, Petrie, Goto 1, and Shibahashi 1 TOR rules because there were insufficient studies. Our meta-analysis suggests that the Petrie rule performed poorly (AUC, 0.56), whereas the ALS rule failed to reach acceptable standards (AUC, 0.63). Both the BLS (AUC, 0.79) and Shibahashi 1 (AUC, 0.75) rules achieved acceptable performance, whereas the UTOR (AUC, 0.85), Marsden (AUC, 0.81), and Goto 1 (AUC, 0.85) rules had excellent performance (Figure 2 and eFigure 4 in Supplement 1). Pooled estimates of effect are reported in Table 1, a summary of findings with estimates of effect is reported in Table 2, and estimated performance at different prevalence rates is reported in Table 3.

Table 1. Pooled Summary Estimates for External Data Validation Studies.

TOR rule	No. of studies	Sensitivity (95% CI)	Specificity (95% CI)	DOR (95% CI)
BLS	6	0.66 (0.59-0.74)	0.81 (0.70-0.91)	13.70 (3.76-49.83)
ALS	17	0.27 (0.20-0.34)	0.96 (0.93-0.99)	9.21 (5.77-14.69)
UTOR	19	0.63 (0.54-0.72)	0.88 (0.81-0.94)	21.86 (13.81-34.60)
Marsden	2	0.42 (0-0.86)	0.97 (0.88-1.00)	44.97 (5.54-365.23)
Petrie	2	0.21 (0-0.43)	0.98 (0.93-1.00)	20.05 (2.14-187.57)
Goto 1	2	0.46 (0.32-0.61)	0.93 (0.90-0.97)	12.33 (10.96-13.86)
Shibahashi 1	2	0.30 (0.17-0.41)	0.96 (0.93-0.99)	9.51 (7.70-11.75)

Open in a new tab

Abbreviations: ALS, advanced life support; BLS, basic life support; DOR, diagnostic odds ratio; UTOR, universal termination of resuscitation.

Table 2. Summary of Findings for Estimates of Effect for TOR Rules at Different Survival Rates^a.

TOR rule	No. of studies (cases)^b	Factors that may decrease certainty of evidence					Certainty of evidence	Outcome	Estimate of effect per 1000 patients (95% CI)			Pooled sensitivity, pooled specificity, and survival range^c
TOR rule	No. of studies (cases)^b	Risk of bias	Indirectness	Inconsistency	Imprecision	Other	Certainty of evidence	Outcome	8% Survival	10% Survival	12% Survival
BLS	6 (36 325)	Very serious	Very serious	Serious	Not serious	Publication bias	Very low	TP	641 (559-722)	627 (547-707)	613 (535-691)	0.66 (0.59-0.74), 0.81 (0.70-0.91), 1.4-14.7
								FP	18 (10-27)	23 (12-34)	28 (14-41)
								FN	279 (198-361)	273 (193-353)	267 (189-345)
								TN	62 (53-70)	77 (66-88)	92 (79-106)
UTOR	19 (369 631)	Very serious	Very serious	Serious	Not serious	Publication bias	Very low	TP	574 (497-652)	562 (486-638)	549 (475-624)	0.62 (0.54-0.71), 0.88 (0.81-0.94), 1.4-19.9
								FP	10 (5-15)	12 (6-19)	15 (7-22)
								FN	346 (268-423)	338 (262-414)	331 (256-405)
								TN	70 (65-75)	88 (81-94)	105 (98-113)
ALS	17 (276 861)	Very serious	Very serious	Serious	Not serious	Publication bias	Very low	TP	236 (183-290)	231 (179-283)	226 (175-277)	0.26 (0.20-0.31), 0.96 (0.93-0.99), 1.2-15.6
								FP	3 (1-5)	4 (1-7)	5 (1-8)
								FN	684 (630-737)	669 (617-721)	654 (603-705)
								TN	77 (75-79)	96 (93-99)	115 (112 to 119)
Marsden	2 (15 953)	Very serious	Very serious	Serious	Not serious	Publication bias	Very low	TP	387 (0-795)	378 (0-778)	370 (0-760)	0.42 (0.0-0.86), 0.97 (0.89-1.0), 1.4-4.6
								FP	3 (0-9)	3 (0-11)	4 (0-13)
								FN	533 (125-920)	522 (122-900)	510 (120-880)
								TN	77 (71-80)	97 (89-100)	116 (107-120)
Petrie	2 (15 953)	Very serious	Very serious	Serious	Not serious	Publication bias	Very low	TP	193 (0-392)	188 (0-383)	184 (0-375)	0.21 (0.0-0.43), 0.98 (0.93-1.0), 1.4-4.6
								FP	1 (0-6)	2 (0-7)	2 (0-8)
								FN	727 (528-920)	712 (517-900)	696 (505-880)
								TN	79 (74-80)	98 (93-100)	118 (112-120)
Goto 1	2 (59 672)	Very serious	Very serious	Serious	Not serious	Publication bias	Very low	TP	426 (291-561)	417 (284-549)	408 (278-537)	0.46 (0.32-0.61), 0.93 (0.90-0.97), 6.0-11.7
								FP	6 (3-8)	7 (3-10)	8 (4-13)
								FN	494 (359-629)	483 (351-616)	472 (343-602)
								TN	74 (72-77)	93 (90-97)	112 (107-116)
Shibahashi 1	2 (80 939)	Very serious	Very serious	Serious	Not serious	Publication bias	Very low	TP	266 (158-375)	261 (154-367)	255 (151-359)	0.29 (0.17-0.41), 0.96 (0.93-0.99), 6.0%-9.3%
								FP	3 (1-6)	4 (1-7)	5 (1-9)
								FN	654 (545-762)	639 (533-746)	625 (521-729)
								TN	77 (74-79)	96 (93-99)	115 (111-119)

Open in a new tab

Abbreviations: ALS, advanced life support; BLS, basic life support; FN, false negative; FP, false positive; TN, true negative; TOR, termination of resuscitation; TP, true positive; UTOR, universal termination of resuscitation.

^{^a}

Included studies met the following criteria: (1) population: patients experiencing out-of-hospital cardiac arrest; (2) index test: TOR rule recommending termination of resuscitation; (3) reference standard: true outcome (patient died or survived); and (4) study type: external validation studies of TOR rules.

^{^b}

All were nonrandomized.

^{^c}

Pooled sensitivity and specificity values are presented with 95% CIs. Percent survival is presented as a range.

Table 3. Estimated Performance of TOR Rule Use at Different Prevalence Rates^a.

TOR rule, % survival	Discontinue, %^b	Discontinue decision, %^c		Miss, %^d	Miss frequency^e	Transport, %^f	Transport decision, %^g		Survivor, %^h	Survivor frequencyⁱ	Futile transport, %^j
TOR rule, % survival	Discontinue, %^b	Correct	Incorrect	Miss, %^d	Miss frequency^e	Transport, %^f	Correct	Incorrect	Survivor, %^h	Survivor frequencyⁱ	Futile transport, %^j
BLS
8	65.9 (63.5-69.4)	97.3 (96.4-98.2)	2.7 (1.8-3.6)	1.8 (1.2-2.3)	56 (44-82)	34.1 (30.6-36.5)	18.2 (16.2-21.1)	81.8 (78.9-83.8)	6.2 (5.9-6.5)	16.1 (15.5-16.9)	28 (24-31)
10	65.0 (62.7-68.3)	96.5 (95.4-97.9)	3.5 (2.1-4.6)	2.3 (1.5-2.9)	44 (35-68)	35.0 (31.7-37.3)	22.0 (20.0-25.5)	78.0 (74.5-80.0)	7.7 (7.4-8.1)	13.0 (12.4-13.4)	27 (24-30)
12	64.1 (61.9-67.7)	95.6 (94.4-97.4)	4.4 (2.6-5.6)	2.8 (1.7-3.5)	36 (29-58)	35.9 (32.8-38.1)	25.6 (23.5-29.5)	74.4 (70.5-76.5)	9.2 (9.0-9.7)	10.9 (10.3-11.2)	27 (23-29)
UTOR
8	58.4 (57.3-60.1)	98.3(97.8-99.0)	1.7 (1.0-2.2)	1.0 (0.6-1.3)	100 (167-78)	41.6 (39.9-42.7)	16.8 (15.1-19.5)	83.2 (80.5-84.9)	7.0 (6.4-7.8)	14 (13-16)	35 (32-36)
10	57.4 (56.4-58.9)	97.9 (97.1-98.8)	2.1 (1.2-2.9)	1.2 (0.7-1.6)	83 (61-139)	42.6 (41.1-43.6)	20.7 (18.5-23.6)	79.3 (76.4-81.5)	8.8 (8.1-9.7)	11 (10-12)	34 (31-36)
12	56.4 (55.5-57.7)	97.3 (96.6-98.5)	2.7 (1.5-3.4)	1.5 (0.8-1.9)	67 (53-119)	43.6 (42.3-44.5)	24.1 (21.8-27.7)	75.9 (72.3-78.2)	10.5 (9.7-11.7)	9.5 (8.5-10.3)	33 (31-35)
ALS
8	23.9 (20.7-26.6)	98.7 (98.3-99.5)	1.3 (0.5-1.7)	0.3 (0.1-0.5)	333 (222-889)	76.1 (73.4-79.3)	10.1 (9.7-10.6)	89.8 (89.4-90.3)	7.7 (7.1-8.4)	13 (12-14)	68 (66-71)
10	23.5 (20.2-23.5)	98.3 (97.6-99.4)	1.7 (0.6-2.4)	0.4 (0.1-0.6)	250 (159-890)	76.5 (73.9-79.8)	12.5 (12.1-13.1)	87.5 (86.9-87.9)	9.6 (8.9-10.4)	10.4 (9.6-11.2)	67 (65-69)
12	23.1 (19.8-25.7)	97.8 (97.2-99.4)	2.2 (0.6-2.8)	0.5 (0.1-0.7)	22 (139-891)	76.9 (74.3-80.2)	15.0 (14.4-15.7)	85.0 (84.3-85.6)	11.5 (10.7-12.6)	8.7 (8.0-9.3)	65 (64-68)
Marsden
8	39.0 (0.1-44.6)	99.2 (50.0-98.9)	0.8 (1.1-50.0)	0.3 (0.1-0.5)	333 (200-1962)	61.0 (55.4-99.9)	12.6 (8.0-36.2)	87.4 (63.8-92.0)	7.7 (4.4-36.2)	13.0 (2.8-22.6)	53 (51-64)
10	38.1 (0.1-44.1)	99.2 (98.9-1.0)	0.8 (0-1.1)	0.3 (0-0.6)	333 (163-2111)	61.9 (55.9-100)	15.7 (10.0-42.2)	84.3 (57.8-90.0)	9.7 (5.6-42.2)	10.3 (2.4-17.9)	52 (50-58)
12	37.4 (0.1-43.6)	98.9 (98.3-1.0)	1.1 (0-1.7)	0.4 (0-0.7)	250 (136-2271)	62.6 (56.4-100)	18.5 (12.0-47.1)	81.5 (52.9-88.0)	11.6 (6.8-47.1)	8.6 (2.1-14.8)	51 (50-53)
Petrie
8	19.4 (0-28.5)	99.5 (98.5-1.0)	0.5 (0-1.5)	0.1 (0-0.4)	1000(233-5811)	80.6 (71.5-100)	9.8 (8.0-12.3)	90.2 (92.0-87.7)	7.9 (5.7-12.3)	12.7 (8.1-17.5)	73 (66-88)
10	19.0 (0-28.1)	98.9 (98.2-1.0)	1.1 (0-1.8)	0.2 (0-0.5)	500 (198-6102)	81.0 (71.9-100)	12.1 (10.0-15.2)	87.9 (84.8-90.0)	9.8 (7.2-15.2)	10.2 (6.6-13.9)	71 (65-85)
12	18.6 (0-27.7)	98.9 (97.9-1.0)	1.1 (0-2.1)	0.2 (0-0.6)	500 (173-6171)	81.4 (72.3-100)	14.5 (12.0-18.2)	85.5 (81.8-88.0)	11.8 (8.7-18.1)	8.5 (5.5-11.5)	70 (64-82)
Goto 1
8	43.2 (40.6-44.6)	98.6 (98.6-99.0)	1.4 (1.0-1.4)	0.6 (0.4-0.6)	167 (159-242)	56.8 (55.4-59.4)	13.0 (10.9-16.7)	87.0 (83.3-89.1)	7.4 (6.0-9.9)	13.5 (10.1-16.6)	49 (49-50)
10	42.4 (39.4-43.9)	98.3 (98.2-99.0)	1.7 (1.0-1.8)	0.7 (0.4-0.8)	143 (127-243)	57.6 (56.1-60.6)	16.1 (13.6-20.4)	83.9 (79.6-86.4)	9.3 (7.6-12.4)	10.8 (8.1-13.1)	48 (48-48)
12	41.6 (38.5-43.4)	98.1 (97.6-98.6)	1.9 (1.4-2.4)	0.8 (0.5-0.8)	125 (98-183)	58.4 (56.6-61.5)	19.2 (16.2-23.8)	80.8 (76.2-83.8)	11.2 (9.1-14.6)	8.9 (6.8-10.9)	47 (47-47)
Shibahashi 1
8	26.9 (20.4-31.2)	98.9 (98.4-99.4)	1.1 (0.6-1.6)	0.3 (0.1-0.5)	333 (204-778)	73.1 (68.8-79.6)	10.5 (9.4-12.0)	89.5 (88.0-90.6)	7.7 (6.5-9.5)	13.0 (10.5-15.5)	65 (62-70)
10	26.5 (19.8-30.7)	98.5 (98.1-99.4)	1.5 (0.6-1.9)	0.4 (0.1-0.6)	250 (174-781)	73.5 (69.3-80.2)	13.1 (11.7-14.9)	86.9 (85.1-88.3)	9.6 (8.1-11.9)	10.4 (8.4-12.3)	64 (61-68)
12	26.0 (19.4-30.3)	98.1 (97.6-99.3)	1.9 (0.7-2.4)	0.5 (0.1-0.7)	200 (135-784)	74.0 (69.7-80.6)	15.5 (14.0-17.6)	88.3 (82.4-84.5)	11.5 (9.8-14.2)	8.7 (7.1-10.2)	63 (60-66)

Open in a new tab

^{^a}

Values are presented with 95% CIs.

^{^b}

Calculated as [(TP + FP)/(TP + FP + FN + TN)].

^{^c}

Calculated as [(TP)/(TP + FP)] for correct and [(FP)/(TP + FP)] for incorrect.

^{^d}

Calculated as [(FP)/(TP + FP + FN + TN)].

^{^e}

Calculated as [1/(% Miss)].

^{^f}

Calculated as [(TN + FN)/(TP + FP + FN + TN)].

^{^g}

Calculated as correct – [(TN)/(FN + TN)] for correct and [(FN)/(FN + TN)] for incorrect.

^{^h}

Calculated as [(TN)/(TP + FP + FN + TN)].

^ⁱ

Calculated as [1/(% Survivor)].

^{^j}

Calculated as [(FN)/(TP + FP + FN + TN)].

Clinical Studies

We identified very low-certainty evidence (downgraded for indirectness) from 1 Canadian study⁵⁵ involving 954 cases (eTable 9 in Supplement 1). This study described the clinical validation of the BLS rule.⁵⁵ The study had sensitivity of 0.64 (95% CI, 0.61-0.68), specificity of 1.00 (95% CI, 0.92-1.00), and positive predictive value of 1.00 (95% CI, 0.99-1.00). The BLS rule recommended transport for 367 of 953 cases (38.5%); of these 367, 44 (12.0%) survived to discharge and 323 (88.0%) died in the hospital. The BLS rule recommended TOR for the remaining 586 cases; however, resuscitation was terminated for only 388 (66.2%). Ambulance crews transported 198 patients to the hospital despite the recommendation of the BLS rule to stop resuscitation; none of these patients survived.

Discussion

To our knowledge, this review is the first to analyze studies by derivation, external data validation, and clinical categories to minimize the bias that may be introduced by pooling these categories. This meta-analysis is also the first, to our knowledge, to reclassify studies incorrectly reporting the efficacy of the BLS rule rather than the UTOR rule, enabling an accurate performance assessment of these 2 TOR rules by their intended clinician populations. Finally, our study is the only meta-analysis to date to estimate TOR rule performance at different prevalence levels (88%, 90%, and 92% [12%, 10%, and 8% survival]).

The TOR rules are intended to differentiate between those patients for whom resuscitation can be safely discontinued and those who might benefit from further (hospital) treatment.²⁶ Traditionally reported metrics to describe TOR rule performance include sensitivity, specificity, positive predictive value, transport rate, and miss rate. These metrics frame the performance of the TOR rule with respect to patient safety (how many potential survivors are missed) and resource utilization (reduced number of hospital transports).

Early TOR rule studies reported 100% specificity. However, it may be unrealistic to expect a TOR rule with 100% specificity that does not miss any potential survivors.¹² This argument asserts that the ethically acceptable threshold for medical futility is 1% and that specificity of 99% (a miss rate of 1%) should therefore represent acceptable TOR rule performance.⁷¹ Conversely, the European Resuscitation Council argued that success rates of less than 1% still justify a resuscitation effort, questioning the acceptability of a 1% miss rate.⁷² Our analysis suggests that miss rates could range from 0.1% (95% CI, 0%-0.4%) for the Petrie rule to 1.8% (95% CI, 1.2%-2.3%) for the BLS rule (assuming 8% survival; Table 3).

Studies on TOR rules frequently report statistically significant reductions in transport rates. However, these estimates seldom reflect practice in Western EMS systems. Verhaert et al⁶⁶ reported that the ALS rule recommended transport for 94% of cases in a system that only transported 54% of cases. The most recent data from English ambulance services show that 41.7% of cases were transported to the hospital.⁷³ Our analysis suggests that transport rates would vary from 34.1% (range, 30.6%-36.5%) for the BLS rule to 80.6% (range, 71.5%-100%) for the Petrie rule when prevalence is 8% (Table 3). Lower transport rates most likely occur because clinical practice guidelines provide additional scope not to start resuscitation in cases in which it would be futile, for patients with terminal illness, or where the patient has expressed a wish not to be resuscitated. Current TOR rules lack this flexibility.

Of the TOR rules identified in our meta-analysis, the BLS rule could not be implemented in the UK because the EMS system utilizes ALS-level paramedics. Neither the Goto 1 and Shibahashi 1 TOR rules could be implemented legally because they both discriminate by age and would contravene the UK Equality Act of 2010.⁷⁴ The ALS, Goto 1, and Marsden TOR rules include bystander cardiopulmonary resuscitation (CPR) as a variable, requiring continued resuscitation (eTable 11 in Supplement 1). Currently, bystander CPR rates in England approach 70%, suggesting that these rules may be less helpful in a UK context because they will automatically recommend that the majority of patients be transported. Of the remaining TOR rules examined in this review, the Petrie TOR rule had the best specificity but poor sensitivity. We found that the UTOR rule had the best sensitivity and DOR (Table 1). Based on both our AUC and DOR data, the UTOR rule had the best performance.

Our analysis suggests that the UTOR rule would miss 1.0% (95% CI, 0.6%-1.3%) of survivors (8% survival; Table 3). This means that 1 survivor would be missed for every 100 (95% CI, 78-167) resuscitation attempts. Paramedic exposure to cardiac arrest is low (range, 2-5 cardiac arrests per year).^75,76,77 If a paramedic attended 3 cardiac arrests per year, then each paramedic would miss a survivor every 33.3 years. However, if survival were to improve to 10% (90% prevalence), then the miss rate would increase to 1.2% (higher than the “acceptable” 1% miss rate⁷¹), equating to an additional 68 missed survivors annually in England (based on 2021 data).⁷³ Similarly, at 12% survival (88% prevalence), the miss rate would rise to 1.5%, equating to 170 missed survivors. In the context of only 2700 cardiac survivors nationally, the number of missed survivors is not insignificant and is unlikely to be deemed acceptable.

Unfortunately, common TOR rule metrics also overlook incorrect (false-negative) recommendations to continue resuscitation. This may be driven by a belief that continuing resuscitation to the hospital is not harmful. However, recent data indicate that transporting patients during resuscitation is associated with reduced probability of survival compared with resuscitation on scene.⁷⁸ Furthermore, Wampler et al⁷⁹ reported that survival was rare where ROSC was not achieved before initiating transport. International guidelines now recommend against routine transportation to the hospital unless needed to access treatment that EMS cannot provide on scene or when legal or cultural considerations mandate transfer.⁸⁰ Transporting patients for whom resuscitation is futile is not benign: it consumes scarce emergency department resources and increases risk for ambulance clinicians. Up to 81.4% of work-related injuries among ambulance staff have been attributed to ambulance collisions.^81,82 Compared with vehicles of a similar size, ambulance collisions occur more frequently,⁸³ involve a greater number of casualties, and are more likely to result in substantial injury.⁸⁴

Our findings suggest that in addition to mitigating risk, reducing futile transport will also realize financial benefits. At 8% survival, the UTOR rule recommends futile transport in 35% (95% CI, 32%-36%) of cases (Table 3). Recent data indicate that this equates to 11 900 futile transports in English EMS systems annually.⁷³ Cost-effectiveness data from the PARAMeDIC2 trial suggested that death at the scene was associated with mean (SD) ambulance costs of £1793.89 (£1056.61) (US $2279.63 [$1342.71]), whereas transported patients who died within 24 hours had associated mean (SD) ambulance service costs of £1507.69 (£562.56) (US $1915.93 [$714.89]) and hospital costs of £682.44 (£1515.93) (US $867.23 [$1926.41]).⁸⁵ This finding implies an incremental cost of £396.24 (US $503.53) associated with death following transport. Minimizing futile transport rates therefore has the potential to realize substantial savings. In England, the UTOR rule would incur potentially avoidable costs of £4.7 million (95% CI, £4.3-4.9 million) (approximately US $5.10 million [95% CI, US $5.5-6.2 million]), assuming 34 000 resuscitation attempts each year (assuming 8% survival).⁷³

Limitations

This review has the following limitations. All included studies were observational in design, and the majority were retrospective in nature. This limitation has important implications for the quality of the evidence and inferences that can be made from these data. We excluded studies of TOR rules predicting favorable neurologic outcomes rather than death, because patients, families, and communities place different value on survival with poor neurological outcome. Furthermore, estimation of neurological outcome at discharge or 30 days is unreliable due to improvements seen in postresuscitation care.⁸⁶ Rigorous synthesis of the literature was further limited by heterogeneity in the populations studied, differences in the scope of practice of EMS personnel, diverse EMS system design, substantial variation in the quality of CPR and the prevalence of resuscitation outcomes, differences in how decisions are made to cease resuscitation, and the inherent risk of TOR rules creating a self-fulfilling prophecy.

Conclusions

The findings of this systematic review and meta-analysis suggest that there is very low-certainty evidence concerning the ability of TOR rules to discriminate between patients who will die and those who will survive. The literature comprises mainly derivation and external data validation studies. Clinical studies are almost nonexistent. Our findings suggest that TOR rules may miss substantial numbers of survivors. In addition, futile transport is not consistent with evidence-based practice; it reduces the likelihood of survival, increases risk, consumes scarce emergency department resources, and incurs substantial avoidable costs. Therefore, we suggest that there is an urgent need to review the role of TOR rules.

Supplement 1.

eTable 1. PICOST Criteria

eTable 2. MEDLINE Search Strategy

eTable 3. Embase Search Strategy

eTable 4. CINAHL Search Strategy

eTable 5. Cochrane Search Strategy

eTable 6. Web of Science Search Strategy

eTable 7. Derivation Studies Paired Sensitivity and Specificity

eTable 8. External Validation Studies Paired Sensitivity and Specificity

eTable 9. Clinical Studies Paired Sensitivity and Specificity

eTable 10. Summary of Included Studies

eTable 11. Description of Termination of Resuscitation (TOR) Rules

eFigure 1. QUADAS-2 Summary Risk of Bias and Applicability

eFigure 2. Study Risk of Bias

eFigure 3. Deeks Funnel Plot Asymmetry Test

eFigure 4. Bivariate Summary Receiver Operating Characteristic (SROC) Curves

eReferences

jamanetwopen-e2420040-s001.pdf^{(698.1KB, pdf)}

Supplement 2.

Data Sharing Statement

jamanetwopen-e2420040-s002.pdf^{(15.9KB, pdf)}

References

1.Gräsner JT, Herlitz J, Tjelmeland IBM, et al. European Resuscitation Council Guidelines 2021: epidemiology of cardiac arrest in Europe. Resuscitation. 2021;161:61-79. doi: 10.1016/j.resuscitation.2021.02.007 [DOI] [PubMed] [Google Scholar]
2.Anderson NE, Gott M, Slark J. Commence, continue, withhold or terminate? a systematic review of decision-making in out-of-hospital cardiac arrest. Eur J Emerg Med. 2017;24(2):80-86. doi: 10.1097/MEJ.0000000000000407 [DOI] [PubMed] [Google Scholar]
3.Hick JL, Mahoney BD, Lappe M. Factors influencing hospital transport of patients in continuing cardiac arrest. Ann Emerg Med. 1998;32(1):19-25. doi: 10.1016/S0196-0644(98)70094-0 [DOI] [PubMed] [Google Scholar]
4.Mao DRH, Ong MEH, Bang C, Salim MDT, Ng YY, Lie DA. Psychological comfort of paramedics with field death pronouncement: a national Asian study to prepare paramedics for field termination of resuscitation. Prehosp Emerg Care. 2018;22(2):260-265. doi: 10.1080/10903127.2017.1376132 [DOI] [PubMed] [Google Scholar]
5.Grudzen CR, Timmermans S, Koenig WJ, et al. Paramedic and emergency medical technicians views on opportunities and challenges when forgoing and halting resuscitation in the field. Acad Emerg Med. 2009;16(6):532-538. doi: 10.1111/j.1553-2712.2009.00427.x [DOI] [PubMed] [Google Scholar]
6.National Association of EMS Physicians . Termination of resuscitation in nontraumatic cardiopulmonary arrest. Prehosp Emerg Care. 2011;15(4):542. doi: 10.3109/10903127.2011.598621 [DOI] [PubMed] [Google Scholar]
7.National Association of EMS Physicians; American College of Surgeons Committee on Trauma . Withholding of resuscitation for adult traumatic cardiopulmonary arrest. Prehosp Emerg Care. 2013;17(2):291. doi: 10.3109/10903127.2012.755586 [DOI] [PubMed] [Google Scholar]
8.Association of Ambulance Services Chief Executives . UK Ambulance Services Clinical Practice Guidelines. Class Professional Publishing; 2021. [Google Scholar]
9.Greif R, Bhanji F, Bigham BL, et al. ; Education, Implementation, and Teams Collaborators . Education, implementation, and teams: 2020 international consensus on cardiopulmonary resuscitation and emergency cardiovascular care science with treatment recommendations. Circulation. 2020;142(16 suppl 1):S222-S283. doi: 10.1161/CIR.0000000000000896 [DOI] [PubMed] [Google Scholar]
10.Lauridsen KG, Baldi E, Smyth M, et al. ; Education Implementation and Team Task Force of the International Liaison Committee on Resuscitation (ILCOR) . Clinical decision rules for termination of resuscitation during in-hospital cardiac arrest: a systematic review of diagnostic test accuracy studies. Resuscitation. 2021;158:23-29. doi: 10.1016/j.resuscitation.2020.10.036 [DOI] [PubMed] [Google Scholar]
11.Leeflang MM, Deeks JJ, Takwoingi Y, Macaskill P. Cochrane diagnostic test accuracy reviews. Syst Rev. 2013;2(1):82. doi: 10.1186/2046-4053-2-82 [DOI] [PMC free article] [PubMed] [Google Scholar]
12.Morrison LJ. Prehospital termination of resuscitation rule. Curr Opin Crit Care. 2019;25(3):199-203. doi: 10.1097/MCC.0000000000000614 [DOI] [PubMed] [Google Scholar]
13.Ebell MH, Vellinga A, Masterson S, Yun P. Meta-analysis of the accuracy of termination of resuscitation rules for out-of-hospital cardiac arrest. Emerg Med J. 2019;36(8):479-484. doi: 10.1136/emermed-2018-207833 [DOI] [PubMed] [Google Scholar]
14.Nas J, Kleinnibbelink G, Hannink G, et al. Diagnostic performance of the basic and advanced life support termination of resuscitation rules: a systematic review and diagnostic meta-analysis. Resuscitation. 2020;148:3-13. doi: 10.1016/j.resuscitation.2019.12.016 [DOI] [PubMed] [Google Scholar]
15.Sterne JA, Hernán MA, Reeves BC, et al. ROBINS-I: a tool for assessing risk of bias in non-randomised studies of interventions. BMJ. 2016;355:i4919. doi: 10.1136/bmj.i4919 [DOI] [PMC free article] [PubMed] [Google Scholar]
16.Sterne JAC, Savović J, Page MJ, et al. RoB 2: a revised tool for assessing risk of bias in randomised trials. BMJ. 2019;366:l4898. doi: 10.1136/bmj.l4898 [DOI] [PubMed] [Google Scholar]
17.Whiting PF, Rutjes AW, Westwood ME, et al. ; QUADAS-2 Group . QUADAS-2: a revised tool for the quality assessment of diagnostic accuracy studies. Ann Intern Med. 2011;155(8):529-536. doi: 10.7326/0003-4819-155-8-201110180-00009 [DOI] [PubMed] [Google Scholar]
18.Macaskill PTY, Deeks JJ, Gatsonis C. Understanding meta-analysis. In: Deeks JJBP, Leeflang MM, Takwoingi Y, eds. Cochrane Handbook for Systematic Reviews of Diagnostic Test Accuracy (Version 2). Cochrane Collaboration; 2022:chap 9. [Google Scholar]
19.Viechtbauer W, Viechtbauer MW. Package metafor. Comprehensive R Archive Network. 2015. Accessed November 10, 2023. http://cran.r-project.org/web/packages/metafor/metafor.pdf
20.Guyatt GH, Oxman AD, Vist GE, et al. ; GRADE Working Group . GRADE: an emerging consensus on rating quality of evidence and strength of recommendations. BMJ. 2008;336(7650):924-926. doi: 10.1136/bmj.39489.470347.AD [DOI] [PMC free article] [PubMed] [Google Scholar]
21.Allaire J. RStudio: Integrated Development Environment for R. R Project for Statistical Computing; 2012. [Google Scholar]
22.Stevenson M, Stevenson MM. BiasedUrn I. Package epiR; 2018.
23.Schwarzer G. Meta: an R package for meta-analysis. R News. 2007;7(3):40-45. [Google Scholar]
24.Reitsma JB, Glas AS, Rutjes AW, Scholten RJ, Bossuyt PM, Zwinderman AH. Bivariate analysis of sensitivity and specificity produces informative summary measures in diagnostic reviews. J Clin Epidemiol. 2005;58(10):982-990. doi: 10.1016/j.jclinepi.2005.02.022 [DOI] [PubMed] [Google Scholar]
25.McMaster University . Evidence Prime. GRADEpro Guideline Development Tool. 2022. Accessed September 1, 2023. https://www.gradepro.org
26.Bonnin MJ, Pepe PE, Kimball KT, Clark PS Jr. Distinct criteria for termination of resuscitation in the out-of-hospital setting. JAMA. 1993;270(12):1457-1462. doi: 10.1001/jama.1993.03510120079035 [DOI] [PubMed] [Google Scholar]
27.Cheong RW, Li H, Doctor NE, et al. Termination of resuscitation rules to predict neurological outcomes in out-of-hospital cardiac arrest for an intermediate life support prehospital system. Prehosp Emerg Care. 2016;20(5):623-629. doi: 10.3109/10903127.2016.1162886 [DOI] [PubMed] [Google Scholar]
28.Chiang WC, Ko PC, Chang AM, et al. Predictive performance of universal termination of resuscitation rules in an Asian community: are they accurate enough? Emerg Med J. 2015;32(4):318-323. doi: 10.1136/emermed-2013-203289 [DOI] [PMC free article] [PubMed] [Google Scholar]
29.Chiang WC, Huang YS, Hsu SH, et al. Performance of a simplified termination of resuscitation rule for adult traumatic cardiopulmonary arrest in the prehospital setting. Emerg Med J. 2017;34(1):39-45. doi: 10.1136/emermed-2014-204493 [DOI] [PubMed] [Google Scholar]
30.Cone DC, Bailey ED, Spackman AB. The safety of a field termination-of-resuscitation protocol. Prehosp Emerg Care. 2005;9(3):276-281. doi: 10.1080/10903120590961996 [DOI] [PubMed] [Google Scholar]
31.Diskin FJ, Camp-Rogers T, Peberdy MA, Ornato JP, Kurz MC. External validation of termination of resuscitation guidelines in the setting of intra-arrest cold saline, mechanical CPR, and comprehensive post resuscitation care. Resuscitation. 2014;85(7):910-914. doi: 10.1016/j.resuscitation.2014.02.028 [DOI] [PubMed] [Google Scholar]
32.Drennan IR, Lin S, Sidalak DE, Morrison LJ. Survival rates in out-of-hospital cardiac arrest patients transported without prehospital return of spontaneous circulation: an observational cohort study. Resuscitation. 2014;85(11):1488-1493. doi: 10.1016/j.resuscitation.2014.07.011 [DOI] [PubMed] [Google Scholar]
33.Fukuda T, Ohashi N, Matsubara T, et al. Applicability of the prehospital termination of resuscitation rule in an area dense with hospitals in Tokyo: a single-center, retrospective, observational study: is the pre hospital TOR rule applicable in Tokyo? Am J Emerg Med. 2014;32(2):144-149. doi: 10.1016/j.ajem.2013.10.032 [DOI] [PubMed] [Google Scholar]
34.Glober NK, Tainter CR, Abramson TM, Staats K, Gilbert G, Kim D. A simple decision rule predicts futile resuscitation of out-of-hospital cardiac arrest. Resuscitation. 2019;142:8-13. doi: 10.1016/j.resuscitation.2019.06.011 [DOI] [PubMed] [Google Scholar]
35.Glober NK, Lardaro T, Christopher S, Tainter CR, Weinstein E, Kim D. Validation of the NUE rule to predict futile resuscitation of out-of-hospital cardiac arrest. Prehosp Emerg Care. 2021;25(5):706-711. doi: 10.1080/10903127.2020.1831666 [DOI] [PubMed] [Google Scholar]
36.Goto Y, Funada A, Maeda T, Okada H, Goto Y. Field termination-of-resuscitation rule for refractory out-of-hospital cardiac arrests in Japan. J Cardiol. 2019;73(3):240-246. doi: 10.1016/j.jjcc.2018.12.002 [DOI] [PubMed] [Google Scholar]
37.Grunau B, Taylor J, Scheuermeyer FX, et al. External validation of the universal termination of resuscitation rule for out-of-hospital cardiac arrest in British Columbia. Ann Emerg Med. 2017;70(3):374-381. doi: 10.1016/j.annemergmed.2017.01.030 [DOI] [PubMed] [Google Scholar]
38.Grunau B, Scheuermeyer F, Kawano T, et al. North American validation of the Bokutoh criteria for withholding professional resuscitation in non-traumatic out-of-hospital cardiac arrest. Resuscitation. 2019;135:51-56. doi: 10.1016/j.resuscitation.2019.01.008 [DOI] [PubMed] [Google Scholar]
39.Harris MI, Crowe RP, Anders J, D’Acunto S, Adelgais KM, Fishe J. Applying a set of termination of resuscitation criteria to paediatric out-of-hospital cardiac arrest. Resuscitation. 2021;169:175-181. doi: 10.1016/j.resuscitation.2021.09.015 [DOI] [PubMed] [Google Scholar]
40.Haukoos JS, Lewis RJ, Niemann JT. Prediction rules for estimating neurologic outcome following out-of-hospital cardiac arrest. Resuscitation. 2004;63(2):145-155. doi: 10.1016/j.resuscitation.2004.04.014 [DOI] [PubMed] [Google Scholar]
41.House M, Gray J, McMeekin P. Reducing the futile transportation of out-of-hospital cardiac arrests: a retrospective validation. Br Paramed J. 2018;3(2):1-6. doi: 10.29045/14784726.2018.09.3.2.1 [DOI] [PMC free article] [PubMed] [Google Scholar]
42.Hreinsson JP, Thorvaldsson AP, Magnusson V, Fridriksson BT, Libungan BG, Karason S. Identifying out-of-hospital cardiac arrest patients with no chance of survival: an independent validation of prediction rules. Resuscitation. 2020;146:19-25. doi: 10.1016/j.resuscitation.2019.11.001 [DOI] [PubMed] [Google Scholar]
43.Hsu SH, Sun JT, Huang EP, et al. The predictive performance of current termination-of-resuscitation rules in patients following out-of-hospital cardiac arrest in Asian countries: a cross-sectional multicentre study. PLoS One. 2022;17(8):e0270986. doi: 10.1371/journal.pone.0270986 [DOI] [PMC free article] [PubMed] [Google Scholar]
44.Jabre P, Bougouin W, Dumas F, et al. Early identification of patients with out-of-hospital cardiac arrest with no chance of survival and consideration for organ donation. Ann Intern Med. 2016;165(11):770-778. doi: 10.7326/M16-0402 [DOI] [PubMed] [Google Scholar]
45.Jordan MR, O’Keefe MF, Weiss D, Cubberley CW, MacLean CD, Wolfson DL. Implementation of the universal BLS termination of resuscitation rule in a rural EMS system. Resuscitation. 2017;118:75-81. doi: 10.1016/j.resuscitation.2017.07.004 [DOI] [PubMed] [Google Scholar]
46.Kajino K, Kitamura T, Iwami T, et al. Current termination of resuscitation (TOR) guidelines predict neurologically favorable outcome in Japan. Resuscitation. 2013;84(1):54-59. doi: 10.1016/j.resuscitation.2012.05.027 [DOI] [PubMed] [Google Scholar]
47.Kashiura M, Hamabe Y, Akashi A, et al. ; SOS-KANTO 2012 Study Group . Applying the termination of resuscitation rules to out-of-hospital cardiac arrests of both cardiac and non-cardiac etiologies: a prospective cohort study. Crit Care. 2016;20:49. doi: 10.1186/s13054-016-1226-4 [DOI] [PMC free article] [PubMed] [Google Scholar]
48.Kim TH, Shin SD, Kim YJ, Kim CH, Kim JE. The scene time interval and basic life support termination of resuscitation rule in adult out-of-hospital cardiac arrest. J Korean Med Sci. 2015;30(1):104-109. doi: 10.3346/jkms.2015.30.1.104 [DOI] [PMC free article] [PubMed] [Google Scholar]
49.Lee DE, Lee MJ, Ahn JY, et al. ; Korean Cardiac Arrest Research Consortium (KoCARC) . New termination-of-resuscitation models and prognostication in out-of-hospital cardiac arrest using electrocardiogram rhythms documented in the field and the emergency department. J Korean Med Sci. 2019;34(17):e134. doi: 10.3346/jkms.2019.34.e134 [DOI] [PMC free article] [PubMed] [Google Scholar]
50.Lin YY, Lai YY, Chang HC, et al. Predictive performances of ALS and BLS termination of resuscitation rules in out-of-hospital cardiac arrest for different resuscitation protocols. BMC Emerg Med. 2022;22(1):53. doi: 10.1186/s12873-022-00606-8 [DOI] [PMC free article] [PubMed] [Google Scholar]
51.Marsden AK, Ng GA, Dalziel K, Cobbe SM. When is it futile for ambulance personnel to initiate cardiopulmonary resuscitation? BMJ. 1995;311(6996):49-51. doi: 10.1136/bmj.311.6996.49 [DOI] [PMC free article] [PubMed] [Google Scholar]
52.Matsui S, Kitamura T, Kurosawa H, et al. Application of adult prehospital resuscitation rules to pediatric out of hospital cardiac arrest. Resuscitation. 2023;184:109684. doi: 10.1016/j.resuscitation.2022.109684 [DOI] [PubMed] [Google Scholar]
53.Morrison LJ, Verbeek PR, Vermeulen MJ, et al. Derivation and evaluation of a termination of resuscitation clinical prediction rule for advanced life support providers. Resuscitation. 2007;74(2):266-275. doi: 10.1016/j.resuscitation.2007.01.009 [DOI] [PubMed] [Google Scholar]
54.Morrison LJ, Verbeek PR, Zhan C, Kiss A, Allan KS. Validation of a universal prehospital termination of resuscitation clinical prediction rule for advanced and basic life support providers. Resuscitation. 2009;80(3):324-328. doi: 10.1016/j.resuscitation.2008.11.014 [DOI] [PubMed] [Google Scholar]
55.Morrison LJ, Eby D, Veigas PV, et al. Implementation trial of the basic life support termination of resuscitation rule: reducing the transport of futile out-of-hospital cardiac arrests. Resuscitation. 2014;85(4):486-491. doi: 10.1016/j.resuscitation.2013.12.013 [DOI] [PubMed] [Google Scholar]
56.Ong MEH, Jaffey J, Stiell I, Nesbitt L; OPALS Study Group . Comparison of termination-of-resuscitation guidelines for basic life support: defibrillator providers in out-of-hospital cardiac arrest. Ann Emerg Med. 2006;47(4):337-343. doi: 10.1016/j.annemergmed.2005.05.012 [DOI] [PubMed] [Google Scholar]
57.Ong ME, Tan EH, Ng FS, et al. ; CARE study group . Comparison of termination-of-resuscitation guidelines for out-of-hospital cardiac arrest in Singapore EMS. Resuscitation. 2007;75(2):244-251. doi: 10.1016/j.resuscitation.2007.04.013 [DOI] [PubMed] [Google Scholar]
58.Park SY, Lim D, Ryu JH, Kim YH, Choi B, Kim SH. Modification of termination of resuscitation rule with compression time interval in South Korea. Sci Rep. 2023;13(1):1403. doi: 10.1038/s41598-023-28789-5 [DOI] [PMC free article] [PubMed] [Google Scholar]
59.Petrie DA, De Maio V, Stiell IG, Dreyer J, Martin M, O’Brien JA. Factors affecting survival after prehospital asystolic cardiac arrest in a basic life support-defibrillation system. CJEM. 2001;3(3):186-192. doi: 10.1017/S1481803500005522 [DOI] [PubMed] [Google Scholar]
60.Sasson C, Hegg AJ, Macy M, Park A, Kellermann A, McNally B; CARES Surveillance Group . Prehospital termination of resuscitation in cases of refractory out-of-hospital cardiac arrest. JAMA. 2008;300(12):1432-1438. doi: 10.1001/jama.300.12.1432 [DOI] [PubMed] [Google Scholar]
61.Shibahashi K, Sugiyama K, Hamabe Y. A potential termination of resuscitation rule for EMS to implement in the field for out-of-hospital cardiac arrest: an observational cohort study. Resuscitation. 2018;130:28-32. doi: 10.1016/j.resuscitation.2018.06.026 [DOI] [PubMed] [Google Scholar]
62.Skrifvars MB, Vayrynen T, Kuisma M, et al. Comparison of Helsinki and European Resuscitation Council “do not attempt to resuscitate” guidelines, and a termination of resuscitation clinical prediction rule for out-of-hospital cardiac arrest patients found in asystole or pulseless electrical activity. Resuscitation. 2010;81(6):679-684. doi: 10.1016/j.resuscitation.2010.01.033 [DOI] [PubMed] [Google Scholar]
63.Smits RLA, Sødergren STF, van Schuppen H, et al. Termination of resuscitation in out-of-hospital cardiac arrest in women and men: an ESCAPE-NET project. Resuscitation. 2023;185:109721. doi: 10.1016/j.resuscitation.2023.109721 [DOI] [PubMed] [Google Scholar]
64.SOS–KANTO 2012 Study Group . A new rule for terminating resuscitation of out-of-hospital cardiac arrest patients in Japan: a prospective study. J Emerg Med. 2017;53(3):345-352. doi: 10.1016/j.jemermed.2017.05.025 [DOI] [PubMed] [Google Scholar]
65.Verbeek PR, Vermeulen MJ, Ali FH, Messenger DW, Summers J, Morrison LJ. Derivation of a termination-of-resuscitation guideline for emergency medical technicians using automated external defibrillators. Acad Emerg Med. 2002;9(7):671-678. [DOI] [PubMed] [Google Scholar]
66.Verhaert DV, Bonnes JL, Nas J, et al. Termination of resuscitation in the prehospital setting: a comparison of decisions in clinical practice vs. recommendations of a termination rule. Resuscitation. 2016;100:60-65. doi: 10.1016/j.resuscitation.2015.12.014 [DOI] [PubMed] [Google Scholar]
67.Yates EJ, Schmidbauer S, Smyth AM, et al. Out-of-hospital cardiac arrest termination of resuscitation with ongoing CPR: an observational study. Resuscitation. 2018;130:21-27. doi: 10.1016/j.resuscitation.2018.06.021 [DOI] [PubMed] [Google Scholar]
68.Yoon JC, Kim YJ, Ahn S, et al. ; Korean Cardiac Arrest Research Consortium KoCARC . Factors for modifying the termination of resuscitation rule in out-of-hospital cardiac arrest. Am Heart J. 2019;213:73-80. doi: 10.1016/j.ahj.2019.04.003 [DOI] [PubMed] [Google Scholar]
69.Deeks JJ, Macaskill P, Irwig L. The performance of tests of publication bias and other sample size effects in systematic reviews of diagnostic test accuracy was assessed. J Clin Epidemiol. 2005;58(9):882-893. doi: 10.1016/j.jclinepi.2005.01.016 [DOI] [PubMed] [Google Scholar]
70.Shibahashi K, Sugiyama K, Hamabe Y. Pediatric out-of-hospital traumatic cardiopulmonary arrest after traffic accidents and termination of resuscitation. Ann Emerg Med. 2020;75(1):57-65. doi: 10.1016/j.annemergmed.2019.05.036 [DOI] [PubMed] [Google Scholar]
71.Schneiderman LJ, Jecker NS, Jonsen AR. Medical futility: its meaning and ethical implications. Ann Intern Med. 1990;112(12):949-954. doi: 10.7326/0003-4819-112-12-949 [DOI] [PubMed] [Google Scholar]
72.Bossaert LL, Perkins GD, Askitopoulou H, et al. ; ethics of resuscitation and end-of-life decisions section Collaborators . European Resuscitation Council Guidelines for Resuscitation 2015: Section 11. The ethics of resuscitation and end-of-life decisions. Resuscitation. 2015;95:302-311. doi: 10.1016/j.resuscitation.2015.07.033 [DOI] [PubMed] [Google Scholar]
73.University of Warwick . Out-of-Hospital Cardiac Arrest Outcomes (OHCAO): annual epidemiology reports. 2023. Accessed May 7, 2023. https://warwick.ac.uk/fac/sci/med/research/ctu/trials/ohcao/publications/epidemiologyreports/
74.UK Government . Equality Act 2010. 2010. Accessed September 1, 2023. https://www.gov.uk/guidance/equality-act-2010-guidance
75.Perkins GD, Lall R, Quinn T, et al. ; PARAMEDIC trial collaborators . Mechanical versus manual chest compression for out-of-hospital cardiac arrest (PARAMEDIC): a pragmatic, cluster randomised controlled trial. Lancet. 2015;385(9972):947-955. doi: 10.1016/S0140-6736(14)61886-9 [DOI] [PubMed] [Google Scholar]
76.Dyson K, Bray JE, Smith K, Bernard S, Straney L, Finn J. Paramedic exposure to out-of-hospital cardiac arrest resuscitation is associated with patient survival. Circ Cardiovasc Qual Outcomes. 2016;9(2):154-160. doi: 10.1161/CIRCOUTCOMES.115.002317 [DOI] [PubMed] [Google Scholar]
77.Tuttle JE, Hubble MW. Paramedic out-of-hospital cardiac arrest case volume is a predictor of return of spontaneous circulation. West J Emerg Med. 2018;19(4):654-659. doi: 10.5811/westjem.2018.3.37051 [DOI] [PMC free article] [PubMed] [Google Scholar]
78.Grunau BE, Kime N, Rea T, et al. The relationship between intra-arrest transport and survival for refractory out-of-hospital cardiac arrest. Circulation. 2017;136(suppl 1):A13973. [Google Scholar]
79.Wampler DA, Collett L, Manifold CA, Velasquez C, McMullan JT. Cardiac arrest survival is rare without prehospital return of spontaneous circulation. Prehosp Emerg Care. 2012;16(4):451-455. doi: 10.3109/10903127.2012.695435 [DOI] [PubMed] [Google Scholar]
80.Wyckoff MH, Greif R, Morley PT, et al. ; Collaborators . 2022 International Consensus on Cardiopulmonary Resuscitation and Emergency Cardiovascular Care Science With Treatment Recommendations: summary from the Basic Life Support; Advanced Life Support; Pediatric Life Support; Neonatal Life Support; Education, Implementation, and Teams; and First Aid Task Forces. Resuscitation. 2022;181:208-288. doi: 10.1016/j.resuscitation.2022.10.005 [DOI] [PubMed] [Google Scholar]
81.Yilmaz A, Serinken M, Dal O, Yaylacı S, Karcioglu O. Work-related injuries among emergency medical technicians in western Turkey. Prehosp Disaster Med. 2016;31(5):505-508. doi: 10.1017/S1049023X16000741 [DOI] [PubMed] [Google Scholar]
82.Gülen B, Serinken M, Hatipoğlu C, et al. Work-related injuries sustained by emergency medical technicians and paramedics in Turkey. Ulus Travma Acil Cerrahi Derg. 2016;22(2):145-149. [DOI] [PubMed] [Google Scholar]
83.Sanddal ND, Albert S, Hansen JD, Kupas DF. Contributing factors and issues associated with rural ambulance crashes: literature review and annotated bibliography. Prehosp Emerg Care. 2008;12(2):257-267. doi: 10.1080/10903120801907661 [DOI] [PubMed] [Google Scholar]
84.Ray AF, Kupas DF. Comparison of crashes involving ambulances with those of similar-sized vehicles. Prehosp Emerg Care. 2005;9(4):412-415. doi: 10.1080/10903120500253813 [DOI] [PubMed] [Google Scholar]
85.Achana F, Petrou S, Madan J, et al. ; PARAMEDIC2 Collaborators . Cost-effectiveness of adrenaline for out-of-hospital cardiac arrest. Crit Care. 2020;24(1):579. doi: 10.1186/s13054-020-03271-0 [DOI] [PMC free article] [PubMed] [Google Scholar]
86.Perkins GD, Callaway CW, Haywood K, et al. Brain injury after cardiac arrest. Lancet. 2021;398(10307):1269-1278. doi: 10.1016/S0140-6736(21)00953-3 [DOI] [PubMed] [Google Scholar]

Associated Data

This section collects any data citations, data availability statements, or supplementary materials included in this article.

Supplementary Materials

Supplement 1.

eTable 1. PICOST Criteria

eTable 2. MEDLINE Search Strategy

eTable 3. Embase Search Strategy

eTable 4. CINAHL Search Strategy

eTable 5. Cochrane Search Strategy

eTable 6. Web of Science Search Strategy

eTable 7. Derivation Studies Paired Sensitivity and Specificity

eTable 8. External Validation Studies Paired Sensitivity and Specificity

eTable 9. Clinical Studies Paired Sensitivity and Specificity

eTable 10. Summary of Included Studies

eTable 11. Description of Termination of Resuscitation (TOR) Rules

eFigure 1. QUADAS-2 Summary Risk of Bias and Applicability

eFigure 2. Study Risk of Bias

eFigure 3. Deeks Funnel Plot Asymmetry Test

eFigure 4. Bivariate Summary Receiver Operating Characteristic (SROC) Curves

eReferences

jamanetwopen-e2420040-s001.pdf^{(698.1KB, pdf)}

Supplement 2.

Data Sharing Statement

jamanetwopen-e2420040-s002.pdf^{(15.9KB, pdf)}

[zoi240647r1] 1.Gräsner JT, Herlitz J, Tjelmeland IBM, et al. European Resuscitation Council Guidelines 2021: epidemiology of cardiac arrest in Europe. Resuscitation. 2021;161:61-79. doi: 10.1016/j.resuscitation.2021.02.007 [DOI] [PubMed] [Google Scholar]

[zoi240647r2] 2.Anderson NE, Gott M, Slark J. Commence, continue, withhold or terminate? a systematic review of decision-making in out-of-hospital cardiac arrest. Eur J Emerg Med. 2017;24(2):80-86. doi: 10.1097/MEJ.0000000000000407 [DOI] [PubMed] [Google Scholar]

[zoi240647r3] 3.Hick JL, Mahoney BD, Lappe M. Factors influencing hospital transport of patients in continuing cardiac arrest. Ann Emerg Med. 1998;32(1):19-25. doi: 10.1016/S0196-0644(98)70094-0 [DOI] [PubMed] [Google Scholar]

[zoi240647r4] 4.Mao DRH, Ong MEH, Bang C, Salim MDT, Ng YY, Lie DA. Psychological comfort of paramedics with field death pronouncement: a national Asian study to prepare paramedics for field termination of resuscitation. Prehosp Emerg Care. 2018;22(2):260-265. doi: 10.1080/10903127.2017.1376132 [DOI] [PubMed] [Google Scholar]

[zoi240647r5] 5.Grudzen CR, Timmermans S, Koenig WJ, et al. Paramedic and emergency medical technicians views on opportunities and challenges when forgoing and halting resuscitation in the field. Acad Emerg Med. 2009;16(6):532-538. doi: 10.1111/j.1553-2712.2009.00427.x [DOI] [PubMed] [Google Scholar]

[zoi240647r6] 6.National Association of EMS Physicians . Termination of resuscitation in nontraumatic cardiopulmonary arrest. Prehosp Emerg Care. 2011;15(4):542. doi: 10.3109/10903127.2011.598621 [DOI] [PubMed] [Google Scholar]

[zoi240647r7] 7.National Association of EMS Physicians; American College of Surgeons Committee on Trauma . Withholding of resuscitation for adult traumatic cardiopulmonary arrest. Prehosp Emerg Care. 2013;17(2):291. doi: 10.3109/10903127.2012.755586 [DOI] [PubMed] [Google Scholar]

[zoi240647r8] 8.Association of Ambulance Services Chief Executives . UK Ambulance Services Clinical Practice Guidelines. Class Professional Publishing; 2021. [Google Scholar]

[zoi240647r9] 9.Greif R, Bhanji F, Bigham BL, et al. ; Education, Implementation, and Teams Collaborators . Education, implementation, and teams: 2020 international consensus on cardiopulmonary resuscitation and emergency cardiovascular care science with treatment recommendations. Circulation. 2020;142(16 suppl 1):S222-S283. doi: 10.1161/CIR.0000000000000896 [DOI] [PubMed] [Google Scholar]

[zoi240647r10] 10.Lauridsen KG, Baldi E, Smyth M, et al. ; Education Implementation and Team Task Force of the International Liaison Committee on Resuscitation (ILCOR) . Clinical decision rules for termination of resuscitation during in-hospital cardiac arrest: a systematic review of diagnostic test accuracy studies. Resuscitation. 2021;158:23-29. doi: 10.1016/j.resuscitation.2020.10.036 [DOI] [PubMed] [Google Scholar]

[zoi240647r11] 11.Leeflang MM, Deeks JJ, Takwoingi Y, Macaskill P. Cochrane diagnostic test accuracy reviews. Syst Rev. 2013;2(1):82. doi: 10.1186/2046-4053-2-82 [DOI] [PMC free article] [PubMed] [Google Scholar]

[zoi240647r12] 12.Morrison LJ. Prehospital termination of resuscitation rule. Curr Opin Crit Care. 2019;25(3):199-203. doi: 10.1097/MCC.0000000000000614 [DOI] [PubMed] [Google Scholar]

[zoi240647r13] 13.Ebell MH, Vellinga A, Masterson S, Yun P. Meta-analysis of the accuracy of termination of resuscitation rules for out-of-hospital cardiac arrest. Emerg Med J. 2019;36(8):479-484. doi: 10.1136/emermed-2018-207833 [DOI] [PubMed] [Google Scholar]

[zoi240647r14] 14.Nas J, Kleinnibbelink G, Hannink G, et al. Diagnostic performance of the basic and advanced life support termination of resuscitation rules: a systematic review and diagnostic meta-analysis. Resuscitation. 2020;148:3-13. doi: 10.1016/j.resuscitation.2019.12.016 [DOI] [PubMed] [Google Scholar]

[zoi240647r15] 15.Sterne JA, Hernán MA, Reeves BC, et al. ROBINS-I: a tool for assessing risk of bias in non-randomised studies of interventions. BMJ. 2016;355:i4919. doi: 10.1136/bmj.i4919 [DOI] [PMC free article] [PubMed] [Google Scholar]

[zoi240647r16] 16.Sterne JAC, Savović J, Page MJ, et al. RoB 2: a revised tool for assessing risk of bias in randomised trials. BMJ. 2019;366:l4898. doi: 10.1136/bmj.l4898 [DOI] [PubMed] [Google Scholar]

[zoi240647r17] 17.Whiting PF, Rutjes AW, Westwood ME, et al. ; QUADAS-2 Group . QUADAS-2: a revised tool for the quality assessment of diagnostic accuracy studies. Ann Intern Med. 2011;155(8):529-536. doi: 10.7326/0003-4819-155-8-201110180-00009 [DOI] [PubMed] [Google Scholar]

[zoi240647r18] 18.Macaskill PTY, Deeks JJ, Gatsonis C. Understanding meta-analysis. In: Deeks JJBP, Leeflang MM, Takwoingi Y, eds. Cochrane Handbook for Systematic Reviews of Diagnostic Test Accuracy (Version 2). Cochrane Collaboration; 2022:chap 9. [Google Scholar]

[zoi240647r19] 19.Viechtbauer W, Viechtbauer MW. Package metafor. Comprehensive R Archive Network. 2015. Accessed November 10, 2023. http://cran.r-project.org/web/packages/metafor/metafor.pdf

[zoi240647r20] 20.Guyatt GH, Oxman AD, Vist GE, et al. ; GRADE Working Group . GRADE: an emerging consensus on rating quality of evidence and strength of recommendations. BMJ. 2008;336(7650):924-926. doi: 10.1136/bmj.39489.470347.AD [DOI] [PMC free article] [PubMed] [Google Scholar]

[zoi240647r21] 21.Allaire J. RStudio: Integrated Development Environment for R. R Project for Statistical Computing; 2012. [Google Scholar]

[zoi240647r22] 22.Stevenson M, Stevenson MM. BiasedUrn I. Package epiR; 2018.

[zoi240647r23] 23.Schwarzer G. Meta: an R package for meta-analysis. R News. 2007;7(3):40-45. [Google Scholar]

[zoi240647r24] 24.Reitsma JB, Glas AS, Rutjes AW, Scholten RJ, Bossuyt PM, Zwinderman AH. Bivariate analysis of sensitivity and specificity produces informative summary measures in diagnostic reviews. J Clin Epidemiol. 2005;58(10):982-990. doi: 10.1016/j.jclinepi.2005.02.022 [DOI] [PubMed] [Google Scholar]

[zoi240647r25] 25.McMaster University . Evidence Prime. GRADEpro Guideline Development Tool. 2022. Accessed September 1, 2023. https://www.gradepro.org

[zoi240647r26] 26.Bonnin MJ, Pepe PE, Kimball KT, Clark PS Jr. Distinct criteria for termination of resuscitation in the out-of-hospital setting. JAMA. 1993;270(12):1457-1462. doi: 10.1001/jama.1993.03510120079035 [DOI] [PubMed] [Google Scholar]

[zoi240647r27] 27.Cheong RW, Li H, Doctor NE, et al. Termination of resuscitation rules to predict neurological outcomes in out-of-hospital cardiac arrest for an intermediate life support prehospital system. Prehosp Emerg Care. 2016;20(5):623-629. doi: 10.3109/10903127.2016.1162886 [DOI] [PubMed] [Google Scholar]

[zoi240647r28] 28.Chiang WC, Ko PC, Chang AM, et al. Predictive performance of universal termination of resuscitation rules in an Asian community: are they accurate enough? Emerg Med J. 2015;32(4):318-323. doi: 10.1136/emermed-2013-203289 [DOI] [PMC free article] [PubMed] [Google Scholar]

[zoi240647r29] 29.Chiang WC, Huang YS, Hsu SH, et al. Performance of a simplified termination of resuscitation rule for adult traumatic cardiopulmonary arrest in the prehospital setting. Emerg Med J. 2017;34(1):39-45. doi: 10.1136/emermed-2014-204493 [DOI] [PubMed] [Google Scholar]

[zoi240647r30] 30.Cone DC, Bailey ED, Spackman AB. The safety of a field termination-of-resuscitation protocol. Prehosp Emerg Care. 2005;9(3):276-281. doi: 10.1080/10903120590961996 [DOI] [PubMed] [Google Scholar]

[zoi240647r31] 31.Diskin FJ, Camp-Rogers T, Peberdy MA, Ornato JP, Kurz MC. External validation of termination of resuscitation guidelines in the setting of intra-arrest cold saline, mechanical CPR, and comprehensive post resuscitation care. Resuscitation. 2014;85(7):910-914. doi: 10.1016/j.resuscitation.2014.02.028 [DOI] [PubMed] [Google Scholar]

[zoi240647r32] 32.Drennan IR, Lin S, Sidalak DE, Morrison LJ. Survival rates in out-of-hospital cardiac arrest patients transported without prehospital return of spontaneous circulation: an observational cohort study. Resuscitation. 2014;85(11):1488-1493. doi: 10.1016/j.resuscitation.2014.07.011 [DOI] [PubMed] [Google Scholar]

[zoi240647r33] 33.Fukuda T, Ohashi N, Matsubara T, et al. Applicability of the prehospital termination of resuscitation rule in an area dense with hospitals in Tokyo: a single-center, retrospective, observational study: is the pre hospital TOR rule applicable in Tokyo? Am J Emerg Med. 2014;32(2):144-149. doi: 10.1016/j.ajem.2013.10.032 [DOI] [PubMed] [Google Scholar]

[zoi240647r34] 34.Glober NK, Tainter CR, Abramson TM, Staats K, Gilbert G, Kim D. A simple decision rule predicts futile resuscitation of out-of-hospital cardiac arrest. Resuscitation. 2019;142:8-13. doi: 10.1016/j.resuscitation.2019.06.011 [DOI] [PubMed] [Google Scholar]

[zoi240647r35] 35.Glober NK, Lardaro T, Christopher S, Tainter CR, Weinstein E, Kim D. Validation of the NUE rule to predict futile resuscitation of out-of-hospital cardiac arrest. Prehosp Emerg Care. 2021;25(5):706-711. doi: 10.1080/10903127.2020.1831666 [DOI] [PubMed] [Google Scholar]

[zoi240647r36] 36.Goto Y, Funada A, Maeda T, Okada H, Goto Y. Field termination-of-resuscitation rule for refractory out-of-hospital cardiac arrests in Japan. J Cardiol. 2019;73(3):240-246. doi: 10.1016/j.jjcc.2018.12.002 [DOI] [PubMed] [Google Scholar]

[zoi240647r37] 37.Grunau B, Taylor J, Scheuermeyer FX, et al. External validation of the universal termination of resuscitation rule for out-of-hospital cardiac arrest in British Columbia. Ann Emerg Med. 2017;70(3):374-381. doi: 10.1016/j.annemergmed.2017.01.030 [DOI] [PubMed] [Google Scholar]

[zoi240647r38] 38.Grunau B, Scheuermeyer F, Kawano T, et al. North American validation of the Bokutoh criteria for withholding professional resuscitation in non-traumatic out-of-hospital cardiac arrest. Resuscitation. 2019;135:51-56. doi: 10.1016/j.resuscitation.2019.01.008 [DOI] [PubMed] [Google Scholar]

[zoi240647r39] 39.Harris MI, Crowe RP, Anders J, D’Acunto S, Adelgais KM, Fishe J. Applying a set of termination of resuscitation criteria to paediatric out-of-hospital cardiac arrest. Resuscitation. 2021;169:175-181. doi: 10.1016/j.resuscitation.2021.09.015 [DOI] [PubMed] [Google Scholar]

[zoi240647r40] 40.Haukoos JS, Lewis RJ, Niemann JT. Prediction rules for estimating neurologic outcome following out-of-hospital cardiac arrest. Resuscitation. 2004;63(2):145-155. doi: 10.1016/j.resuscitation.2004.04.014 [DOI] [PubMed] [Google Scholar]

[zoi240647r41] 41.House M, Gray J, McMeekin P. Reducing the futile transportation of out-of-hospital cardiac arrests: a retrospective validation. Br Paramed J. 2018;3(2):1-6. doi: 10.29045/14784726.2018.09.3.2.1 [DOI] [PMC free article] [PubMed] [Google Scholar]

[zoi240647r42] 42.Hreinsson JP, Thorvaldsson AP, Magnusson V, Fridriksson BT, Libungan BG, Karason S. Identifying out-of-hospital cardiac arrest patients with no chance of survival: an independent validation of prediction rules. Resuscitation. 2020;146:19-25. doi: 10.1016/j.resuscitation.2019.11.001 [DOI] [PubMed] [Google Scholar]

[zoi240647r43] 43.Hsu SH, Sun JT, Huang EP, et al. The predictive performance of current termination-of-resuscitation rules in patients following out-of-hospital cardiac arrest in Asian countries: a cross-sectional multicentre study. PLoS One. 2022;17(8):e0270986. doi: 10.1371/journal.pone.0270986 [DOI] [PMC free article] [PubMed] [Google Scholar]

[zoi240647r44] 44.Jabre P, Bougouin W, Dumas F, et al. Early identification of patients with out-of-hospital cardiac arrest with no chance of survival and consideration for organ donation. Ann Intern Med. 2016;165(11):770-778. doi: 10.7326/M16-0402 [DOI] [PubMed] [Google Scholar]

[zoi240647r45] 45.Jordan MR, O’Keefe MF, Weiss D, Cubberley CW, MacLean CD, Wolfson DL. Implementation of the universal BLS termination of resuscitation rule in a rural EMS system. Resuscitation. 2017;118:75-81. doi: 10.1016/j.resuscitation.2017.07.004 [DOI] [PubMed] [Google Scholar]

[zoi240647r46] 46.Kajino K, Kitamura T, Iwami T, et al. Current termination of resuscitation (TOR) guidelines predict neurologically favorable outcome in Japan. Resuscitation. 2013;84(1):54-59. doi: 10.1016/j.resuscitation.2012.05.027 [DOI] [PubMed] [Google Scholar]

[zoi240647r47] 47.Kashiura M, Hamabe Y, Akashi A, et al. ; SOS-KANTO 2012 Study Group . Applying the termination of resuscitation rules to out-of-hospital cardiac arrests of both cardiac and non-cardiac etiologies: a prospective cohort study. Crit Care. 2016;20:49. doi: 10.1186/s13054-016-1226-4 [DOI] [PMC free article] [PubMed] [Google Scholar]

[zoi240647r48] 48.Kim TH, Shin SD, Kim YJ, Kim CH, Kim JE. The scene time interval and basic life support termination of resuscitation rule in adult out-of-hospital cardiac arrest. J Korean Med Sci. 2015;30(1):104-109. doi: 10.3346/jkms.2015.30.1.104 [DOI] [PMC free article] [PubMed] [Google Scholar]

[zoi240647r49] 49.Lee DE, Lee MJ, Ahn JY, et al. ; Korean Cardiac Arrest Research Consortium (KoCARC) . New termination-of-resuscitation models and prognostication in out-of-hospital cardiac arrest using electrocardiogram rhythms documented in the field and the emergency department. J Korean Med Sci. 2019;34(17):e134. doi: 10.3346/jkms.2019.34.e134 [DOI] [PMC free article] [PubMed] [Google Scholar]

[zoi240647r50] 50.Lin YY, Lai YY, Chang HC, et al. Predictive performances of ALS and BLS termination of resuscitation rules in out-of-hospital cardiac arrest for different resuscitation protocols. BMC Emerg Med. 2022;22(1):53. doi: 10.1186/s12873-022-00606-8 [DOI] [PMC free article] [PubMed] [Google Scholar]

[zoi240647r51] 51.Marsden AK, Ng GA, Dalziel K, Cobbe SM. When is it futile for ambulance personnel to initiate cardiopulmonary resuscitation? BMJ. 1995;311(6996):49-51. doi: 10.1136/bmj.311.6996.49 [DOI] [PMC free article] [PubMed] [Google Scholar]

[zoi240647r52] 52.Matsui S, Kitamura T, Kurosawa H, et al. Application of adult prehospital resuscitation rules to pediatric out of hospital cardiac arrest. Resuscitation. 2023;184:109684. doi: 10.1016/j.resuscitation.2022.109684 [DOI] [PubMed] [Google Scholar]

[zoi240647r53] 53.Morrison LJ, Verbeek PR, Vermeulen MJ, et al. Derivation and evaluation of a termination of resuscitation clinical prediction rule for advanced life support providers. Resuscitation. 2007;74(2):266-275. doi: 10.1016/j.resuscitation.2007.01.009 [DOI] [PubMed] [Google Scholar]

[zoi240647r54] 54.Morrison LJ, Verbeek PR, Zhan C, Kiss A, Allan KS. Validation of a universal prehospital termination of resuscitation clinical prediction rule for advanced and basic life support providers. Resuscitation. 2009;80(3):324-328. doi: 10.1016/j.resuscitation.2008.11.014 [DOI] [PubMed] [Google Scholar]

[zoi240647r55] 55.Morrison LJ, Eby D, Veigas PV, et al. Implementation trial of the basic life support termination of resuscitation rule: reducing the transport of futile out-of-hospital cardiac arrests. Resuscitation. 2014;85(4):486-491. doi: 10.1016/j.resuscitation.2013.12.013 [DOI] [PubMed] [Google Scholar]

[zoi240647r56] 56.Ong MEH, Jaffey J, Stiell I, Nesbitt L; OPALS Study Group . Comparison of termination-of-resuscitation guidelines for basic life support: defibrillator providers in out-of-hospital cardiac arrest. Ann Emerg Med. 2006;47(4):337-343. doi: 10.1016/j.annemergmed.2005.05.012 [DOI] [PubMed] [Google Scholar]

[zoi240647r57] 57.Ong ME, Tan EH, Ng FS, et al. ; CARE study group . Comparison of termination-of-resuscitation guidelines for out-of-hospital cardiac arrest in Singapore EMS. Resuscitation. 2007;75(2):244-251. doi: 10.1016/j.resuscitation.2007.04.013 [DOI] [PubMed] [Google Scholar]

[zoi240647r58] 58.Park SY, Lim D, Ryu JH, Kim YH, Choi B, Kim SH. Modification of termination of resuscitation rule with compression time interval in South Korea. Sci Rep. 2023;13(1):1403. doi: 10.1038/s41598-023-28789-5 [DOI] [PMC free article] [PubMed] [Google Scholar]

[zoi240647r59] 59.Petrie DA, De Maio V, Stiell IG, Dreyer J, Martin M, O’Brien JA. Factors affecting survival after prehospital asystolic cardiac arrest in a basic life support-defibrillation system. CJEM. 2001;3(3):186-192. doi: 10.1017/S1481803500005522 [DOI] [PubMed] [Google Scholar]

[zoi240647r60] 60.Sasson C, Hegg AJ, Macy M, Park A, Kellermann A, McNally B; CARES Surveillance Group . Prehospital termination of resuscitation in cases of refractory out-of-hospital cardiac arrest. JAMA. 2008;300(12):1432-1438. doi: 10.1001/jama.300.12.1432 [DOI] [PubMed] [Google Scholar]

[zoi240647r61] 61.Shibahashi K, Sugiyama K, Hamabe Y. A potential termination of resuscitation rule for EMS to implement in the field for out-of-hospital cardiac arrest: an observational cohort study. Resuscitation. 2018;130:28-32. doi: 10.1016/j.resuscitation.2018.06.026 [DOI] [PubMed] [Google Scholar]

[zoi240647r62] 62.Skrifvars MB, Vayrynen T, Kuisma M, et al. Comparison of Helsinki and European Resuscitation Council “do not attempt to resuscitate” guidelines, and a termination of resuscitation clinical prediction rule for out-of-hospital cardiac arrest patients found in asystole or pulseless electrical activity. Resuscitation. 2010;81(6):679-684. doi: 10.1016/j.resuscitation.2010.01.033 [DOI] [PubMed] [Google Scholar]

[zoi240647r63] 63.Smits RLA, Sødergren STF, van Schuppen H, et al. Termination of resuscitation in out-of-hospital cardiac arrest in women and men: an ESCAPE-NET project. Resuscitation. 2023;185:109721. doi: 10.1016/j.resuscitation.2023.109721 [DOI] [PubMed] [Google Scholar]

[zoi240647r64] 64.SOS–KANTO 2012 Study Group . A new rule for terminating resuscitation of out-of-hospital cardiac arrest patients in Japan: a prospective study. J Emerg Med. 2017;53(3):345-352. doi: 10.1016/j.jemermed.2017.05.025 [DOI] [PubMed] [Google Scholar]

[zoi240647r65] 65.Verbeek PR, Vermeulen MJ, Ali FH, Messenger DW, Summers J, Morrison LJ. Derivation of a termination-of-resuscitation guideline for emergency medical technicians using automated external defibrillators. Acad Emerg Med. 2002;9(7):671-678. [DOI] [PubMed] [Google Scholar]

[zoi240647r66] 66.Verhaert DV, Bonnes JL, Nas J, et al. Termination of resuscitation in the prehospital setting: a comparison of decisions in clinical practice vs. recommendations of a termination rule. Resuscitation. 2016;100:60-65. doi: 10.1016/j.resuscitation.2015.12.014 [DOI] [PubMed] [Google Scholar]

[zoi240647r67] 67.Yates EJ, Schmidbauer S, Smyth AM, et al. Out-of-hospital cardiac arrest termination of resuscitation with ongoing CPR: an observational study. Resuscitation. 2018;130:21-27. doi: 10.1016/j.resuscitation.2018.06.021 [DOI] [PubMed] [Google Scholar]

[zoi240647r68] 68.Yoon JC, Kim YJ, Ahn S, et al. ; Korean Cardiac Arrest Research Consortium KoCARC . Factors for modifying the termination of resuscitation rule in out-of-hospital cardiac arrest. Am Heart J. 2019;213:73-80. doi: 10.1016/j.ahj.2019.04.003 [DOI] [PubMed] [Google Scholar]

[zoi240647r69] 69.Deeks JJ, Macaskill P, Irwig L. The performance of tests of publication bias and other sample size effects in systematic reviews of diagnostic test accuracy was assessed. J Clin Epidemiol. 2005;58(9):882-893. doi: 10.1016/j.jclinepi.2005.01.016 [DOI] [PubMed] [Google Scholar]

[zoi240647r70] 70.Shibahashi K, Sugiyama K, Hamabe Y. Pediatric out-of-hospital traumatic cardiopulmonary arrest after traffic accidents and termination of resuscitation. Ann Emerg Med. 2020;75(1):57-65. doi: 10.1016/j.annemergmed.2019.05.036 [DOI] [PubMed] [Google Scholar]

[zoi240647r71] 71.Schneiderman LJ, Jecker NS, Jonsen AR. Medical futility: its meaning and ethical implications. Ann Intern Med. 1990;112(12):949-954. doi: 10.7326/0003-4819-112-12-949 [DOI] [PubMed] [Google Scholar]

[zoi240647r72] 72.Bossaert LL, Perkins GD, Askitopoulou H, et al. ; ethics of resuscitation and end-of-life decisions section Collaborators . European Resuscitation Council Guidelines for Resuscitation 2015: Section 11. The ethics of resuscitation and end-of-life decisions. Resuscitation. 2015;95:302-311. doi: 10.1016/j.resuscitation.2015.07.033 [DOI] [PubMed] [Google Scholar]

[zoi240647r73] 73.University of Warwick . Out-of-Hospital Cardiac Arrest Outcomes (OHCAO): annual epidemiology reports. 2023. Accessed May 7, 2023. https://warwick.ac.uk/fac/sci/med/research/ctu/trials/ohcao/publications/epidemiologyreports/

[zoi240647r74] 74.UK Government . Equality Act 2010. 2010. Accessed September 1, 2023. https://www.gov.uk/guidance/equality-act-2010-guidance

[zoi240647r75] 75.Perkins GD, Lall R, Quinn T, et al. ; PARAMEDIC trial collaborators . Mechanical versus manual chest compression for out-of-hospital cardiac arrest (PARAMEDIC): a pragmatic, cluster randomised controlled trial. Lancet. 2015;385(9972):947-955. doi: 10.1016/S0140-6736(14)61886-9 [DOI] [PubMed] [Google Scholar]

[zoi240647r76] 76.Dyson K, Bray JE, Smith K, Bernard S, Straney L, Finn J. Paramedic exposure to out-of-hospital cardiac arrest resuscitation is associated with patient survival. Circ Cardiovasc Qual Outcomes. 2016;9(2):154-160. doi: 10.1161/CIRCOUTCOMES.115.002317 [DOI] [PubMed] [Google Scholar]

[zoi240647r77] 77.Tuttle JE, Hubble MW. Paramedic out-of-hospital cardiac arrest case volume is a predictor of return of spontaneous circulation. West J Emerg Med. 2018;19(4):654-659. doi: 10.5811/westjem.2018.3.37051 [DOI] [PMC free article] [PubMed] [Google Scholar]

[zoi240647r78] 78.Grunau BE, Kime N, Rea T, et al. The relationship between intra-arrest transport and survival for refractory out-of-hospital cardiac arrest. Circulation. 2017;136(suppl 1):A13973. [Google Scholar]

[zoi240647r79] 79.Wampler DA, Collett L, Manifold CA, Velasquez C, McMullan JT. Cardiac arrest survival is rare without prehospital return of spontaneous circulation. Prehosp Emerg Care. 2012;16(4):451-455. doi: 10.3109/10903127.2012.695435 [DOI] [PubMed] [Google Scholar]

[zoi240647r80] 80.Wyckoff MH, Greif R, Morley PT, et al. ; Collaborators . 2022 International Consensus on Cardiopulmonary Resuscitation and Emergency Cardiovascular Care Science With Treatment Recommendations: summary from the Basic Life Support; Advanced Life Support; Pediatric Life Support; Neonatal Life Support; Education, Implementation, and Teams; and First Aid Task Forces. Resuscitation. 2022;181:208-288. doi: 10.1016/j.resuscitation.2022.10.005 [DOI] [PubMed] [Google Scholar]

[zoi240647r81] 81.Yilmaz A, Serinken M, Dal O, Yaylacı S, Karcioglu O. Work-related injuries among emergency medical technicians in western Turkey. Prehosp Disaster Med. 2016;31(5):505-508. doi: 10.1017/S1049023X16000741 [DOI] [PubMed] [Google Scholar]

[zoi240647r82] 82.Gülen B, Serinken M, Hatipoğlu C, et al. Work-related injuries sustained by emergency medical technicians and paramedics in Turkey. Ulus Travma Acil Cerrahi Derg. 2016;22(2):145-149. [DOI] [PubMed] [Google Scholar]

[zoi240647r83] 83.Sanddal ND, Albert S, Hansen JD, Kupas DF. Contributing factors and issues associated with rural ambulance crashes: literature review and annotated bibliography. Prehosp Emerg Care. 2008;12(2):257-267. doi: 10.1080/10903120801907661 [DOI] [PubMed] [Google Scholar]

[zoi240647r84] 84.Ray AF, Kupas DF. Comparison of crashes involving ambulances with those of similar-sized vehicles. Prehosp Emerg Care. 2005;9(4):412-415. doi: 10.1080/10903120500253813 [DOI] [PubMed] [Google Scholar]

[zoi240647r85] 85.Achana F, Petrou S, Madan J, et al. ; PARAMEDIC2 Collaborators . Cost-effectiveness of adrenaline for out-of-hospital cardiac arrest. Crit Care. 2020;24(1):579. doi: 10.1186/s13054-020-03271-0 [DOI] [PMC free article] [PubMed] [Google Scholar]

[zoi240647r86] 86.Perkins GD, Callaway CW, Haywood K, et al. Brain injury after cardiac arrest. Lancet. 2021;398(10307):1269-1278. doi: 10.1016/S0140-6736(21)00953-3 [DOI] [PubMed] [Google Scholar]

PERMALINK

Termination of Resuscitation Rules and Survival Among Patients With Out-of-Hospital Cardiac Arrest

Michael A Smyth, PhD

Imogen Gunson, MSc

Alison Coppola, MClinRes

Samantha Johnson, MA

Robert Greif, PhD

Kasper G Lauridsen, MD, PhD

Sian Taylor-Philips, PhD

Gavin D Perkins, MD

Key Points

Question

Findings

Meaning

Abstract

Importance

Objective

Data Sources

Study Selection

Data Extraction and Synthesis

Main Outcomes and Measures

Results

Conclusions and Relevance

Introduction

Methods

Eligibility Criteria

Literature Search

Study Selection

Data Extraction

Quality Assessment

Outcomes

Statistical Analysis

Results

Figure 1. Study Flow Diagram.

Study Characteristics

Risk of Bias

Publication Bias

Derivation Studies

External Validation Studies

Figure 2. Bivariate Summary Receiver Operating Characteristic (SROC) Curves.

Table 1. Pooled Summary Estimates for External Data Validation Studies.

Table 2. Summary of Findings for Estimates of Effect for TOR Rules at Different Survival Ratesa.

Table 3. Estimated Performance of TOR Rule Use at Different Prevalence Ratesa.

Clinical Studies

Discussion

Limitations

Conclusions

References

Associated Data

Supplementary Materials

ACTIONS

PERMALINK

RESOURCES

Similar articles

Cited by other articles

Links to NCBI Databases

Table 2. Summary of Findings for Estimates of Effect for TOR Rules at Different Survival Rates^a.

Table 3. Estimated Performance of TOR Rule Use at Different Prevalence Rates^a.