Clinical Predictive Models of Sudden Cardiac Arrest: A Survey of the Current Science and Analysis of Model Performances

Abstract

Background

More than 500 000 sudden cardiac arrests (SCAs) occur annually in the United States. Clinical predictive models (CPMs) may be helpful tools to differentiate between patients who are likely to survive or have good neurologic recovery and those who are not. However, which CPMs are most reliable for discriminating between outcomes in SCA is not known.

Methods and Results

We performed a systematic review of the literature using the Tufts PACE (Predictive Analytics and Comparative Effectiveness) CPM Registry through February 1, 2020, and identified 81 unique CPMs of SCA and 62 subsequent external validation studies. Initial cardiac rhythm, age, and duration of cardiopulmonary resuscitation were the 3 most commonly used predictive variables. Only 33 of the 81 novel SCA CPMs (41%) were validated at least once. Of 81 novel SCA CPMs, 56 (69%) and 61 of 62 validation studies (98%) reported discrimination, with median c‐statistics of 0.84 and 0.81, respectively. Calibration was reported in only 29 of 62 validation studies (41.9%). For those novel models that both reported discrimination and were validated (26 models), the median percentage change in discrimination was −1.6%. We identified 3 CPMs that had undergone at least 3 external validation studies: the out‐of‐hospital cardiac arrest score (9 validations; median c‐statistic, 0.79), the cardiac arrest hospital prognosis score (6 validations; median c‐statistic, 0.83), and the good outcome following attempted resuscitation score (6 validations; median c‐statistic, 0.76).

Conclusions

Although only a small number of SCA CPMs have been rigorously validated, the ones that have been demonstrate good discrimination.

Nonstandard Abbreviations and Acronyms

SCA

sudden cardiac arrest

CPM

clinical predictive model

OHCA

out‐of‐hospital cardiac arrest

IHCA

in‐hospital cardiac arrest

IQR

interquartile range

Clinical Perspective

What Is New?

• Sudden cardiac arrest (SCA) is a common but disastrous event that can leave both physicians and surrogate decision makers in the difficult position of determining treatment plans in the setting of unclear prognosis; clinical predictive models represent objective, quantitative tools for guiding this type of decision on the behalf of critically ill victims of SCA.

• There are many unique clinical predictive models available for use in SCA, and these tools generally have excellent ability to discriminate between those patients who are likely and those who are unlikely to survive with good neurologic outcome following SCA; only a few of these models have been rigorously validated.

What Are the Clinical Implications?

• The out‐of‐hospital cardiac arrest score, the cardiac arrest hospital prognosis score, and the good outcome following attempted resusciation score are the 3 most rigorously validated tools for predicting the prognosis of SCA victims; however, the predictions made using these tools should be interpreted cautiously and in the context of an individual patient's clinical picture to avoid inappropriate early withdrawal of life‐sustaining treatment.

Sudden cardiac arrest (SCA) is the abrupt cessation of cardiac activity such that an individual becomes unresponsive, without breathing or signs of circulation. ¹ In the United States, there are ≈360 000 out‐of‐hospital cardiac arrest (OHCA) events and 210 000 in‐hospital cardiac arrest (IHCA) events annually. ² Prognosis after an SCA is dismal, with survival from OHCA and IHCA estimated to be ≈10% ³ and 25%, ⁴ respectively; rates of good neurologic outcome are even lower. Because of the often‐precipitous nature of SCA, surrogate decision makers may find themselves in the position of having to make unexpected, difficult choices about care for these patients. Critical decisions, such as withdrawal of care, tracheostomy or percutaneous gastrostomy tube placement, and subsequent changes in code status, are particularly difficult when overall prognosis is unclear. Effectively differentiating patients who are likely to do well after an SCA event from those who are unlikely to do well may help to guide these decisions. Unfortunately, this task is made more difficult by a lack of clear criteria or published guidelines on when and from whom care should be withdrawn after an SCA.

Clinical predictive models (CPMs) can help stratify patients by outcome risk. These models use patient‐specific data to make personalized clinical predictions. However, although some CPMs have been validated rigorously and incorporated into clinical practice guidelines, there is currently no CPM related to SCA outcome that has gained widespread use. In the present study, we assessed currently available SCA CPMs with special attention to how rigorously models have been validated and which variables emerge as being consistently important for predicting outcomes.

Methods

The data that support the findings of this study are available from the corresponding author on reasonable request.

Model and Validation Identification

We performed a systematic review of novel SCA CPMs and their validations (Figure 1). We included previously identified SCA CPMs from the Tufts PACE (Predictive Analytics and Comparative Effectiveness) CPM Registry. The registry, which is free and available to the public at http://pace.tufts​medic​alcen​ter.org/com, contains a field synopsis of CPMs in cardiovascular disease, including SCA, published between January 1990 and December 2015. These methods have been previously reported. ⁵ We identified additional English‐language abstracts containing potential SCA CPMs via a targeted PubMed search (Figure S1) of the OVID Medline database extending through February 1, 2020. Two independent reviewers screened potential abstracts using Abstrackr, a semiautomated online screening program. Discrepancies were discussed until consensus was achieved. We then selected abstracts for full‐text review if they met the following inclusion criteria: (1) made specific mention of multivariate modeling, (2) specified SCA as the index condition, and (3) were based on data from a primarily adult population. We then doubly screened full‐text publications and included them for further analysis if, in addition to meeting the inclusion criteria, they contained a novel, useable (meaning that an end user could generate an outcome prediction given knowledge of the appropriate set of patient variables) SCA CPM. We identified CPM validation studies by performing a Scopus citation search on all novel SCA CPMs identified as above. Two independent reviewers screened abstracts and full‐text publications in the same manner as for novel models. We included validation studies for further analysis if they assessed a previously published SCA CPM in a population temporally and/or spatially distinct from the population used in the initial development of that model. Novel models that were incidentally identified during validation search were also included, and the cycle of validation search was repeated until no further novel models were identified.

Figure 1. A flowchart showing methods by which both novel sudden cardiac arrest clinical predictive models and their validations were identified.

CPM indicates clinical predictive model; PACE, Predictive Analytics and Comparative Effectiveness; and SCA, sudden cardiac arrest.

Data Extraction and Statistical Analysis

We extracted data on the studied population, the proposed model, and SCA outcomes from both novel model and validation studies in accordance with the checklist for systematic reviews of prediction modeling studies. ⁶ Collected fields included location of data origin, whether data were collected prospectively or retrospectively, the approach to and amount of missingness in the data set, time frame of the predicted outcome, sample size, number of SCA events and whether the arrest occurred out of hospital or in hospital, model discrimination, and calibration. A modified version of the Prediction Model Risk of Bias Assessment Tool ⁷ , ⁸ was used to assess the risk of bias in model development and applicability of the models by a trained research assistant. The simplified version, Prediction Model Risk of Bias Assessment Tool Short Form, is a structured judgment system focusing on the analytic items in Prediction Model Risk of Bias Assessment Tool. It was collaboratively developed by clinicians and modeling experts for use and tested for agreement with the complete Prediction Model Risk of Bias Assessment Tool on models included in the Tufts PACE Center CPM Registry; the results are currently pending publication. We categorized the time frame of predicted outcome into 3 categories: (1) early, through 1 day after SCA, (2) intermediate, >1 day post‐SCA to hospital discharge, and (3) long‐term, beyond hospital discharge. We used the c‐statistic (or area under the curve of the receiver operator curve) to assess model discrimination. Because the c‐statistic is bounded between 0.5 and 1.0, we used percentage change in discrimination (equation 1) to make direct comparisons between discrimination of novel models and validation studies.

$$\begin{array}{c}\text{Percent change in discrimination}=\hfill \\ \left(\mathrm{\text{c}}‐{\text{statistic}}_{\text{Validation}}-\mathrm{c}‐{\text{statistic}}_{\text{Novel}}\right)/\hfill \\ \left(\mathrm{c}‐{\text{statistic}}_{\text{Novel}}-0.5\right)\times 100.\hfill \end{array}$$ (1)

Results

Novel SCA CPMs

We identified 81 unique CPMs of SCA published between July 1981 and February 2020 (Figure 2). Table 1 summarizes characteristics of the populations used in these novel SCA CPMs, and Table 2, ⁹ , ¹⁰ , ¹¹ , ¹² , ¹³ , ¹⁴ , ¹⁵ , ¹⁶ , ¹⁷ , ¹⁸ , ¹⁹ , ²⁰ , ²¹ , ²² , ²³ , ²⁴ , ²⁵ , ²⁶ , ²⁷ , ²⁸ , ²⁹ , ³⁰ , ³¹ , ³² , ³³ , ³⁴ , ³⁵ , ³⁶ , ³⁷ , ³⁸ , ³⁹ , ⁴⁰ , ⁴¹ , ⁴² , ⁴³ , ⁴⁴ , ⁴⁵ , ⁴⁶ , ⁴⁷ , ⁴⁸ , ⁴⁹ , ⁵⁰ , ⁵¹ , ⁵² , ⁵³ , ⁵⁴ , ⁵⁵ , ⁵⁶ , ⁵⁷ , ⁵⁸ , ⁵⁹ , ⁶⁰ , ⁶¹ , ⁶² , ⁶³ , ⁶⁴ , ⁶⁵ , ⁶⁶ , ⁶⁷ , ⁶⁸ , ⁶⁹ , ⁷⁰ , ⁷¹ presents detailed information on the complete set of identified models. Herein, models are identified using PubMed identification numbers; a model identification number further differentiates between multiple models contained within a single publication. Fifty‐five of those models (68%) predicted outcomes following OHCA, 17 (21%) following IHCA, and 9 (11%) following a mixture of both. Nine (11%) models predicted early outcomes, 42 (52%) predicted intermediate time frame outcomes, and 28 (35%) predicted long‐term outcomes. Thirty‐one (38%) models used European populations during derivation, 24 (30%) used North American populations, and 17 (21%) used Asian populations. Thirty‐one (38%) models were developed with prospective cohort data, and 49 (61%) models were developed using retrospective cohort data. Thirty‐three (41%) models reported their approach to missingness, and 27 (33%) models reported amount of missingness in the derivation cohort. Models took various forms, with point score‐based models constituting 35 (43%) models, logistic regression constituting 29 (36%) models, and characteristic decision trees constituting 16 (20%) models. The median sample size of derivation populations was 591 (interquartile range [IQR], 140–1028). The number of studies at low risk of bias was 6 (7%); the remaining 75 (93%) were high risk of bias. The median number of predictive covariates was 5 (IQR, 3–6), and the 3 most commonly used covariates were initial rhythm (n=51, 63%), age (n=42, 52%), and the duration of cardiopulmonary resuscitation (n=31, 38%) (Figure 3). Of models that reported discrimination (n=56, 69%), the median c‐statistic was 0.84 (IQR, 0.80–0.89) (Figure 4A).

Figure 2. Histogram showing the number of both novel sudden cardiac arrest clinical predictive models (blue) and validation (orange) studies that were published per 5‐year interval between January 1980 and February 2020.

 

Table 1.

Characteristics of the SCA CPM Derivation and Validation Populations

Characteristic	Novel Model	Validation Study
No. of models	81	62
Average age, y	65 (61–67)	65 (62–70)
Men, %	66 (60–75)	63 (58–70)
Sample size	591 (140–1028)	430 (212–1657)

Data are given as median (interquartile range), unless otherwise indicated. CPM indicates clinical predictive model; and SCA, sudden cardiac arrest.

Table 2.

List of the Identified SCA CPMs With Information Detailing the Derivation Population, Including Whether These Models Were Derived for a More Specific Index Condition (eg, in Patients Undergoing ECPR or TTM), Associated Model Outcomes, Discrimination if Investigated, and Number of External Validation Studies

PMID	Model No.	Population Size	OHCA vs IHCA	Specific Index Condition	Outcome Type	Outcome Time Frame	% Good Outcome	C‐Statistic	No. of External Validations
1315072 9	1	112	Mixed	None	Survival	Intermediate	55	0.83	0
1578040 10	1	6179	Mixed	None	Survival	Intermediate	11	NR	0
1661018 11	1	710	IHCA	None	Survival	Short‐term	28	0.78	0
1661018 11	2	198	IHCA	None	Survival	Intermediate	47	0.8	0
2246419 12	1	347	OHCA	None	Neurologic	Intermediate	11	NR	0
2551011 13	1	2235	Mixed	None	Survival	Intermediate	16	NR	1
2741977 14	1	140	IHCA	None	Survival	Intermediate	24	NR	2
7241728 15	1	611	OHCA	None	Survival	Intermediate	19	NR	2
9396421 16	1	1872	OHCA	None	Survival	Intermediate	31	0.65	0
9462599 17	1	127	OHCA	Witnessed arrest; cardiac cause	Survival	Intermediate	42	0.81	1
9462599 17	2	127	OHCA	Witnessed arrest; cardiac cause	Neurologic	Intermediate	39	0.89	1
9547842 18	1	100	OHCA	Cardiac cause; ventricular fibrillation	Survival	Intermediate	29	NR	0
12533358 19	1	741	IHCA	None	Survival	Short‐term	10	NR	1
1253335819	2	707	IHCA	None	Survival	Short‐term	9	NR	0
12626988 20	1	34	Mixed	None	Neurologic	Intermediate	47	NR	0
15246581 21	1	219	IHCA	None	Survival	Intermediate	15	NR	0
15246581 21	2	219	IHCA	None	Survival	Long‐term	14	NR	0
15246581 21	3	219	IHCA	None	Survival	Long‐term	11	NR	0
15531065 22	1	754	OHCA	None	Survival	Intermediate	1	NR	0
15531065 22	2	754	OHCA	None	Neurologic	Intermediate	2	NR	0
15531065 22	3	754	OHCA	None	Neurologic	Intermediate	2	NR	0
17082207 23	1	130	OHCA	None	Neurologic	Intermediate	22	0.82	9
18573589 24	1	1028	OHCA	Cardiac cause; shockable rhythm	Survival	Long‐term	20	0.74	0
18584503 25	1	748	OHCA	Ventricular fibrillation	Survival	Short‐term	NR	NR	0
20655699 26	1	591	OHCA	None	Survival	Short‐term	21	0.83	0
20655699 26	2	591	OHCA	None	Survival	Intermediate	13	0.88	0
20655699 26	3	591	OHCA	None	Survival	Long‐term	10	0.91	0
21482007 27	1	285	OHCA	Shockable rhythm	Neurologic	Long‐term	32	0.85	1
21482007 27	2	577	OHCA	Nonshockable rhythm	Neurologic	Long‐term	6	0.89	1
21515626 28	1	5471	OHCA	None	Survival	Intermediate	43	0.71	4
21756969 29	1	457	Mixed	None	Survival	Long‐term	47	NR	2
22281226 30	1	66	OHCA	TTM	Neurologic	Intermediate	61	0.95	0
22641228 31	1	28 629	IHCA	None	Neurologic	Intermediate	25	0.8	2
23844724 32	1	307 896	OHCA	None	Survival	Long‐term	4	0.79	1
23844724 32	2	307 896	OHCA	None	Neurologic	Long‐term	2	0.85	1
24018585 33	1	22 626	IHCA	None	Neurologic	Long‐term	11	0.78	6
24107638 34	1	38 092	IHCA	None	Neurologic	Long‐term	10	0.76	2
24107638 34	2	38 092	IHCA	None	Neurologic	Long‐term	10	0.73	2
24309445 35	1	750	OHCA	None	Survival	Long‐term	6	NR	0
24830872 36	1	14 688	IHCA	None	Survival	Short‐term	45	0.73	2
24830872 36	2	14 688	IHCA	None	Survival	Intermediate	20	0.81	2
24960427 37	1	1068	OHCA	None	Survival	Long‐term	40	NR	1
25443259 38	1	152	IHCA	ECPR	Survival	Intermediate	32	0.86	1
25828128 39	1	32	Mixed	TTM; ventricular fibrillation	Neurologic	Long‐term	47	0.98	1
25911585 40	1	92	OHCA	TTM	Survival	Long‐term	54	0.82	0
25911585 40	2	66	OHCA	TTM	Survival	Long‐term	67	0.88	0
26322336 41	1	96	OHCA	None	Neurologic	Intermediate	20	0.84	0
26497161 42	1	819	OHCA	None	Neurologic	Intermediate	27	0.93	6
26689743 43	1	207	OHCA	Cardiac cause	Survival	Intermediate	65	0.81	1
28410590 44	1	933	OHCA	Cardiac cause; TTM	Neurologic	Long‐term	47	0.84	0
28490379 45	1	151	OHCA	TTM	Neurologic	Intermediate	42	0.96	0
28528323 46	1	122	OHCA	TTM	Neurologic	Intermediate	27	0.82	1
28629472 47	1	687	OHCA	Cardiac cause; TTM	Neurologic	Long‐term	51	0.84	0
28647407 48	1	547	OHCA	None	Survival	Short‐term	59	0.66	0
28856660 49	1	111	Mixed	ECPR	Survival	Intermediate	19	0.88	0
29074504 50	1	638	OHCA	None	Survival	Intermediate	81	0.73	0
29317350 51	1	420 959	OHCA	None	Neurologic	Intermediate	1	NR	0
29481910 52	1	286	OHCA	Hypothermic arrest; ECPR	Survival	Intermediate	37	0.9	1
29580960 53	1	658	OHCA	Hypothermic arrest; ECPR	Neurologic	Continuous	40	NR	0
29677083 54	1	81	OHCA	Hanging‐induced arrest; TTM	Survival	Intermediate	25	0.91	0
29677083 54	2	81	OHCA	Hanging‐induced arrest; TTM	Neurologic	Intermediate	20	0.86	0
29942359 55	1	129	OHCA	None	Neurologic	Intermediate	30	0.84	1
30001950 56	1	153	OHCA	TTM	Neurologic	Long‐term	43	0.9	1
30261969 57	1	198	OHCA	Patients undergoing angiography	Survival	Long‐term	53	NR	1
30292802 58	1	456	OHCA	None	Neurologic	Long‐term	19	0.82	0
30345531 59	1	768	Mixed	None	Survival	Continuous	52	NR	0
30413210 60	1	107	OHCA	Cardiac cause; TTM	Neurologic	Long‐term	47	0.92	0
30601816 61	1	19 609	OHCA	None	Survival	Short‐term	41	NR	0
30650128 62	1	40	OHCA	None	Survival	Long‐term	30%	0.94	0
30650128 62	2	40	OHCA	None	Survival	Long‐term	30	0.95	0
30650128 62	3	40	OHCA	None	Survival	Long‐term	30	0.99	0
30807816 63	1	580	Mixed	None	Neurologic	Intermediate	37	0.88	0
30819521 64	1	852	OHCA	None	Neurologic	Intermediate	4	0.82	1
30848327 65	1	3855	OHCA	None	Neurologic	Intermediate	34	NR	0
31153943 66	1	460	OHCA	TTM	Neurologic	Long‐term	38	0.89	1
31153943 66	2	460	OHCA	TTM	Neurologic	Long‐term	29	0.9	0
31412292 67	1	628	IHCA	None	Neurologic	Intermediate	28	0.81	0
31539610 68	1	2685	OHCA	None	Survival	Intermediate	34	0.72	0
31730900 69	1	7985	OHCA	None	Neurologic	Intermediate	23	0.88	1
31821836 70	1	23 713	IHCA	None	Neurologic	Intermediate	22	0.7	0
31980268 71	1	1962	OHCA	None	Survival	Short‐term	22	0.83	1

CPM indicates clinical predictive model; ECPR, extracorporeal cardiopulmonary resuscitation; IHCA, in‐hospital cardiac arrest; NR, not reported; OHCA, out‐of‐hospital cardiac arrest; PMID, PubMed identification; SCA, sudden cardiac arrest; and TTM, targeted temperature management.

Figure 3. The top 10 most frequently included predictive covariates (or covariate classes) included in novel sudden cardiac arrest clinical predictive models.

CPR indicates cardiopulmonary resuscitation.

Figure 4. Histograms showing distributions of discrimination for novel sudden cardiac arrest clinical predictive models (A) and validation studies (B).

 

Validation Studies

We identified 62 SCA CPM validation studies published between April 1997 and February 2020 (Figure 1). Table 1 summarizes characteristics of the populations used in these validation studies, and Table 3, ⁷² , ⁷³ , ⁷⁴ , ⁷⁵ , ⁷⁶ , ⁷⁷ , ⁷⁸ , ⁷⁹ , ⁸⁰ , ⁸¹ , ⁸² , ⁸³ , ⁸⁴ , ⁸⁵ , ⁸⁶ , ⁸⁷ , ⁸⁸ , ⁸⁹ , ⁹⁰ , ⁹¹ , ⁹² , ⁹³ , ⁹⁴ presents detailed information on the complete set of identified validation studies. Of the 81 novel SCA CPMs, 33 (41%) were validated at least once, and only 4 (5%) were validated at least 3 times (Figure 5). The 3 most rigorously validated models were the OHCA score, ²³ the cardiac arrest hospital prognosis score, ⁴² and the good outcome following attempted resuscitation score ³³ (Table 4). All but one validation study reported discrimination (n=61, 98%). The median c‐statistic was 0.81 (IQR, 0.74–0.85) (Figure 4B). Only 29 of the 62 validations (47%) reported information on calibration. Of the 33 validated models, discrimination was reported in both model generation and validation publications for 26 models. For these models, the median percentage change in discrimination was −1.6% (IQR, −10.6% to 8.2%) (Figure 6).

Table 3.

List of the Identified Validation Studies With Information Detailing the Validation Population, Associated Outcome Rates, Discrimination, and Calibration Method if Investigated

Validation PMID	Novel Model PMID	Model No.	Population Size	Event Rate, %	C‐Statistic	Calibration Reported
9107612 72	2741977	1	656	5	0.52	No
9541764 73	9462599	1	62	53	0.92	No
9541764 73	9462599	2	62	47	0.93	No
12782522 74	7241728	1	575	4	0.33	Yes
17082207 23	17082207	1	210	25	0.88	Yes
20655699 26	2551011	1	591	10	0.82	No
20655699 26	7241728	1	591	13	0.79	No
20655699 26	12533358	1	591	21	0.73	No
21482007 27	21482007	1	212	46	0.87	No
21482007 27	21482007	2	423	5	0.87	No
21494106 75	17082207	1	128	23	0.85	Yes
21515626 28	21515626	1	2218	44	0.73	No
22281225 76	17082207	1	122	35	0.79	No
23844724 32	23844724	1	82 330	5	0.81	No
23844724 32	23844724	2	82 330	2	0.88	No
24107638 34	24107638	1	14 435	12	0.73	No
24107638 34	24107638	2	14 435	12	0.71	No
24830872 36	24830872	1	7791	45	0.72	Yes
24830872 36	24830872	1	1657	46	0.72	Yes
24830872 36	24830872	2	7791	18	0.81	Yes
24830872 36	24830872	2	1657	19	0.80	Yes
24960427 37	24960427	1	297	58	0.81	No
25636896 77	21756969	1	393	41	0.82	Yes
25636896 77	21756969	1	214	41	0.83	Yes
25828128 39	25828128	1	29	66	0.89	Yes
26393849 78	2741977	1	26 327	24	0.69	No
26393849 78	22641228	1	26 327	24	0.79	Yes
26393849 78	24018585	1	26 327	24	0.71	No
26497161 42	26497161	1	367	33	0.85	Yes
26497161 42	26497161	1	1129	25	0.91	Yes
26689743 43	26689743	1	96	65	0.82	No
27404694 79	24018585	1	287	16	0.85	No
28049389 80	24107638	1	287	16	0.77	No
28049389 80	24107638	2	287	16	0.71	No
28356134 81	21515626	1	680	50	0.71	Yes
28410590 44	17082207	1	933	47	0.75	Yes
28410590 44	26497161	1	933	47	0.75	Yes
28528323 46	28528323	1	344	21	0.81	Yes
29500154 82	17082207	1	150	22	0.57	No
29723201 83	17082207	1	173	31	0.74	No
29723607 84	24018585	1	717	22	0.82	Yes
29942359 55	29942359	1	31	NR	0.90	No
30001950 56	30001950	1	91	46	0.82	No
30138383 85	22641228	1	796	12	0.79	Yes
30261969 57	30261969	1	67	NR	NR	No
30391369 86	17082207	1	349	43	0.81	Yes
30391369 86	26497161	1	349	43	0.86	Yes
30447262 87	21515626	1	2041	29	0.76	Yes
30807816 63	17082207	1	437	NR	0.86	No
30819521 64	30819521	1	859	2	0.88	Yes
30940473 88	29481910	1	122	42	0.82	Yes
30981847 89	25443259	1	274	29	0.82	Yes
31078496 90	24018585	1	2845	11	0.75	Yes
31078496 90	24018585	1	16 154	15	0.76	Yes
31153943 66	31153943	1	151	42	0.93	No
31306716 91	17082207	1	336	45	0.79	No
31306716 91	26497161	1	336	45	0.81	No
31512185 92	24018585	1	403	17	0.68	No
31730900 69	31730900	1	1806	23	0.88	Yes
31980268 71	31980268	1	747	26	0.77	No
31987887 93	26497161	1	176	6	0.81	No
32035177y 94	21515626	1	63 059	8	0.74	Yes

NR indicates not reported; PMID, PubMed identification number.

Figure 5. A bar chart showing the number of times each of the novel sudden cardiac arrest clinical predictive models was validated.

 

Table 4.

Characteristics of the Top 3 Most Rigorously Validated SCA CPMs

Model Name	Arrest Setting	Outcome Time Frame	C‐Statistic	No. of Validation Studies	Median Validation C‐Statistic	% Change in Discrimination
OHCA score	OHCA	Intermediate	0.82	9	0.79	−9
CAHP score	OHCA	Intermediate	0.93	6	0.83	−23
GO‐FAR score	IHCA	Intermediate	0.78	6	0.755	−9

CAHP indicates cardiac arrest hospital prognosis; CPM, clinical predictive model; GO‐FAR, good outcome following attempted resuscitation; IHCA, in‐hospital cardiac arrest; OHCA, out‐of‐hospital cardiac arrest; and SCA, sudden cardiac arrest.

Figure 6. Histogram of the percentage change in discrimination between initial sudden cardiac arrest clinical predictive model derivation and subsequent external validation.

Some models were validated more than once.

IHCAs Versus OHCAs

We stratified SCA CPMs by whether the index SCA occurred in out‐of‐hospital or in‐hospital settings. We identified 55 models (68%) of OHCA with a median derivation population of 577 (IQR, 128–835) and median rate of events per variable of 17 (IQR, 8–56). We identified 17 models (21%) of IHCA with median derivation population of 710 (IQR, 219–22 626) and median rate of events per variable of 35 (IQR, 9–515). Discrimination was higher for OHCA models (median c‐statistic, 0.85; IQR, 0.82–0.90) than for IHCA models (median c‐statistic, 0.78; IQR, 0.75–0.80).

Discussion

In the present study, we have shown that there are a broad variety of models available for predicting clinical outcomes following SCA. We found that the median c‐statistic of novel SCA CPMs was 0.84, suggesting that in general these models are good at discriminating between patients who are likely to have a good outcome from those who are likely to have a poor outcome after a SCA (to put this in context, the cardiac failure or dysfunction, hypertension, age ≥75 [doubled], diabetes, stroke [doubled], vascular disease, age 65–74 and sex category [female] score which has been widely used for determining stroke risk in patients with atrial fibrillation had discriminations of 0.61 and 0.67 during derivation and validation, respectively ⁹⁵ , ⁹⁶ ).

This strong discrimination was maintained during external validation; in SCA CPMs that were validated at least once, matched comparison of discrimination from model generation and validation studies showed a median percentage change in discrimination of only −1.6%. This is in stark contrast to CPMs in other areas of cardiovascular disease. For example, we have previously examined CPMs related to valvular heart disease. The percentage change in discrimination of valvular heart disease CPMs was on the order of −30%. ⁹⁷ In another study in which we validated 3 major CPMs of acute heart failure, we found percentage decrements in discrimination of between −19% and −30%. ⁹⁸ Other groups have found similar effects in carotid revascularation ⁹⁹ and hospital readmission following acute myocardial infarction. ¹⁰⁰

Predictive Variables in SCA CPMs

One of the unique aspects of SCA CPMs compared with CPMs in other cardiovascular diseases is that predictions are made not just on characteristics of the individual patient, but also on characteristics of the cardiac arrest that a particular patient experiences. We found that the most frequently used predictive variables were event specific (eg, duration of cardiopulmonary resuscitation and initial SCA cardiac rhythm) rather than patient specific (eg, age and sex). The fact that these variables were so frequently selected after multivariate analysis suggests that they are strong predictors of outcome. This reliance on event‐specific variables may make SCA CPMs less sensitive to difference in the composition of patient population.

From a clinical perspective, this finding that predictions were largely independent of patient‐specific variables is counterintuitive. Several studies have shown that comorbidities, such as diabetes mellitus, ¹⁰¹ liver disease, ¹⁰² and malignancy, ¹⁰³ , ¹⁰⁴ are independent predictors of poor outcome in SCA. One possible explanation for this discrepancy is that there is covariance between these comorbidity variables and other variables that are more strongly associated with SCA outcome. Nonshockable rhythm, for example, is significantly more likely in patients experiencing SCA with underlying diabetes mellitus, liver disease, and malignancy. ¹⁰⁵ In SCA CPMs identified in this study, we identified several examples of comorbidity (eg, diabetes mellitus, ³⁰ , ⁴³ chronic kidney disease, ³⁹ , ⁴⁶ and malignancy ⁴⁴ ) dropout in favor of initial rhythm or other strong event‐specific variables.

Impacts of Location of Arrest

Models that examined outcomes after OHCA performed better on average than those that examined outcomes after IHCA, with median C‐statistics of 0.85 and 0.78, respectively. Although the CPMs for these 2 different populations share many of the same predictive variables, the magnitudes/values of these variables are different. Medical response times to OHCA are slower compared with IHCA ¹⁰⁶ ; it follows that much longer durations of no‐flow and low‐flow circulation ⁵¹ , ¹⁰⁷ are found in OHCA. Large surveys of both OHCA and IHCA have also shown that initial rhythm is less likely to be shockable in OHCA (13%) ¹⁰⁸ compared with IHCA (21%). ¹⁰⁹ The impact of arrest location on variable magnitudes is further complicated by the fact that the directionality of these changes may differ depending on the variable. For example, although OHCA tends to be longer and less likely to be shockable than IHCA, victims of IHCA tend to be sicker and have higher burdens of comorbidity at baseline compared with their OHCA counterparts. ¹¹⁰

Patients experiencing OHCA also have lower rates of survival and neurologic recovery than those experiencing IHCA. ¹¹¹ Although sensitivity and specificity are often assumed to be independent of the outcome rate in a population, these metrics ¹¹² , ¹¹³ can differ based on the underlying case mix of the population being studied. Discrimination is thus affected by heterogeneity and will tend to be better in more heterogeneous populations.

Finally, OHCA models were derived from smaller populations than IHCA models and had lower numbers of positive outcome per model covariate. This may have predisposed these OHCA models to relative overfitting compared with their IHCA counterparts and may in part explain the better discrimination of OHCA models.

Clinical Implications

The primary clinical use of these CPMs is in assisting physicians and surrogate decision makers with decisions on intensification, continuation, or withdrawal of care. For this purpose, SCA CPMs offer several advantages compared with guidance based on the anecdotal experiences of an individual physician. In studies of end‐of‐life counseling, miscommunication between physician and surrogate decision makers has been identified as a primary driver of inappropriately optimistic expectations of prognosis. ¹¹⁴ These optimistic expectations have been shown to significantly increase duration of intensive care unit hospitalization and cost without improving patient outcomes. ¹¹⁵ Quantitative assessments of prognosis, such as those offered by SCA CPMs, also leave less room for misinterpretation than qualitative assessments. ¹¹⁶ , ¹¹⁷

Inappropriate early withdrawal of life‐sustaining treatment attributable to perceived poor prognosis is a major cause of preventable death in victims of SCA (and may in part contribute to the high c‐statistics found in these CPMs by making bad outcomes easier to predict). Two cohort studies that matched SCA victims for whom care was withdrawn before 72 hours to those who received continued treatment estimated that 16% to 19% of patients who received withdrawal of care would have otherwise gone on to have good neurologic recovery. ¹¹⁸ , ¹¹⁹ Subjective impressions of poor prognosis from physicians are thought to be a major contributor to this inappropriate withdrawal of life‐sustaining treatment. ¹²⁰ In this case, SCA CPMs have the advantage of objectivity and may help to reduce the intrusion of physician‐held personal biases into discussions on withdrawal of care. ¹²¹ Nevertheless, these theoretical advantages should be examined empirically, ideally in clinical trials.

Because prophesies of mortality can be self‐fulfilling, ¹²² , ¹²³ , ¹²⁴ predictions in SCA should be made with care. Identifying when medical care is likely to be futile generally requires a high degree of certainty ¹²⁵ because the consequences of a false‐positive prediction are so dire. Although we identified 3 SCA CPMs (the OHCA score, the cardiac arrest hospital prognosis score, and the good outcome following attempted resuscitation score) that performed well using conventional measures of discrimination, it is unclear whether they can provide the confidence necessary to support futility claims. ¹²⁶ Any CPM‐based prediction should be interpreted in the broader context of an individual patient's overall clinical picture.

Limitations

There are several limitations to this work. Although we applied a systematic approach to the identification of novel SCA CPMs and their validations, our search was limited to the Medline and Scopus databases. It is possible that there are models and/or validation studies present in alternative databases that we failed to include. In addition, our ability to examine variable effects across models was limited by model heterogeneity. The inconsistent reporting of c‐statistic SE made formal, weighted statistical comparisons between groups of CPMs impossible.

Conclusions

There is a wide selection of CPMs designed for prognostication following SCA. These models demonstrated excellent ability to discriminate between patients experiencing SCA with good and poor prognosis. The most commonly used predictive variables were initial cardiac rhythm, patient age, and whether an SCA was witnessed. Discrimination remained high for those models that underwent external validation; however, few CPMs have been rigorously validated, and calibration is rarely reported. Although these quantitative assessments of prognosis may be helpful for decision making on withdrawal of care in arrest survivors, they should be interpreted in the broader context of an individual patient's overall clinical picture.

Sources of Funding

Research reported in this work was partially funded through a Patient‐Centered Outcomes Research Institute Award (ME‐1606‐35555).

Disclosures

None.

Supporting information

Figure S1

(J Am Heart Assoc. 2020;9:e017625 DOI: 10.1161/JAHA.119.017625.)