A Joint Shock/Death Risk Prediction Model for Patients Considering Implantable Cardioverter-Defibrillators: A Secondary Analysis of the SCD-HeFT Trial

Abstract

Background

The risk of death or appropriate therapy varies widely among recipients of implantable cardioverter-defibrillators (ICDs). The goals of this study were to develop a risk prediction tool that jointly considers future outcome probabilities of ICD shock and death

Methods and Results

We performed a secondary analysis of patients receiving ICDs as part of the Sudden Cardiac Death in Heart Failure Trial. We applied an illness-death regression model to jointly model both ICD shocks and death under the semi-competing risks framework, which predicts for each patient their probability of having received ICD shocks and/or dying at any given point in time. Among 803 ICD recipients (mean age 60, 23% women) followed for a median of 41.1 months, 430 (53.5%) patients completed the study without dying or receiving an ICD shock, 206 (25.7%) received at least one shock but survived, 113 (14.1%) died prior to experiencing a shock, and 54 (6.7%) received at least one shock and subsequently died. Predicted outcome probabilities based on baseline demographic and clinical variables reveal substantial heterogeneity in joint shock and death risks, both between patients at each time point and for each single patient across time. Overall, predictive performance for ICD shock and death individually was adequate, based on area under the curve (AUC) at 5 years of 0.65 for shocks and for 0.79 for death.

Conclusions

Our analysis of outcomes following ICD implantation provides an alternative predictive model for individual risk of death or ICD shocks. If validated, this may provide a useful tool for individualized counseling regarding likely outcomes following device implantation, while also informing the design of further studies to focus the clinical- and cost-effectiveness of ICD therapy.

Background

Implantable cardioverter-defibrillators (ICDs) are the only proven therapy for the reduction of sudden cardiac death risk from ventricular arrhythmias.¹ However, many recipients of ICDs are older patients with multiple comorbidities, for whom the trade-offs between extended survival, ICD shocks, and quality of life concerns may not be straightforward.² Some patients with heart failure may actually prefer a relatively swift arrhythmic death to progressive decompensation, and will need to weigh the risks and inconvenience of living with an ICD against whatever value they place on the expected survival advantage of device-based therapy.³ At the same time, many recipients of ICDs are at very low risk for death with or without a device, further complicating decisions both at the individual and policy level.

In recognition of this tension, the Center for Medicare and Medicaid Services (CMS) now mandates the use of shared decision-making tools prior to new primary prevention ICD implantation.⁴ However, recommended decision aids⁵ generally characterize survival outcomes according to average treatment effect, despite known marked heterogeneity in patient benefit from ICDs.^{6, 7} Several studies have leveraged models distinguishing between sudden and non-sudden death as an avenue towards defining groups of ICD recipients most likely to benefit from the device.⁸ However, these models fail to account for competing risks over time between ICD shocks and death. That is, patients who survive without experiencing shocks will not have yet benefited from ICD implantation. In addition, some ICD patients will die prior to receiving shocks, while others will receive shocks but experience short-term mortality regardless. The projected risk on these distinct scenarios has significant implications in decision-making for some patients. Indeed, the elusive goal of clinically-meaningful and cost-effective patient selection for ICD therapy is to identify those patients with meaningful survival after receiving appropriate shocks that were presumably life-saving.⁹

Thus, the goals of this study were to develop a risk prediction tool that jointly considers future outcome probabilities of ICD shock and death.

Methods

Data Source and Patient Population

This study was approved by the Institution Review Board at Beth Israel Deaconess Medical Center through the exemption category. We analyzed data from the Sudden Cardiac Death in Heart Failure Trial (SCD-HeFT) prospective randomized controlled trial (ClinicalTrials.gov number, ), which were obtained from the National Heart, Lung, and Blood Institute (NHLBI) Biologic Specimen and Data Repository Information Coordinating Center. The primary findings of SCD-HeFT were published in 2005.¹⁰ In brief, the SCD-HeFT trial enrolled patients aged >18 with New York Heart Association (NYHA) class II or III congestive heart failure and a left ventricular ejection fraction of ≤35%. A total of 2521 patients were randomized in a 1:1:1 fashion to receive optimal medical therapy plus either placebo, amiodarone, or a single-chamber ICD. As described previously,¹¹ ICDs were programmed to be conservative with the goal of delivering ICD shocks only for arrhythmias likely to be life-threatening. Accordingly, by protocol only a single zone of therapy as used, with tachycardia episodes defined by at least 18 of 24 beats at a rate of ≥188 beats per minute or more (≤320 msec).

Of 829 patients randomized to receive an ICD, 17 declined, and 1 left the study prior to implantation. We obtained data on the 811 patients who received ICDs.

Patient Characteristics

Baseline patient characteristics prior to ICD insertion were identified from the clinical trial dataset, and selected for inclusion based on clinical relevance. Demographic variables included age and sex. Clinical conditions included New York Heart Association (NYHA) heart failure class, atrial fibrillation, diabetes mellitus, renal insufficiency, and etiology of heart failure (ischemic versus nonischemic). Ischemic heart disease was defined in the study protocol as left ventricular systolic dysfunction associated with >75% narrowing of at least one major coronary artery or a documented history of myocardial infarction. Diagnostic tests included left ventricular ejection fraction (LVEF) (%), duration of the QRS complex (milliseconds), serum creatinine (mg/dL), and serum sodium (mEq/L).

Outcomes

The primary outcomes of interest were time to first ICD shock and overall survival. For ICD shocks, the available data do not identify the exact time of first ICD shock in these patients, instead identifying the interval between patient visits in which the shock occurred. As the median interval width among intervals in which shocks occurred small relative to the overall median follow-up time, we assumed for analysis that each shock occurred at the midpoint time of its time interval. Of note, the available data did not distinguish between appropriate shocks (e.g. those for ventricular arrhythmias) and inappropriate shocks, such as those received for atrial fibrillation, other supraventricular tachycardias, or lead malfunction. In a prior report from the clinical trial investigators, 17% of all patients received an inappropriate shock, compared with 33.2% receiving a shock for any cause. ¹¹

Statistical Analysis

All baseline demographic data and clinical information were described using frequencies for categorical variables and means (standard deviations) and medians (interquartile range) for continuous variables. To predict patient outcome probability profiles, we used an illness-death regression model to jointly model both ICD shock and death under the semi-competing risks framework.^12,13 We previously described this approach as a method to account for the interaction between endpoints that may interfere with each other (see prior report in Circulation – Cardiovascular Quality and Outcomes).¹² Specifically, “semi-competing risks” refers to the setting in which there is a non-terminal event of interest (e.g., ICD shock), which can only occur for subjects who have not yet experienced the terminal event (e.g., death). By considering ICD shock and mortality simultaneously within the semi-competing risks framework, one can predict for each patient their probability profile of being in one of four outcome groups at any given point in time: (i) they will have experienced neither a shock event nor died; (ii) they will have died without having experienced a shock; (iii) they will have experienced a shock and subsequently died; and, (iv) they will have experienced a shock and remain alive. Crucially, by exploiting the interplay between the two outcomes through the statistical analysis (see below), this strategy provides a more comprehensive assessment of future risk for a patient than would be obtained through separate analyses.

To model time to first ICD shock and time to death we use an illness-death regression model which extends the standard Cox proportional hazards model for univariate outcomes by jointly specifying three transition-specific hazard functions: (1) time to ICD shock, (2) time to death among those without prior ICD shock, and (3) time to death given prior ICD shock (Figure 1). Formally, let T₁ and T₂ be the time of first ICD shock and time of death, respectively. Then the corresponding hazard functions linking T₁ and T₂ are as follows:

$$h_1(t_1\mid {\gamma }_i,{\mathit{X}}_i)={\gamma }_i{h}_{01}(t_1)\mathit{exp}({\mathit{X}}_i^{\mathrm{T}}{\beta }_1),t_1>0$$

$$h_2(t_2\mid {\gamma }_i,{\mathit{X}}_i)={\gamma }_i{h}_{02}(t_2)\mathit{exp}({\mathit{X}}_i^{\mathrm{T}}{\beta }_2),t_2>0$$

$$h_3(t_2\mid t_1,{\gamma }_i,{\mathit{X}}_i)={\gamma }_i{h}_{03}(t_2∕t_1)\text{exp}({\mathit{X}}_i^{\mathrm{T}}{\beta }_3),0<t_1<t_2$$

where X_i represents the set of covariates of the i^th patient, γ_i represents a patient-specific frailty term, and h₀₁, h₀₂, and h₀₃ are the baseline hazard functions. β₁, β₂ and β₃ are the model parameters corresponding to the included covariates. Together, these three hazards jointly specify the distribution of the two outcomes and thus can be used to connect a patients’ individual covariate profile with their future risk.

Figure 1:

Illness-death model jointly estimates proportional hazard models of time to shock, time to mortality without shock, and time from shock to mortality.

Using the ‘SemiCompRisks’ package in R, we estimate this model through maximum likelihood estimation with each baseline hazard function assuming a Weibull probability distribution.¹⁴ To select which baseline predictors to include in the three hazard models, we used a forward selection algorithm based on the Akaike Information Criterion (AIC).¹⁵ Note, in doing so, we let each of the three components of the illness-death model take its own set of baseline predictors, allowing for more complex interplay between patient characteristics and outcomes.

Individualized Risk Prediction and Validation

From the fitted model, we generated prospective risk prediction profiles, which show outcome probabilities across varying future time points according to baseline demographic and clinical characteristics.¹⁶ To evaluate predictive performance area under a time-dependent receiver operator characteristic curve (AUC) for each of ICD shock and death.¹⁷ Specifically, we used 10-fold cross-validation to assess predictive accuracy of all-cause shock and all-cause mortality.¹⁸ Briefly, for each outcome, we divided our data into 10 mutually exclusive sub-samples, then fit our model 10 times omitting one sub-sample each time and assessing prediction performance on the excluded sub-sample. The average of the AUCs computed over these iterations is our estimated predictive performance. Throughout, to account for potential verification bias in the estimation of the AUC due to censoring of the patient outcomes we used inverse probability of censoring weighting.^19,20

Data files and statistical analysis code used in this analysis may be made available by the authors upon reasonable request. Analyses were performed in R, the code for which is described in the Supplemental File.

Results

Clinical Characteristics and Outcomes

After excluding 8 patients due to baseline covariate missingness, 803 patients were retained for analysis. Table 1 summarizes the baseline characteristics of the study cohort. Consistent with the main trial findings, the median age was 60, 23% were women, 52% reported an ischemic etiology of heart failure, and the mean LVEF was 23.6%.

Table 1:

Characteristics of study population overall and according to receipt of implantable cardioverter-defibrillator shocks, death, or both during follow-up.

	Total (n=803)	None (n=430)	Shock Only (n=206)	Death Only (n=113)	Both (n=54)
Atrial fibrillation by ECG - no. (%)	63 (7.8)	20 (4.7)	20 (9.7)	12 (10.6)	11 (20.4)
Diabetes mellitus - no. (%)	245 (30.5)	131 (30.5)	38 (18.4)	60 (53.1)	16 (29.6)
Ischemic HF etiology - no. (%)	418 (52.1)	238 (55.3)	72 (35)	75 (66.4)	33 (61.1)
Renal insufficiency - no. (%)	206 (25.7)	105 (24.4)	30 (14.6)	49 (43.4)	22 (40.7)
NYHA Class III - no. (%)	254 (31.6)	113 (26.3)	52 (25.2)	62 (54.9)	27 (50)
Male - no. (%)	619 (77.1)	326 (75.8)	157 (76.2)	92 (81.4)	44 (81.5)
QRS duration ≥ 120ms - no. (%)	330 (41.1)	167 (38.8)	87 (42.2)	57 (50.4)	19 (35.2)
Patient age - Median (IQR)	60 (51, 69)	59 (51, 68.8)	56 (48, 66)	65 (58, 71)	65 (57, 69)
Weight in kg - Median (IQR)	86.3 (74.2, 99.9)	85.8 (74.2, 99.3)	89.1 (75.8, 104.1)	82.6 (68.1, 98.1)	85.5 (71.6, 97.4)
Creatinine (MG/DL) - Median (IQR)	1.1 (0.9, 1.4)	1.1 (0.9, 1.3)	1.1 (0.9, 1.3)	1.3 (1.1, 1.6)	1.2 (1, 1.5)
Sodium (MEQ/L) - Median (IQR)	139 (137, 141)	140 (137, 142)	139.5 (137, 141)	138 (136, 141)	139 (137, 140)
LVEF - Median (IQR)	24 (19, 30)	25 (20, 30)	22 (18, 28.8)	22.5 (18, 29)	20 (17, 25.8)

The median follow-up time was 41.1 months (Interquartile Range [IQR]: 29.6-53.2). During the follow-up period , 430 (53.5%) patients completed the study without dying or receiving an ICD shock, 206 (25.7%) received at least one shock but survived, 113 (14.1%) died prior to experiencing a shock, and 54 (6.7%) received at least one shock and subsequently died. In these 54 subjects, the median observed time between first ICD shock and death is 18.5 months (IQR: 7.8-33.9).

Joint Prediction Model

Our joint prediction modeling approach first identifies candidate variables independently associated with 3 separate outcomes – ICD shock without death, death without ICD shock, and death with a prior ICD shock. This identified the following variables: atrial fibrillation, diabetes, ischemic heart failure etiology, renal insufficiency, NYHA Class III status (vs. Class II as reference class), LVEF, age, and creatinine and sodium levels at baseline (Table 2).

Table 2:

Illness-Death Model Results.

	Hazard Ratio (95% CI)
	Shock	Death without First Shock	Death given Prior Shock
Ischemic HF etiology	0.57 (0.42, 0.77)†	—	2.03 (1.08, 3.83)*
Atrial fibrillation	3.14 (1.90, 5.22)*	—	—
LVEF (5 percent increment)	0.82 (0.71, 0.94)†	—	0.69 (0.50, 0.96)†
Diabetes mellitus	0.65 (0.47, 0.91)†	1.93 (1.28, 2.92)*	2.23 (1.13, 4.24)*
Renal insufficiency	0.59 (0.40, 0.88)†	—	1.98 (0.94, 4.18)
Creatinine (MG/DL)	1.45 (0.97, 2.19)	1.98 (1.42, 2.75)*	1.48 (0.85, 2.56)
NYHA Class III (vs. Class II)	—	2.43 (1.59, 3.74)*	1.98 (1.09, 3.60)*
Patient age (10 year increment)	—	1.42 (1.17, 1.71)*	1.43 (1.05, 1.95)*
Sodium (10 MEQ/L increment)	—	0.45 (0.26, 0.77)†	—

^*significantly greater than 1

^†significantly less than 1

Considering each model separately, the only clinical feature independently positively associated with an ICD shock was atrial fibrillation (HR 3.14, 95% CI 1.90-5.22), whereas ischemic heart failure etiology (HR 0.57, 95% CI, 0.42-0.77), each 5% increase in LVEF (HR 0.82, 95% CI 0.71-0.94), renal insufficiency (HR 0.59, 95% CI 0.40-0.88), and diabetes (HR 0.65, 95% CI 0.47-0.91) were all protective against this outcome. By contrast, factors independently associated with death without a prior ICD shock included diabetes (HR 1.93, 95% CI 1.28-2.92), each 1 mg/dL increase in creatinine (HR 1.98, 95% CI 1.42-2.75), NYHA class III status (HR 2.43, 95% CI 1.59-3.74), and each 10 year increase in age (HR 1.42, 95% CI 1.17-1.71), while each 10 meQ/L increase in sodium was protective (HR 0.45, 95% CI 0.26-0.77). Lastly, death after a prior shock was found to have positive independent associations with ischemic heart failure etiology (HR 2.03, 95% CI 1.08-3.83), diabetes (HR 2.23, 95% CI 1.13-4.24), NYHA Class III status (HR 1.98, 95% CI 1.09-3.60), and each 10 year increase in age (HR 1.43, 95% CI 1.05-1.95), whereas each 5% increase in LVEF was protective (HR 0.69, 95% CI 0.50-0.96).

This joint model of shock and death enables an individualized prediction framework according to patients’ baseline covariates. For example, Figure 2 illustrates personalized risk profiles for four different patients with differing baseline risk factors, based on 4 sample patients with baseline covariates specified in Table 3. Each plot shows the probability of the patient being in each of the 4 possible outcome categories for up to five years of follow-up. The height of each colored area at any particular month value gives the corresponding probability that the patient will be in that outcome category at that month. For example, we predict that Patient D has a 23% chance of experiencing neither shock nor death by 40 months, an 18% chance of dying by 40 months without having experienced a shock, a 23% chance of experiencing a shock then subsequently dying by 40 months, and a 36% chance of experiencing a shock, and surviving at least to 40 months.

Figure 2:

Predicted Patient Outcome Risk Profiles for Four Sample Patients. Area heights represent patient’s four predicted probabilities at each timepoint post-implantation.

Table 3:

Characteristics of Sample Patients in Figure 2.

	Patient A	Patient B	Patient C	Patient D
Ischemic HF etiology	No	No	No	Yes
Atrial fibrillation	Yes	No	No	No
LVEF (%)	20	20	32	33
Diabetes mellitus	Yes	No	Yes	No
Renal insufficiency	Yes	Yes	Yes	Yes
Creatinine (MG/DL)	1.3	1.5	2.1	1.4
NYHA Class	Class II	Class III	Class III	Class II
Patient age	73	78	60	65
Sodium (MEQ/L)	137	144	145	141

We applied this model to all 803 patients in the analysis, with outcome probabilities specified at 15, 30, 45, and 60 months of follow-up. Figure 3 illustrates the heterogeneity of outcomes across patients, with vertical bars showing 15-, 30-, 45-, and 60-month predicted outcome probabilities of each trial patient. In these panels, the horizontal axis of each includes 1 bar for each patient, ordered according to increasing likelihood of having received an ICD shock.

Figure 3:

Predicted probabilities for all subjects at (a) 15 months, (b) 30 months, (c) 45 months, and (d) 60 months from enrollment. Each vertical bar shows predicted outcome probabilities of a single study subject, ordered by total predicted probability of shock

Based on 10-fold cross-validation, after adjusting for verification bias our adjusted AUC for prediction of shock at 5 years is 0.65 and for death is 0.79. AUC values at 15, 30, and 45 months showed similar predictive performance.

Discussion

This secondary analysis of clinical trial data uniquely characterizes the substantial heterogeneity in the risk for ICD shocks and death, both between patients at each time point and for each single patient across time. Our modeling framework characterizes risks of shock and death jointly, which offers a more comprehensive insight into the interplay between ICD shocks and mortality than individual analyses of either endpoint. If externally validated, this approach may improve personalized decision-making for patients considering ICD implantation by providing individualized probabilities of achieving clinically-meaningful endpoints following device insertion. Policy-makers, funding agencies, and researchers working collaboratively to further refine selection criteria for ICD implantation could leverage our findings to identify patient groups unlikely to benefit from these devices.

Our model builds on prior attempts to characterize the heterogeneous treatment effects of ICDs using both clinical trial and observational datasets. For example, both the Multicenter Unsustained Tachycardia Trial (MUSTT) and the Multicenter Automatic Defibrillator Trial II (MADIT-II) identified an overall survival advantage for primary prevention ICD implantation.^{21, 22} Subsequent models from each were able to identify characteristics that defined populations that appeared not to benefit from ICD implantation, including those at relatively high risk for mortality despite protection from sudden cardiac death, and those at very low risk for therapies.^{6, 7} Neither approach has been prospectively validated, however, or integrated formally into clinical guidance or practice.

Alternative approaches, the Seattle Heart Failure Model (which predicts all-cause mortality) and the Seattle Proportional Risk Model (which predicts sudden cardiac death among heart failure patients) have been used to try to identify patients less likely to benefit from ICDs.^23-25 For example, when applied to a large national registry of ICD recipients with matched controls, the use of both models in concert identified a spectrum of potential ICD benefit in a dose-response fashion with increasing risk according to the Proportional Risk Model.⁸ Unlike these attempts to define heterogeneous treatment effect by comparison to a non-ICD control arm, our study design necessarily evaluated outcomes only in the population that received ICDs, with no control population of non-ICD patients whose survival outcomes might be compared with the findings from our model. However, our approach characterizes the same concept differently by defining probabilities of patients’ never receiving an ICD shock, or those who receive an ICD shock but die shortly thereafter regardless.

This distinction directly bears on potential clinical uses of our model. Indeed, integration of the models above into either shared decision-making or clinical guidance has remained elusive, in part because of persistent reluctance among providers to withhold ICD implantation from patients apparently meeting criteria similar to enrollment in trials such as SCD-HeFT. Our analysis of these same pivotal trial patients shows that only 25.7% received at least one shock and remained alive at the trial’s conclusion. This illustrates that even among “SCD-HeFT” patients, only a fraction actually experience the ideal clinical course of protection from life-limiting ventricular arrhythmias even under ideal trial conditions. That a small but important proportion of patients received shocks and subsequently died anyway during follow-up further highlights the limitation of ICD shocks as an endpoint indicating success of treatment. A model integrating readily available baseline characteristics delineated patient probabilities for these outcomes with good discriminatory performance across time points, and – if further validated – may help refine decision-making for the large proportion of primary prevention ICD recipients who apparently match SCD-HeFT criteria for receipt of a device. However, our model will need prospective validation in a contemporary cohort of patients receiving the current standard of care for medical therapy as well as device-based treatment, including optimal ICD programming and the provision of CRT when appropriate. A prospective cohort designed with this goal in mind is currently in the planning stages.

Our study includes several potential limitations. Again, we emphasize that as a secondary analysis of a randomized trial, our model results would benefit from external validation to better define its performance characteristics. As we drew upon a limited public dataset, our analytic approach could not distinguish between appropriate and inappropriate therapies, the latter of particular interest given the observed association between atrial fibrillation and shocks. Distinguishing between the two may influence the output of our model, but does not affect the demonstration of our semi-competing risk approach for characterizing outcome heterogeneity. Prospective validation using current programming strategies designed to reduce inappropriate shocks would guide model outputs towards contemporary clinical decision-making. In particular, programming of ICDs has also benefitted from improved discrimination and more permissive detection criteria,²⁶ although the recommended parameters in SCD-HeFT included relatively high detection rates and intervals, similar to current practice for primary prevention systems. External validation would also expand the generalizability of our findings beyond the inclusion criteria for this trial, which trended younger and less burdened by comorbidities than real-world cohorts of otherwise similar ICD patients.^{9, 27} Greater detail between the timing of shocks and subsequent death would also enrich the usefulness of this model, as smaller prior studies have suggested that shocks immediately proximate to dying may be relatively common.²⁸ Replication would also support testing the mathematical interactions between the different endpoints of interest, to explore (for example) if the relative or absolute performance characteristics of the model for each endpoint influences the other model or overall model fit. This trial also enrolled patients in a time period prior to wide use of cardiac resynchronization therapy and new therapeutic agents, such as angiotensin receptor–neprilysin inhibition,²⁹ which may limit the applicability of our findings to a more contemporary cohort. That said, despite these interventions, overall mortality for heart failure may actually be rising, which only amplifies the need to identify which of those patients apparently eligible for ICDs under current criteria will actually receive ICD shocks and continue to live meaningfully thereafter.^{30, 31}

In summary, our analysis of outcomes following ICD implantation provides an alternative predictive model for individual risk of death or ICD shocks. If validated, this may provide a useful tool for individualized counseling regarding likely outcomes following device implantation, while also informing the design of further studies to focus the clinical- and cost-effectiveness of ICD therapy.

What Is Known.

• The risk of death or appropriate therapy varies widely among recipients of implantable cardioverter-defibrillators (ICDs).

• Accounting for competing risks challenges traditional characterization of these outcomes for individual patients.

What the Study Adds.

• Using data from the Sudden Cardiac Death in Heart Failure (SCD-HeFT) Trial, we applied an illness-death regression model to jointly model both ICD shocks and death under the semi-competing risks framework, which predicts for each patient their probability of having received ICD shocks and/or dying at any given point in time.

• Predicted outcome probabilities based on baseline demographic and clinical variables reveal substantial heterogeneity in joint shock and death risks, both between patients at each time point and for each single patient across time.