Predicting Quality of Life at 1 Year after Transcatheter Aortic Valve Replacement in a Real-World Population

Abstract

Background:

Among clinical trial patients at high surgical risk, a model has been developed and externally validated to estimate patient risk for poor outcomes after transcatheter aortic valve replacement (TAVR). How this model performs in lower risk and unselected patients is not known. We sought to examine and optimize the performance of the TAVR Poor Outcome Risk Model among patients in the US Society of Thoracic Surgeons/American College of Cardiology Transcatheter Valve Therapy (TVT) Registry.

Methods and Results:

Among 13,351 patients who underwent TAVR at 252 US sites between November 9, 2011-June 30, 2015, the rate of poor outcome at 1 year after TAVR was 38.9%, which was due to death in 20.7% and poor quality of life or quality of life decline in 18.2%. The rate of poor outcome has decreased slightly over time, from 42.0% in 2012 to 37.8% in 2015 (p for trend=0.076). The original TAVR Poor Outcome risk model did not calibrate well on this population. We then re-estimated the intercept and coefficients in the model and retested model performance, after which it performed well (both overall and in sub-groups), with a c-index 0.65 and excellent calibration.

Conclusions:

In a large cohort of unselected patients in the US, we found that while a substantial minority of patients continue to have a poor outcome after TAVR, outcomes have slowly improved over time. After recalibration, the TAVR Poor Outcome Risk Model performed well. This model could potentially be used prior to TAVR to help patients have appropriate expectations of recovery.

Transcatheter aortic valve replacement (TAVR) substantially improves survival and quality of life in the majority of patients with severe aortic stenosis.^{1, 2} Nonetheless, a substantial minority of patients continue to have poor quality of life or die soon after undergoing TAVR. If these patients could be identified prior to the procedure, this would allow for appropriate counseling as to choice of treatment (including the possibility of no treatment) and for realistic expectations for recovery. We previously developed³ and externally validated⁴ a model to estimate the risk of a poor outcome (using a composite of death or poor quality of life⁵) at 1 year among high-risk and inoperable patients who underwent TAVR as part of the PARTNER and CoreValve U.S. Pivotal trials. While the model performed well in development and in the external trial dataset with different patient populations and a different device, it is important to assess model performance (and recalibrate it, as needed) as the patient population changes (e.g., patients at lower surgical risk and those ineligible for the trials), as the technology changes (e.g., newer devices and delivery systems), as periprocedural care changes (e.g., less intensive care and general anesthesia), and as the sites and operators change (e.g., outside of careful oversight of the clinical trials). Models such as these have been planned for inclusion in a set of tools for patients to understand the benefits and risks of the procedure and set appropriate expectations for recovery. In addition, this model could be used to risk adjustment quality of life outcomes for site reporting and quality improvement. However, prior to implementation in such broad applications, we sought to ensure the model’s clinical applicability by examining its performance in a real-world cohort of patients enrolled in the Society of Thoracic Surgeons (STS)/American College of Cardiology (ACC) Transcatheter Valve Therapy (TVT) Registry.⁶ Furthermore, we determined how much the predicted risk of poor outcome varies across hospitals in the TVT registry. If substantial variability is observed, this could illustrate the potential role for such models by allowing providers to better identify high risk patients and provide appropriate pre-operative counseling.

METHODS

Study Sample and Protocol.

The data will not be made available to other researchers for purposes of reproducing the results or replicating the procedure; however, the analytic methods are described below along with the coefficients for the model. The TVT Registry is a quality improvement registry that was launched in 2011 as a joint initiative of the STS and ACC.^{6, 7} In order for hospitals to be reimbursed for the procedure, Medicare mandates participation in TVT, and so the registry includes data on nearly all commercial TAVR procedures performed in the US. Sites collect baseline and follow-up data on patient demographics, comorbidities, hemodynamics, functional status, and patient-reported health status. Rehospitalizations and survival are assessed through linkage to Medicare administrative claims using direct patient identifiers by the Centers for Medicare & Medicaid Services.⁸ Registry activities have been approved by a central institutional review board, and the Duke University School of Medicine institutional review board granted a waiver of informed consent for this study.

Outcomes Definition.

The primary outcome for our study was poor outcome at 1 year after TAVR, which was a composite outcome of death, poor quality of life, or decline in quality of life.^{5, 9} This combined definition integrates the two potential benefits of TAVR—reduced mortality and improved quality of life—but also recognizes that patients who have good quality of life at baseline may not improve symptomatically after TAVR but could still derive a survival benefit, which would be a clinically meaningful benefit of the procedure. In the TVT registry, quality of life is assessed at baseline, 30 days, and 1 year after TAVR by means of the overall summary score of the 12-item Kansas City Cardiomyopathy Questionnaire (KCCQ-OS),^{10, 11} a reliable and valid measure of symptoms, functional status, and quality of life in patients with severe, symptomatic aortic stenosis.¹² Values for the KCCQ-OS range from 0 to 100, with higher scores indicating fewer symptoms and better quality of life. As previously described, a poor outcome after TAVR was defined as any of the following at 1 year after TAVR: 1) death; 2) poor quality of life (KCCQ-OS score <60; roughly equivalent to NYHA III-IV^{12, 13}); or 3) moderate worsening in quality of life (decrease of ≥10 points in the KCCQ-OS score from baseline ¹³).

TAVR Poor Outcome Risk Models.

The TAVR Poor Outcome Risk Models were constructed using data from patients who were considered inoperable or at high surgical risk and who underwent TAVR in the PARTNER A and B trials and their associated continued access registries³ and were subsequently validated in the CoreValve U.S. Pivotal High Risk and Extreme Risk trials and their associated continued access registries.⁴ Four models were developed and validated: full models that included 6-minute walk test and mini-mental status exam data and clinical models that replaced the walk test with the KCCQ-12, categorized dementia, and reduced the number of variables further; both using 6-month and 1-year endpoints. The 4 models demonstrated moderate discrimination (c-indices 0.64–0.67) and good calibration in both the derivation, internal validation, and external validation cohorts, which is a key factor when determining the clinical usefulness of a model when perfect discrimination is not achievable.¹⁴ This indicates that while we cannot state with certainty which patient will or will not have a poor outcome, our ability to inform patients about their probability of a poor outcome is quite good. Given the time frame of KCCQ collection, only the 1-year clinical model was examined in the TVT registry. Furthermore, since dementia is not collected as a part of the TVT registry, the 1-year clinical model was recalibrated in the CoreValve trials without this variable.

Missing Data.

Missing data are an important consideration in any QOL analysis in clinical registries.^{7, 15} Before exclusions, the rate of missing KCCQ data among surviving patients was 42.8% at 1-year. Similar to prior TVT-based studies using quality of life data,¹⁶ we excluded 125 sites with <50% completion rates for the KCCQ, as sites with infrequent KCCQ completion may have more data missing due to patient factors (i.e., not missing at random). Among the remaining 245 sites, the rate of missing 1-year KCCQ data among survivors was 26.6%. Demographic and clinical characteristics of patients who survived 1 year but were missing follow-up KCCQ data were compared with those with data using 2-sided Wilcoxon Rank Sum Test for median values and standardized differences (>10% difference is considered clinically relevant) for categorical variables. To account for these missing data, we then used inverse probability weighting framework to increase the weight of patients who were most like those with missing follow-up data.¹⁷ This was done by constructing a multivariable logistic regression model among patients who survived 1 year to determine the probability of having missing follow-up KCCQ data. The model included all pre-specified patient-level factors as well as major in-hospital complications. We then weighted each of the surviving patients in the analytic cohort by the inverse probability of having follow-up KCCQ data to better reflect the overall TAVR population (patients who died within 1 year received a weight of 1). The rate of poor outcome for the overall sample and descriptive comparisons between those with and without a poor outcome were performed using the weighted sample. Rates were compared across years (2012–2015) using Cochran-Armitage trend test.

Model Validation and Recalibration.

As follow-up data were available through June 30, 2016, we limited the primary analyses to patients who underwent TAVR prior to June 30, 2015 to allow for the possibility of 1 year of follow-up data for all patients. To examine the performance of the TAVR Poor Outcomes Risk Model, we used the intercept and coefficients from the prior logistic regression prediction model to calculate the predicted risks of a poor 1-year outcome among patients in the TVT registry. Discrimination was examined with the c-index, and calibration was evaluated by plotting observed versus predicted risks by decile of predicted risk, with the regression line compared against the line of equality (intercept=0 and slope=1). Discrimination and calibration were also examined in several clinically-relevant sub-groups of patients: age <85 and ≥85 years; male and female sex; left ventricular ejection fraction (LVEF) <35% and ≥35%; New York Heart Association (NYHA) class I-III and IV; elective and urgent/shock/emergent acuity; STS mortality risk <4%, 4 to 8%, and ≥8%; and procedures done after January 1, 2014. Since the model calibration was not ideal, we then re-estimated the model parameters (using the same covariates) and re-examined model performance. This was done both on the weighted sample and the unweighted sample. As the parameter estimates were similar and the models had similar discrimination and calibration, the unweighted model was retained as primary. This model was internally validated using 10-fold cross-validation, with discrimination and calibration assessed on the overall cohort and in the aforementioned subgroups.

Variation in Predicted Probabilities.

We examined site level variability in predicted risks of poor outcome by examining the mean predicted risk of poor outcome (with 95% confidence intervals) among patients treated across sites in the TVT Registry. We limited this analysis to patients treated between July 1, 2014 and June 30, 2016 (the most recent complete 2 years of data), to account for changes in patient risk over time (lower risk patients treated now) and the addition of newer sites. To take into account small sample sizes at some sites, the site rates and confidence intervals were calculated from a Bayesian hierarchical model. We compared each site’s distribution of risk against the population mean risk of poor outcome to determine whether particular sites treated a significantly higher proportion of lower or higher risk patients. In addition, for sites that treated at least 5 patients with TAVR, we plotted each site’s mean predicted risk versus standard deviation. We hypothesized that sites that have a higher mean predicted risk would also have larger standard deviation (since they treat a broader distribution of patients). All analyses were performed with SAS version 9.4 (SAS Institute, Cary, North Carolina), and statistical significance was defined as a 2-sided p-value of <0.05.

RESULTS

Patient Population.

Among 26,057 patients at 377 sites who underwent TAVR between November 9, 2011 and June 30, 2015 and were able to be linked to Medicare administrative claims, we excluded 5,665 patients who were missing baseline KCCQ data and 7,041 patients at 125 sites where follow-up KCCQ completion rates were <50%. There were 2,707 patients who died within 1 year of TAVR, and of the remaining 10,644 patients who survived, follow-up KCCQ data were available for 7,863 (73.9%; Supplemental Figure 1). There were few differences between those who were alive but missing KCCQ data compared with those with KCCQ data (Supplemental Table 1). After applying propensity weighting to account for missing follow-up data, the rate of poor outcome at 1 year after TAVR was 38.9%, which was due to death in 20.7% and poor quality of life or quality of life decline in 18.2% (Figure 1). The rate of poor outcome has slightly decreased over time, from 42.0% in 2012 to 37.8% in 2015 (p for trend=0.076), which has been mostly driven by decreases in mortality over time.

Figure 1.

Rate of Poor Outcome at 1 Year After TAVR.

Demographic and clinical characteristics of patients who had a poor versus an acceptable outcome after TAVR are shown in Table 1. Patients who had a poor outcome were more likely to have had a prior stroke, peripheral artery disease, lung disease, renal dysfunction, atrial fibrillation, left ventricular dysfunction, and concomitant mitral valve disease. Patients with a poor outcome were more likely to have higher STS mortality risk scores and worse quality of life prior to TAVR. Patients with a poor outcome were higher acuity at the time of TAVR, were more likely to be treated via non-femoral access, were more likely to have periprocedural complications, and were less likely to be discharged to home (Table 2).

Table 1.

Baseline characteristics of patients with poor or acceptable outcomes after TAVR

	Poor Outcome (38.9%1)	Acceptable Outcome (61.1%*)	p-value
Age (years)	84 (79, 88)	84 (79, 88)	0.514
Female sex	48.4%	49.7%	0.202
White race			0.010
STS mortality risk score (%)	7.7 (5.1, 11.7)	6.2 (4.2, 9.4)	<0.001
<4%	14.2%	22.2%
4–8%	38.6%	43.9%
>8%	47.2%	34.0%
Coronary artery disease	64.7%	64.5%	0.803
Prior open heart surgery	31.2%	33.0%	0.058
Prior stroke	13.6%	11.0%	<0.001
Peripheral arterial disease	34.1%	29.2%	<0.001
Severe chronic lung disease	18.1%	10.5%	<0.001
Home oxygen	17.4%	8.7%	<0.001
Renal function			<0.001
Dialysis dependent	5.4%	2.7%
Creatinine ≥2.0 mg/dL without dialysis	7.5%	5.0%
Creatinine <2.0 mg/dL	87.2%	92.3%
Atrial fibrillation	48.6%	39.0%	<0.001
Permanent pacemaker/ICD	21.2%	17.5%	<0.001
LV ejection fraction			0.004
<30%	7.8%	6.5%
30–45%	18.9%	17.8%
>45%	73.3%	75.8%
Mean aortic gradient (mmHg)	42 (35, 50)	43 (36, 52)	<0.001
Moderate/severe mitral regurgitation	34.1%	31.9%	<0.001
Baseline KCCQ-OS	32.3 (18.8, 50.5)	43.8 (27.1, 62.5)	<0.001

^*Data are inverse propensity weighted to account for patients missing follow-up data

Continuous variables are presented as median (IQR) and compared using the Wilcoxon Rank Sum Test.

STS, Society of Thoracic Surgeons; ICD, implantable cardiac defibrillator; LV, left ventricular; KCCQ-OS, Kansas City Cardiomyopathy Questionnaire-overall summary score

Table 2.

Procedural data and complications

	Poor Outcome (38.9%1)	Acceptable Outcome (61.1%*)	p-value
Access site			<0.001
Transfemoral	64.7%	71.7%
Transapical	23.0%	19.4%
Other	12.4%	8.9%
Stroke	3.3%	1.2%	<0.001
Unplanned cardiac surgery	3.4%	1.4%	<0.001
Vascular complication requiring treatment	5.1%	4.3%	0.052
Major bleed	7.2%	5.8%	0.006
Life threatening or disabling bleed	6.8%	2.9%	<0.001
Discharge location (among those alive)			<0.001
Home	56.3%	73.5%
Extended care/rehabilitation	33.8%	21.4%
Skilled nursing/other hospital	8.9%	4.7%
Hospice	0.8%	0.0%
Stroke (1 year)	6.4%	2.6%	<0.001
KCCQ-OS (median [IQR]; 1 year)	45.8 (33.3–54.2)	87.5 (76.6–95.8)

^*Data are inverse propensity weighted to account for patients missing follow-up data

Model Validation and Recalibration.

When we examined the performance of the original TAVR Poor Outcome Risk Model among patients in the TVT registry, discrimination was moderate, with a c-index of 0.639, but calibration was suboptimal with over-estimation of risk (mean predicted risk of poor outcome of 49% versus actual rate of 39%), which was most notable among patients at higher predicted risks (Figure 2A). As such, we re-estimated the coefficients and intercept using the same covariates as in the original model (Table 3). There were small differences in the coefficients after re-estimation, indicating similar effects of the predictors on the risk of poor outcome. Lower mean aortic gradient was not as strongly associated with higher risk of poor outcome while diabetes was more strongly associated with higher risk of poor outcome (see Supplemental Table 2 for examples of how to calculate predicted risk using the new model). The discrimination of the re-estimated model remained moderate (c-index 0.648) with excellent calibration—both with and without inverse propensity weighting (Supplemental Figures 2A and B) and with 10-fold cross validation (Figure 2B). The model was able to separate patients into a wide range of risk categories, ranging from 23% in the lowest decile of predicted risk to 66% in the highest decile. Discrimination and calibration were similar for multiple key subgroups as well (Supplemental Figures 3–9).

Figure 2. Calibration of the Poor Outcome Risk Model.

(A) Model with the original coefficients and intercept. (B) Model with re-estimated coefficients and intercept using 10-fold cross-validation. The plots show predicted risk of poor outcome by deciles plotted against the observed rate of poor outcome in each decile. The red dashed line represents the regression line for the deciles; the blue line is the regression line forced through the 0 intercept; and the green line is the line of unity (i.e., perfect calibration). The statistical tests compare the red dashed and green lines.

Table 3.

TAVR Poor Outcome Risk Model

	Original Coefficients	New Coefficients	p-value*
KCCQ-OS (per 1 point)	−0.0162	−0.0151	<0.001
Mean aortic valve gradient (per 1 mmHg)	−0.0195	−0.0039	0.006
Home oxygen	0.6361	0.6007	<0.001
Creatinine (per 1 mg/dL)	0.1539	0.1733	<0.001
Atrial fibrillation/flutter	0.3090	0.3529	<0.001
Diabetes mellitus	0.0362	0.0704	0.100
Intercept	1.1047	0.0022	0.980
	c-index=0.639	c-index=0.648

KCCQ-OS, Kansas City Cardiomyopathy Questionnaire-overall summary score

^*P-value reflects the significance of the new coefficients and is not a comparison between new and old coefficients.

See Supplemental Table 2 for examples of how estimated risk can be calculated for individual patients.

Variation in Predicted Probabilities.

Among 13,112 patients who underwent TAVR at 370 sites between July 1, 2014 and June 30, 2016, the mean site-level probability of predicted risk of poor outcome was 40.5% with a range of 23.6%−56.3% (Figure 3A). There were 42 sites (11.4%) with confidence intervals that did not include the population mean risk (20 above the mean and 22 below the mean), indicating some site level variability in the predicted risks of poor outcome of patients being treated with TAVR across sites. When we plotted the mean predicted probability of each site against its standard deviation, there was little association between the two (Figure 3B; R²=0.07, slope=0.14), indicating that sites that treated more patients with higher predicted risk of poor outcome did not necessarily treat a broader distribution of patient risk.

Figure 3. Variability in Predicted Risk of Poor Outcome Across Sites in TVT.

(A) Mean predicted risk of poor outcome (with 95% confidence interval) of patients treated with TAVR at each site from July 1, 2014-June 30, 2016. (B) Site-level mean versus standard deviation predicted risk of poor outcome

DISCUSSION

In a large cohort of unselected US patients undergoing TAVR between 2011 and 2015, we found that the majority of patients have acceptable outcomes 1 year after TAVR. Nonetheless, 39% of patients either died or had a poor quality of life outcome at 1 year after TAVR. As expected, the rate of poor outcome after TAVR has been decreasing over time—most likely due to a combination of 1) treating patients who are younger, have fewer comorbidities, and have better functional status and 2) improvements in device technology, procedural skill, and periprocedural care. Owing to these changes, the original TAVR Poor Outcomes Risk Model required recalibration to be able to accurately assess patient risk. Once recalibrated, the model performed well overall and in key patient subgroups, indicating stability among the factors that previously were identified as being associated with poor outcomes. This model should be able to add to a suite of tools for patients prior to TAVR, to help patients better prepare for the procedure and the recovery after. As with most models, the TAVR Poor Outcome Risk Model should continue to be updated as there are evolutions in patient population, technology, or other key care pathways that may influence outcomes.

Site Variability.

We also found moderate variability in the predicted probability of a poor outcome among patients treated across sites, with some sites treating a higher proportion of patients at high risk for poor outcomes while other sites preferentially treating patients at low risk. We had hypothesized that sites that treated patients at high risk for poor outcomes would also treat patients with a wider distribution of risk; however, we did not observe a strong association between these two measures. As the risk of poor outcome estimated by this model is only one piece of data that should be considered, sites would be expected to appropriately treat some patients who would be predicted to be high risk for poor outcomes after TAVR. However, this model may provide additional information to clinicians as they determine whether or not TAVR is appropriate for particular patients, thereby reducing some of this site level variability.

Implications for Quality Reporting.

Although this model was designed to improve patient and physician decision making at the time of the initial heart team assessment, it can also be used to improve reporting of outcomes to sites. The TVT registry was developed to monitor processes and outcomes of TAVR for both local and national quality improvement. As one part of this mission, TVT provides benchmarked outcome reports to hospitals on an array of performance criteria so providers can identify areas for improvement, track their progress over time, and apply patient-based research to their practice. To date, these outcome reports have not included quality of life outcomes due to concerns of survival bias (complicating interpretation) and lack of risk adjustment (to allow for fair comparisons of outcomes). The approach described in this study could be ideal for this application—providing a longer-term quality of life metric that is both interpretable and risk-adjusted.

Limitations.

Our study should be interpreted in light of several potential limitations. First, although health status data are critical to understanding of benefit of TAVR, collection of these data in real-world registries without specific reimbursement has been challenging.¹⁸ We limited our analyses to sites with higher rates of complete data and used inverse propensity weighting to account for patient characteristics that differed between those with and without data. This latter step was essential for accurately calculating the rate of poor outcome, given the differential missing rates for survival and quality of life. Since there were few differences between those with and without follow-up KCCQ data, our model calibrated well both with and without weighting. Second, while the models were well calibrated with observed outcomes, discrimination remains only moderate. As such, while the model cannot be used to determine futility of valve replacement, it can reliably inform patients about their probability of a poor outcome, which could be quite valuable to improve decisional quality, reduce anxiety associated with the treatment decision, and provide patients and their families with realistic expectations of recovery. In addition, as described above, our model is ideal for quality reporting of risk adjustment outcomes to sites. Third, TAVR is an evolving technology, and improvements in techniques, devices, and periprocedural care (e.g., greater frequency femoral access) will continue to improve outcomes over time. We have previously shown similar model performance in patients treated with femoral and non-femoral access, with higher risk in patients requiring non-femoral access (although much of this risk was captured in the factors of the model).³ In our study, the model did not calibrate as well in patients with lower STS mortality risk scores, which likely reflects some unmeasured factors in these patients that impacts outcomes after TAVR (as these patients are not currently eligible for TAVR commercially unless some extenuating factor). We expect that this model will need to be updated over time as the procedure, the care associated with it, and the patient population continue to evolve. However, until such time, this model should be used with caution in patients with low STS risk scores as well as other patients who are not currently being treated commercially with TAVR (e.g., moderate aortic stenosis, asymptomatic). Finally, we have previously shown that some patient factors related to frailty and disability (unintentional weight loss, inability to perform activities of daily living) modestly improved the estimation of risk for poor outcome after TAVR⁴; however, these factors were not collected in TVT. As such, clinicians should recognize that risk is increased beyond the model-predicted estimation when these factors are present in patients.

Conclusion.

We found that while a substantial minority of patients continue to have a poor outcome after TAVR, outcomes have been improving over time—likely due to changing patient selection, improved device technology and operator experience, and advances in periprocedural care. We therefore recalibrated the model that estimates the risk of poor outcome for patients, which performed well thereafter. As with any emerging technology, this process of recalibration will need to be repeated as there continues to be evolutions in patient population, technology, and care pathways. This model can be used for individual patient counseling at the time of TAVR decision making—to help patients understand their risk and to set appropriate expectations for recovery.

What is known:

• A model to estimate an individual patient’s risk of poor outcome (death or persistently poor quality of life) after TAVR was developed and validated in patients at high risk for surgery.

• The model, which includes the patient factors of baseline health status, mean aortic valve gradient, home oxygen, serum creatinine, atrial fibrillation, and diabetes, was designed to help identify patients who are at high risk for poor recovery after TAVR so as to better inform shared decision making prior to TAVR.

What this study adds:

• In a large cohort of unselected patients in the US, we found that the rate of poor outcome after TAVR has been modestly decreasing over time (~50% in the pivotal clinical trials of high/extreme risk patients versus 39% in TVT [42% in 2012 to 38% in 2015]).

• The existing model did not validate well in the unselected and more contemporary patient cohort and was therefore re-estimated, after which it had moderate discrimination and excellent calibration.

• This study highlights the importance of periodically evaluating and potentially re-estimating prediction models, particularly when the technology, periprocedural care, and eligible patients are evolving over time.