Predicting Short-term Outcomes After Transcatheter Aortic Valve Replacement for Aortic Stenosis

Abstract

Background:

The approved use of transcatheter aortic valve replacement (TAVR) for aortic stenosis has expanded substantially over time. However, gaps remain with respect to accurately delineating risk for poor clinical and patient-centered outcomes. Our objective was to develop prediction models for 30-day clinical and patient-centered outcomes after TAVR within a large, diverse community-based population.

Methods:

We identified all adults who underwent TAVR between 2013–2019 at Kaiser Permanente Northern California, an integrated healthcare delivery system, and were monitored for the following 30-day outcomes: all-cause death, improvement in quality of life, all-cause hospitalizations, all-cause emergency department (ED) visits, heart failure (HF)-related hospitalizations, and HF-related ED visits. We developed prediction models using gradient boosting machines using linked demographic, clinical and other data from the Society for Thoracic Surgeons (STS)/American College of Cardiology (ACC) TVT Registry and electronic health records. We evaluated model performance using area under the curve (AUC) for model discrimination and associated calibration plots. We also evaluated the association of individual predictors with outcomes using logistic regression for quality of life and Cox proportional hazards regression for all other outcomes.

Results:

We identified 1,565 eligible patients who received TAVR. The risks of adverse 30-day post-TAVR outcomes ranged from 1.3% (HF hospitalizations) to 15.3% (all-cause ED visits). In models with the highest discrimination, discrimination was only moderate for death (AUC 0.60) and quality of life (AUC 0.62), but better for HF-related ED visits (AUC 0.76). Calibration also varied for different outcomes. Importantly, STS risk score only independently predicted death and all-cause hospitalization but no other outcomes. Older age also only independently predicted HF-related ED visits, and race/ethnicity was not significantly associated with any outcomes.

Conclusions:

Despite using a combination of detailed STS/ACC TVT Registry and electronic health record data, predicting short-term clinical and patient-centered outcomes after TAVR remains challenging. More work is needed to identify more accurate predictors for post-TAVR outcomes to support personalized clinical decision making and monitoring strategies.

INTRODUCTION

The use of transcatheter aortic valve replacement (TAVR) has expanded substantially since initial U.S. Food and Drug Administration (FDA) approval in November 2011, with 72,991 TAVR procedures performed nationally in 2019 alone. In the context of this rapid increase in TAVR use, there are questions about which patients are most likely to derive a robust benefit from TAVR and how to optimize follow-up care among patients post-TAVR. Based on STS/ACC TVT Registry data, while most TAVR patients experience favorable outcomes at one year post-TAVR, there is a still a sizable minority that either die, have a poor quality of life (QOL), or have a meaningful decline in QOL.⁹ Importantly, however, QOL data are missing in the registry for approximately 30% of patients who survive, and it is possible that these patients may also experience other adverse outcomes that are not being captured through the registry.⁹

Prior studies have attempted to identify TAVR patients at high-risk for poor outcomes. Specifically, risk calculators have been developed to predict mortality,^10–12 30-day readmissions,^{13, 14} in-hospital stroke¹⁵ and QOL.^16–18 However, these risk calculators have either relied exclusively on the STS/ACC TVT Registry data,^{11, 14, 15, 17} clinical trials,^{16, 18} a large claims database,^{12, 13} and a single institution.¹⁰ As with all sources of data, these particular sources have key limitations. As noted, the STS/ACC TVT Registry has a significant proportion of missing QOL data and only selected data elements and clinical outcomes.⁹ Clinical trial samples enroll selected patients that may not be fully representative of patients receiving treatment in real-world settings.¹⁹ Availability of candidate predictors of different outcomes also vary across the populations studied to date. Therefore, there is an opportunity to potentially develop better risk calculators by using linked data to include additional variables to may be relevant predictors of relevant clinical and patient-centered outcomes.

We aimed to address gaps in our understanding of which patients are likely to experience worse 30-day clinical outcomes and QOL following TAVR by leveraging linked STS/ACC TVT Registry data and EHR data from a large, integrated health care system. Our goal was to generate prediction models that may improve treatment decision-making and post-TAVR surveillance and management.

METHODS

Source Population and Data Sources

The source population was based in Kaiser Permanente Northern California (KPNC), an integrated healthcare delivery system currently caring for >4.5 million members in northern and central California. The KPNC membership is highly representative of individuals statewide in terms of age, gender, race, ethnicity and socioeconomic status.²⁰

We linked two data sources for the analysis. The first data source was the KPNC electronic health record system, which includes a wide range of data on the member population including diagnoses, procedures, comprehensive healthcare utilization (network and non-network), lab results, and medication dispensing, as well as overall comorbidity indexes and a score used to predict short-term mortality in hospital.²¹ The second data source is registry data that are collected at KPNC and submitted to the STS/ACC TVT Registry. Data elements in the STS/ACC TVT Registry that are not typically available from the EHR include clinical presentation, pre-procedure echocardiogram results, TVT procedure information, and a heart failure-specific QOL measure (Kansas City Cardiomyopathy Questionnaire).²² In addition, the STS/ACC TVT Registry includes certain complementary measures that overlap with data potentially available in the EHR including targeted diagnoses, procedures, lab results, and medication dispensing.

Study Sample

We identified all eligible KPNC members who received TAVR between 2013–2019 that were captured in the STS/ACC TVT Registry data. We excluded patients who had unknown gender and those who had fewer than 12 months of prior continuous membership before the TAVR to ensure sufficient enrollment to identify prior diagnoses, procedures, medication dispensing, and utilization.

Outcomes

We focused on six outcomes within 30 days post-TAVR: all-cause death, improvement in QOL, ED visits for any cause, hospitalizations for any cause, heart failure (HF)-related ED visits, and HF-related hospitalizations. We identified all-cause death using data from the EHR (which includes proxy reporting), Social Security vital status records, and California state death certificates. Improvement in 30-day QOL was defined as 10-point improvement from baseline or having a KCCQ total score of at least 60 units at follow up. A 10-point improvement in the KCCQ represents a moderate-to-large improvement in QOL.^{23, 24} A KCCQ score of at least 60 has been used in prior work as a cutoff for QOL²⁵ since it is comparable to the symptoms experienced from New York Heart Association Class I or II.²⁴ Hospitalizations and ED visits were comprehensively ascertained from EHR data. HF-related hospitalizations were defined as having a primary discharge diagnosis of HF, while HF-related ED visits were defined as having a diagnosis for HF in any discharge position based on relevant International Classification of Diseases, Ninth or Tenth Edition codes (codes available upon request). We previously demonstrated a positive predictive value ≥95% of HF for hospitalizations with a primary discharge diagnosis of HF when compared with clinical diagnostic criteria.²⁶ We also conducted a sensitivity analysis for QOL using an alternative definition of improvement based on ≥5-point improvement in KCCQ, a smaller but clinically meaningful change,²³ at 30 days.

Statistical Approach

To predict 30-day outcomes, we used gradient boosting machines (GBM). GBM is a machine learning approach that fits many decision trees and combines them together in a single algorithm to optimize prediction. One of the key advantages of GBM is that it automatically includes interactions and higher order terms.²⁷ We used the ‘gbm’ package in R to fit the models.²⁸ We identified the model hyperparameters using K-fold cross-validation with five folds. The hyperparameters determine how the model is fit such as the learning rate and the depth of the trees. We also estimated model performance using cross-validation. We divided the data into five folds and estimated the performance using the omitted fold. Finally, we averaged the performance across the five iterations and used the minimum and maximum performance to estimate the range of performance. We evaluated model discrimination using area under the curve (AUC) and calibration using the Hosmer-Lemeshow test and calibration plots.

Based on prior studies and a priori hypotheses, we evaluated whether STS risk category, age, gender and race/ethnicity were independent predictors of individual outcomes using logistic regression for the QOL outcome and Cox proportional hazards models for all-cause death and hospitalization and emergency department visit outcomes after accounting for potential confounders. In addition, we evaluated whether other patient characteristics were consistent predictors across 30-day post-TAVR clinical outcomes and QOL using backward selection procedures for socioeconomic status, individual cardiovascular conditions and other medical history as well as Charlson comorbidity score, laboratory results, vital signs, prior healthcare utilization, echocardiographic features, and coronary artery disease severity as candidate predictors. We included a total of 136 candidate variables in the GBM modeling algorithm for each outcome of interest (Supplemental Table 1).

Study Support

Funding was provided by the HCSRN-OAIC AGING Initiative from the National Institute on Aging (R33 AG057806) and The Permanente Medical Group Delivery Science Research Program. The authors are solely responsible for the design and conduct of this study, all study analyses, the drafting and editing of the paper and its final contents.

RESULTS

Baseline Characteristics

We identified 1,565 eligible adults who received TAVR from 2013–2019, with mean age of 81 years, 43% women, 3% Black, 6% Asian or Pacific Islander and 9% Hispanic (Figure 1 and Table 1). Comorbidity burden was high overall, including 39% with atrial fibrillation or atrial flutter, 40% with diabetes, and 47% with chronic lung disease. In addition, 68% were classified as high risk based on the STS risk score.

Figure 1. Cohort assembly of adults who received TAVR for aortic stenosis between January 1, 2013 and December 31, 2019.

Note: KPNC refers to Kaiser Permanente Northern California; TAVR refers to transcatheter aortic valve replacement; TVT refers to Transcatheter Valve Therapy.

Table 1.

Baseline characteristics of adults receiving TAVR for aortic stenosis between January 1, 2013 and December 31, 2019, overall and stratified by vital status at 30-days post-TAVR.

Variables	Patients identified receiving TAVR (N=1,565)	Patients who are alive at 30 days (N=1,534)	Patients who are dead at 30 days (N=31)	P
Age, yr
Mean (SD)	81.0 (8.2)	81.0 (8.2)	80.4 (9.3)	0.68
Range	24.2–99.4	24.2–99.4	56.7–94.0
Category				0.37
18–64	58 (3.7)	56 (3.7)	2 (6.5)
65–74	279 (17.8)	273 (17.8)	6 (19.4)
75–85	680 (43.5)	671 (43.7)	9 (29.0)
>85	548 (35.0)	534 (34.8)	14 (45.2)
Gender, N (%)				0.10
Men	886 (56.6)	873 (56.9)	13 (41.9)
Women	679 (43.4)	661 (43.1)	18 (58.1)
Race/ethnicity, N (%)				0.67
White	1260 (80.5)	1235 (80.5)	25 (80.6)
Black/African American	51 (3.3)	51 (3.3)	0 (0.0)
Asian/Pacific Islander	93 (5.9)	90 (5.9)	3 (9.7)
Hispanic	135 (8.6)	132 (8.6)	3 (9.7)
Other	26 (1.7)	26 (1.7)	0 (0.0)
Unknown	0 (0.0)	0 (0.0)	0 (0.0)
Year of TAVR, N (%)				<0.01
2013	77 (4.9)	75 (4.9)	2 (6.5)
2014	146 (9.3)	141 (9.2)	5 (16.1)
2015	139 (8.9)	130 (8.5)	9 (29.0)
2016	159 (10.2)	157 (10.2)	2 (6.5)
2017	200 (12.8)	195 (12.7)	5 (16.1)
2018	323 (20.6)	321 (20.9)	2 (6.5)
2019	521 (33.3)	515 (33.6)	6 (19.4)
Left Ventricular Ejection Fraction, N (%)				0.52
Preserved	1248 (79.7)	1220 (79.5)	28 (90.3)
Mid-range	115 (7.3)	114 (7.4)	1 (3.2)
Reduced	192 (12.3)	190 (12.4)	2 (6.5)
Missing	10 (0.6)	10 (0.7)	0 (0.0)
Pre-TAVR risk assessment
STS Risk Score, Mean (SD)	5.1 (3.6)	5.1 (3.5)	7.4 (5.3)	<0.001
Operator Reason for Procedure, N (%)				0.12
Inoperable/Extreme Risk	63 (4.0)	63 (4.1)	0 (0.0)
High Risk	1058 (67.6)	1031 (67.2)	27 (87.1)
Intermediate Risk	384 (24.5)	381 (24.8)	3 (9.7)
Low Risk	36 (2.3)	36 (2.3)	0 (0.0)
Unknown	24 (1.5)	23 (1.5)	1 (3.2)
Comorbidities, N (%)
Myocardial infarction or unstable angina	181 (11.6)	174 (11.3)	7 (22.6)	0.05
Prior coronary artery bypass surgery	279 (17.8)	272 (17.7)	7 (22.6)	0.48
Prior percutaneous coronary intervention	479 (30.6)	466 (30.4)	13 (41.9)	0.17
Prior heart failure	754 (48.2)	734 (47.8)	20 (64.5)	0.07
Ischemic stroke or transient ischemic attack	153 (9.8)	147 (9.6)	6 (19.4)	0.07
Peripheral artery disease	468 (29.9)	453 (29.5)	15 (48.4)	<0.05
Atrial fibrillation or flutter	609 (38.9)	595 (38.8)	14 (45.2)	0.47
Diabetes mellitus	620 (39.6)	605 (39.4)	15 (48.4)	0.31
Hypertension	1397 (89.3)	1370 (89.3)	27 (87.1)	0.69
Chronic liver disease	224 (14.3)	218 (14.2)	6 (19.4)	0.42
Dyslipidemia	1460 (93.3)	1433 (93.4)	27 (87.1)	0.16
Chronic lung disease	738 (47.2)	723 (47.1)	15 (48.4)	0.89
Hyperthyroidism	61 (3.9)	60 (3.9)	1 (3.2)	0.85
Hypothyroidism	369 (23.6)	361 (23.5)	8 (25.8)	0.77
Diagnosed dementia	52 (3.3)	52 (3.4)	0 (0.0)	0.30
Diagnosed depression	266 (17.0)	264 (17.2)	2 (6.5)	0.11

Crude Outcomes

During the 30-days post-TAVR, only 2.0% of patients died from any cause (Table 2). Among the 1,563 patients who did not die during the index TAVR hospitalization, the 30-day incidence of HF-related hospitalization was 1.3%, 6.9% for HF-related ED visits, 6.8% for all-cause hospitalizations, and 15.3% for ED visits for any cause. Among the 1,055 patients (67.4%) who had complete baseline and follow-up QOL data, 8.6% experienced no significant improvement in QOL at 30 days post-TAVR, and 91.4% experienced significant improvement. Unadjusted clinical outcomes and change in QOL were more favorable among patients with a low (<3) or intermediate (3–8) STS risk score compared with patients who had a high STS risk score (>8) (Table 2).

Table 2.

Unadjusted outcomes at 30-days post-TAVR.

	Overall (N=1565)	STS Risk Score <3 (N=344)	STS Risk Score 3–8 (N=969)	STS Risk Score >8 (N=201)
All-cause death	31 (2.0)	3 (0.9)	17 (1.8)	11 (5.5)
All-cause hospitalization	107 (6.8)	14 (4.1)	73 (7.5)	15 (7.5)
All-cause emergency department visit	240 (15.3)	41 (11.9)	155 (16.0)	37 (18.4)
Heart failure-related hospitalization	21 (1.3)	2 (0.6)	15 (1.6)	3 (1.5)
Heart failure-related emergency department visit	108 (6.9)	13 (3.8)	71 (7.3)	21 (10.5)
	Overall (N=1055)	STS Risk Score <3 (N=264)	STS Risk Score 3–8 (N=659)	STS Risk Score >8 (N=101)
Alive at 30 days and either KCCQ > 60 or KCCQ increase ≥10 between baseline and 30 days post-TAVR	964 (91.4)	252 (95.5)	602 (91.4)	84 (83.2)

Note: STS refers to the Society of Thoracic Surgeons; KCCQ refers to the Kansas City Cardiomyopathy Questionnaire.

Prediction of 30-Day Post-TAVR Outcomes

Model discrimination was moderate for all-cause death (AUC 0.60, [min-max: 0.52–0.68]) and better for HF-related ED visits (AUC 0.76, [min-max, 0.71–0.79]). In contrast, model discrimination was suboptimal for all-cause hospitalization, all-cause ED visits, and HF-related hospitalization (Table 3). With respect to calibration, the results of the Hosmer-Lemeshow test were not statistically significant for all outcomes except for death (p=0.03). However, model calibration was only modest for death, hospitalization, ED visits, and HF-related hospitalization (Figures 2–4). In contrast, model calibration was better for HF-related ED visits (Figure 4).

Table 3.

Model performance for clinical outcomes and quality of life at 30-days post-TAVR.

30-Day Outcome	Discrimination Mean AUC (range)	Calibration Hosmer-Lemeshow P-Value
Death from any cause	0.60 (0.52–0.68)	0.03
Hospitalization for any cause	0.51 (0.48–0.55)	0.69
Emergency department visit for any cause	0.55 (0.50–0.59)	0.14
Heart failure-related hospitalization	0.57 (0.49–0.71)	0.25
Heart failure-related emergency department visit	0.76 (0.71–0.79)	0.19
Quality of life improvement	0.64 (0.54–0.76)	0.33
Sensitivity analysis using alternative definition for quality of life improvement	0.72 (0.60–0.78)	0.39

Note: AUC refers to area under the curve

Figure 2. Calibration plots for prediction models for death from any cause and quality of life improvement at 30-days post-TAVR.

Note: HL refers to Hosmer-Lemeshow test

Figure 4. Calibration plots for prediction models for hospitalizations or emergency department visits related to heart failure at 30 days post-TAVR.

Note: HL refers to Hosmer-Lemeshow test; HF refers to heart failure; ED refers to emergency department

Model discrimination was moderate for QOL improvement at 30 days post-TAVR (AUC 0.64, [min-max: 0.54–0.76]), along with moderate calibration (Figure 2). In a sensitivity analysis that used an alternative definition of QOL improvement, we found significantly better performance for both model discrimination (AUC: 0.72, [min-max: 0.60–0.78]) and calibration (Appendix Figure 1).

Multivariable Predictors of Outcomes

After controlling for other potential confounders, STS risk score had a graded increased risk with hospitalization for any cause, and STS ≥8 was associated with a higher adjusted risk of all-cause death, but STS risk score was not significantly associated with any of the other outcomes (Table 4). Older age was also only significantly associated with HF-related emergency department visits but not with other outcomes, while male sex was associated with a lower risk of experiencing post-TAVR QOL but not with any of the other outcomes (Table 4). Self-reported race and ethnicity were not significantly associated with any outcomes. Furthermore, in examining a wide range of other patient characteristics, there were no consistent predictors across each of the clinical outcomes and unfavorable QOL post-TAVR (Table 4).

Table 4.

Multivariable predictors of adverse clinical outcomes and quality of life at 30 days after TAVR.

	Unfavorable Quality of Life	Death	Hospitalization for Any Cause	Emergency Department Visit for Any Cause	Heart Failure-related Hospitalization	Heart Failure-related Emergency Department Visits
	N=1055	N=1565	N=1563	N=1563	N=1563	N=1563
Variable	91 events	31 events	107 events	240 events	21 events	108 events
STS risk category
STS <3	(ref)	(ref)	(ref)	(ref)	(ref)	(ref)
STS 3–8	1.22 (0.59–2.56)	1.68 (0.51–5.49)	1.83 (1.03–3.26)	1.04 (0.71–1.52)	2.60 (0.57–11.75)	0.96 (0.49–1.87)
STS ≥8	1.76 (0.69–4.50)	4.50 (1.29–15.72)	2.10 (1.01–4.36)	1.03 (0.59–1.77)	1.89 (0.31–11.52)	0.92 (0.41–2.07)
Index age, per year	1.01 (0.98–1.05)		1.00 (0.97–1.02)	1.01 (0.99–1.03)		1.03 (1.00–1.06)
Gender
Female	(ref)		(ref)	(ref)		(ref)
Male	0.44 (0.26–0.75)		1.13 (0.75–1.70)	1.02 (0.76–1.36)		1.28 (0.81–2.01)
Race/ethnicity
White			(ref)	(ref)		(ref)
Black			0.57 (0.14–2.36)	1.32 (0.70–2.46)		1.27 (0.48–3.38)
Asian/Pacific Islander			0.43 (0.13–1.43)	0.69 (0.37–1.31)		0.54 (0.19–1.55)
Hispanic			0.47 (0.18–1.23)	1.18 (0.73–1.89)		1.58 (0.83–3.04)
Other			0.54 (0.07–4.22)	0.76 (0.23–2.51)		0.63 (0.08–4.99)
Comorbidities
Heart failure				1.48 (1.09–2.00)		13.00 (6.40–26.39)
Unstable angina				2.07 (1.02–4.19)
No atrial fibrillation	(ref)
Paroxysmal atrial fibrillation	0.30 (0.11–0.88)
Persistent atrial fibrillation	1.71 (0.97–3.03)
Atrial flutter			2.52 (1.45–4.35)
Ischemic stroke			3.32 (1.58–7.00)		8.17 (1.80–37.10)
Transient ischemic attack	2.15 (1.10–4.19)	2.85 (1.13–7.15)
Peripheral artery disease	2.07 (1.23–3.47)					1.51 (1.02–2.23)
Chronic liver disease	1.89 (1.03–3.47)			0.63 (0.41–0.98)
Dyslipidemia		0.30 (0.10–0.91)
Depression			1.64 (1.05–2.55)
Dementia			2.78 (1.32–5.85)	2.12 (1.12–3.65)
Chronic dialysis				0.24 (0.08–0.71)		0.10 (0.02–0.44)
Short-term mortality score, per %	1.11 (1.01–1.21)	1.14 (1.09–1.19)
Pre-procedural heart rate
Heart rate < 60 beats/min			0.90 (0.51–1.60)
Heart rate 60–100 beats/min			(ref)
Heart rate > 100 beats/min			2.65 (1.26–5.59)
Pre-procedural laboratory tests
eGFR >= 60				(ref)		(ref)
eGFR 45–59				1.36 (0.99–1.88)		1.52 (0.92–2.53)
eGFR 30–44				1.23 (0.82–1.85)		1.50 (0.86–2.59)
eGFR 15–29				1.19 (0.59–2.40)		1.92 (0.79–4.68)
eGFR < 15				6.43 (2.22–18.63)		8.68 (2.62–28.75)
Albumin >= 3.5	(ref)	(ref)			(ref)
Albumin < 3.5	4.32 (2.08–8.98)	4.95 (2.26–10.84)			4.55 (1.42–14.56)
BNP < 100 pg/mL	(ref)
BNP >= 100 pg/mL	2.68 (0.96–7.42)
HDL >= 60	(ref)
HDL 50–59	0.44 (0.16–1.19)
HDL 40–49	0.80 (0.35–1.84)
HDL 35–39	1.46 (0.47–4.55)
HDL < 35	0.87 (0.28–2.64)
Total cholesterol > 240		2.68 (0.38–19.01)
Total cholesterol 200–240		4.54 (1.17–12.04)
Total cholesterol < 200		(ref)
Platelets > 400				(ref)		(ref)
Platelets 150–400				0.56 (0.24–1.32)		0.18 (0.07–0.46)
Platelets < 150				0.61 (0.25–1.49)		0.21 (0.08–0.56)
Cardiac characteristics
Enlarged LV internal diastolic dimension	(ref)
Normal LV internal diastolic dimension	1.22 (0.47–3.18)
Missing LV internal diastolic dimension	0.27 (0.07–1.09)
dimension
AV mean gradient, per mmHg	0.98 (0.96–1.00)
Aortic valve area, per cm2						0.34 (0.11–1.05)
Left main disease >= 50%	2.69 (1.10–6.59)
Aortic stenosis	0.18 (0.04–0.88)
Mitral stenosis				1.52 (1.07–2.14)
Moderate or severe aortic insufficiency			0.43 (0.19–0.97)
0 diseased vessels		(ref)		(ref)
1 diseased vessel		3.30 (1.27–8.59)		1.40 (1.00–1.97)
2 diseased vessels		3.24 (1.14–9.20)		0.88 (0.58–1.34)
3 or more diseased vessels		1.51 (0.48–4.77)		0.82 (0.56–1.18)
Hostile chest				1.50 (1.03–2.17)
Prior aortic valve procedure						0.43 (0.21–0.88)
Degenerative mitral regurgitation						0.13 (0.05–0.33)
Degenerative AVD etiology				0.42 (0.19–0.91)
Leaflet indication - repair mortality risk >= 6%						8.30 (2.99–23.08)
Missing LVEF			6.52 (2.10–20.17)
Aortic valve peak velocity, per m/s					0.45 (0.28–0.71)
Prior utilization
0 prior ED visits				(ref)
1 prior ED visit				1.55 (1.13–2.14)
2 prior ED visits				1.49 (0.99–2.25)
3+ prior ED visits				2.26 (1.59–3.21)
0 prior HF hospitalizations				(ref)		(ref)
1 prior HF hospitalization				0.66 (0.43–1.01)		0.75 (0.45–1.26)
2 prior HF hospitalizations				0.62 (0.22–1.70)		0.51 (0.13–2.08)
3+ prior HF hospitalizations				2.93 (1.19–7.23)		6.40 (2.32–17.68)

DISCUSSION

Among a diverse, community-based population undergoing TAVR with uniquely linked STS/ACC TVT Registry and longitudinal EHR data, we developed and internally validated risk prediction models for 30-day post-TAVR all-cause death, reduced QOL, and HF-related and all-cause hospitalizations and ED visits. We found that depending on the outcome, we observed variable performance in terms of model discrimination and calibration even given the wide range of data elements included as candidate covariates. Despite studying patients with advanced age and a high comorbidity burden, the 30-day absolute risks for 5 of the 6 study outcomes were each less than 10%. We also found that higher STS was not a consistent independent predictor of worse outcomes, and there was substantial variability in the predictive ability of a broad range of patient characteristics for each of the outcomes of interest. Furthermore, older age only independently predicted HF-related ED visits, and race/ethnicity was not a significant multivariable predictor of any studied outcomes.

Compared with previously published models, the moderate or better discrimination (AUC ≥0.60) of our three prediction models in our study population was similar overall. Model discrimination for 30-day post-TAVR all-cause death (AUC 0.60) was lower than for other reported models (AUC 0.66 to 0.69) in selected populations,^10–12 and model discrimination for 30-day QOL was similar to studies that reported on QOL at 6- or 12-months post-TAVR (AUC 0.64 vs. 0.64–0.66).^16–18 For QOL, both discrimination and calibration were improved in the sensitivity analysis that used a more liberal definition for improvement in QOL in our study population. For HF-related ED visits, model discrimination was good (AUC 0.76), and we are unaware of previous studies evaluating this outcome to compare model performance. In contrast, model performance was suboptimal for other outcomes we evaluated including 30-day hospitalizations related to HF or for any cause as well as ED visits for any cause. Prior studies have primarily focused on 30-day all-cause hospitalizations and some have reported moderate model discrimination^{13, 14} but the absolute risk of hospitalization in one study was 16.6%¹³ compared with only 6.6% in our population. These findings highlight the ongoing challenge of accurately predicting short-term outcomes following TAVR for different clinical and patient-centered outcomes. Performance may have been better if separate models were created in subgroups for which the factors affecting outcomes would be more similar. For example, future work with sufficiently large sample sizes could develop separate models stratified by time period in which the TAVR was performed or STS risk score categories.

There are several potential explanations for the variable performance we observed across the different outcomes. First, prediction models often perform worse when outcomes are uncommon²⁹ which was the case in our study for nearly all of the outcomes of interest compared with other studies with larger sample sizes^{11, 12, 14, 15, 17} and/or had higher event rates.^{12, 13, 16–18} Second, rapid changes over the time period (2013–2019) include improved valve technologies linked to better outcomes,³⁰ greater operator experience and development of supportive care pathways,³¹ and a changing TAVR population as clinical indication expanded to less severe patients. Third, prediction of outcomes related to utilization (e.g., hospitalizations or emergency department visits) is generally more difficult than prediction of “hard” clinical outcomes like death.³²

Beyond overall model performance, we did not find patient characteristics that consistently predicted higher risk for every studied outcome. While the STS risk score was predictive of death and all-cause hospitalization, it was not predictive of unfavorable QOL and the other utilization outcomes. The STS risk score is the primary risk assessment tool for patients undergoing cardiac surgeries. However, it was not designed to predict minimally invasive procedures like TAVR.³³ Further, about two thirds of the patients in this study were in the high STS risk category and another one quarter were in the intermediate STS risk. Such imbalance limits the potential predictive performance of the STS risk score. These findings suggest caution is needed when applying the STS risk score to accurately risk stratify patients receiving TAVR. Risk scores specifically designed for the TAVR population³⁴ may have better predictive performance. In addition to the STS risk score, older age also had limited predictive value which is consistent with another study reporting that age was not associated with TAVR outcomes.³⁵ Chronological age may be less important given that it does not always correspond directly with frailty which is consistently associated with poorer outcomes.³⁶ More research is needed to understand the role of alternative risk scores and potential measures of frailty.

Our study has several key strengths. First, the use of a wide range of covariates from linked EHR and STS/ACC TVT Registry data, which was more comprehensive than data used in previous studies that primarily relied on a single source of data.^10–18 Second, our study population was more sociodemographically diverse than the overall STS/ACC TVT Registry, with our study having a larger share of persons of Asian or Pacific Islander race (5.9% vs. 1.5%, respectively) or Hispanic ethnicity (8.6% vs. 4.7%, respectively).⁹ Third, we included a contemporary population of TAVR patients, which is important given the expanding indications and advances in prosthetic valve technology over time. Fourth, we used machine learning-based analytic approaches that have advantages over traditional modeling methods.

There are also some limitations. Given the available sample size and observed 30-day event rates, we had limited precision for certain outcomes of interest. Patients received care within a fully integrated healthcare delivery system in California that prioritizes cardiovascular prevention and treatment, so results may not be completely generalizable to other geographic areas, less integrated health systems, or uninsured patients. In addition, even though we evaluated a rich set of sociodemographic and clinical characteristics, it is possible that other measures such as pre-TAVR frailty and other patient-reported outcomes beyond QOL and measures for social determinants of health could improve both model discrimination and calibration.^{37, 38} Further, deep learning modeling approaches such as neural networks may be able to find patterns in the data that lead to improved performance over approaches like GBM.

In sum, we found that STS risk score, older age, race/ethnicity, and many individual clinical characteristics are not consistently helpful in identifying which patient undergoing TAVR will experience short-term mortality, unfavorable QOL or excess HF-related and all-cause hospitalizations and emergency department utilization. Our findings support the need to identify factors that more accurately predict which patients are most likely to benefit from TAVR across clinical, utilization and patient-centered outcomes. Accurately identifying the subset of patients most likely to suffer adverse post-TAVR short-term outcomes could facilitate more tailored monitoring and potentially earlier intervention.

Figure 3. Calibration plots for prediction models for hospitalization and emergency department visits for any cause at 30-days post-TAVR.

Note: HL refers to Hosmer-Lemeshow test, ED refers to emergency department

APPENDIX

Appendix Figure 1.

Calibration plots for predictive model using alternative definition of quality of life improvement at 30-days post-TAVR.