Skip to main content
Journal of the American Heart Association: Cardiovascular and Cerebrovascular Disease logoLink to Journal of the American Heart Association: Cardiovascular and Cerebrovascular Disease
. 2019 Jun 5;8(12):e012355. doi: 10.1161/JAHA.119.012355

Profiling Hospital Performance on the Basis of Readmission After Transcatheter Aortic Valve Replacement in Ontario, Canada

Gabby Elbaz‐Greener 1,2, Feng Qiu 3, John G Webb 4, Kayley A Henning 3, Dennis T Ko 1,3,5,6, Andrew Czarnecki 1,3,5,6, Idan Roifman 1,3,5,6, Peter C Austin 3,5,6, Harindra C Wijeysundera 1,3,5,6,
PMCID: PMC6645639  PMID: 31165666

Abstract

Background

Readmission rates are a widely accepted quality indicator. Our objective was to develop models for calculating case‐mixed adjusted readmission rates after transcatheter aortic valve replacement for the purpose of profiling hospitals.

Methods and Results

In this population‐based study in Ontario, Canada, we identified all transcatheter aortic valve replacement procedures between April 1, 2012, and March 31, 2016. For each hospital, we first calculated 30‐day and 1‐year risk‐standardized (predicted versus expected) readmission rates, using 2‐level hierarchical logistic regression models, including clustering of patients within hospitals. We also calculated the risk‐adjusted (observed versus expected) readmission rates, accounting for the competing risk of death using a Fine‐Gray competing risk model. We categorized hospitals into 3 groups: those performing worse than expected, those performing better than expected, or those performing as expected, on the basis of whether the 95% CI was above, below, or included the provincial average readmission rate respectively. Our cohort consisted of 2129 transcatheter aortic valve replacement procedures performed at 10 hospitals. The observed readmission rate was 15.4% at 30 days and 44.2% at 1 year, with a range of 10.9% to 21.7% and 38.8% to 55.0%, respectively, across hospitals. Incorporating the competing risk of death translated into meaningful different results between models; as such, we concluded that the risk‐adjusted readmission rate was the preferred metric. On the basis of the 30‐day risk‐adjusted readmission rate, all hospitals performed as expected, with a 95% CI that included the provincial average. However, we found that there was significant variation in 1‐year risk‐adjusted readmission rate.

Conclusions

There is significant interhospital variation in 1‐year adjusted readmission rates among hospitals, suggesting that this should be a focus for quality improvement efforts in transcatheter aortic valve replacement.

Keywords: risk model, transcutaneous aortic valve implantation, TAVR, TAVR outcomes, risk‐standardized readmission rates

Subject Categories: Health Services, Quality and Outcomes


Clinical Perspective

What Is New?

  • We developed case‐mix adjustment models for the purpose of profiling hospitals to determine if is there important variation in 30‐day and 1‐year all‐cause readmission among patients who underwent transcatheter aortic valve replacement (TAVR).

  • We found no significant interhospital variation in risk‐standardized 30‐day all‐cause readmission rates after TAVR.

  • We identified important variation in 1‐year readmission rates among hospitals after case‐mix adjustment using the preferred Fine‐Gray models.

What Are the Clinical Implications?

  • As TAVR has continued to evolve, there has been extensive efforts to define, track, and improve outcomes, including reducing length of stay, readmission rates, and mortality.

  • We found that the approach taken to case‐mix adjustment had meaningful impact in how hospitals were profiled and the subsequent conclusions; the competing risk of death is a crucial consideration for TAVR, and our recommendation is to use methods that account for this when developing models for adjusted readmission rates.

  • We identified important variation in 1‐year readmission rates among hospitals after case‐mix adjustment using the preferred Fine‐Gray models, suggesting that this should be the focus of quality improvement initiatives in TAVR.

Introduction

Transcatheter aortic valve replacement (TAVR) is the preferred therapy for patients with severe symptomatic aortic stenosis who are at prohibitive1 or high risk,2 and it is a reasonable alternative for intermediate‐risk patients.3, 4, 5 This expansion of indications has led to rapid dissemination, which, in turn, has been associated with wide variation in hospital and operator experience, volume, and outcomes.6, 7, 8, 9, 10

With the transition of TAVR to being standard of care, there have been efforts to measure quality indicators to catalyze quality improvement activities, similar to what has been seen in other fields.11, 12, 13 Readmission rates are tracked for several conditions as a key quality indicator14, 15 (eg, the Centers for Medicare and Medicaid Services publicly reports 30‐day readmission rates for acute myocardial infarction, heart failure [HF], percutaneous coronary intervention, and coronary artery bypass grafting).14 The TAVR population is at a particularly high risk for readmission, with 30‐day rates ranging from 8.3% to 20.9%, with almost half related to noncardiac causes.10, 16, 17, 18, 19, 20, 21 Readmission rates are influenced by numerous factors, including patient and hospital characteristics, quality of inpatient and outpatient care, and local practice patterns.22 As more hospitals perform TAVR, it is important to monitor the extent of hospital variation to ensure the optimal delivery of care and help incentivize hospitals to implement institution‐specific strategies to reduce readmissions.

There is a paucity of literature on case‐mix adjustment models for hospital profiling related to TAVR. Prior work from our group developed case‐mix adjustment models for mortality across the 10 centers performing TAVR in Ontario, Canada, and found no significant variation.23 To our knowledge, there is only one study that has evaluated variation in 30‐day readmission rates during early commercial TAVR experience in the United States; this study showed marked variation in hospital performance.14 Accordingly, our objective was to address this gap in knowledge by evaluating a variety of methods for case‐mix adjustment to profile TAVR hospitals and determine if there is important variation in 30‐day and 1‐year all‐cause readmission.

Methods

This retrospective cohort study was approved by the Institutional Research Ethics Board at Sunnybrook Health Sciences Center, at the University of Toronto, Toronto, Ontario, Canada, before data collation and analysis. The use of anonymized administrative data held at ICES, without patient consent, is allowed in Ontario on the basis of provincial privacy legislation. Analytic methods and study materials will be available to other researchers for purposes of reproducing the results or replicating the procedure. However, individual data will not be available to be compliant with privacy regulations in Ontario. Dr Wijeysundera will be responsible for maintaining availability of analytic methods and study materials.

Context

Ontario is the largest province in Canada, with a population of 14.2 million. All residents have universal access to health care and hospital services through a publicly funded healthcare program administered by a single third‐party payer, the Ontario Ministry of Health and Long‐Term Care.

Data Sources

Our study used data collected in the CorHealth Ontario TAVR Registry, which contains information on patient demographics, comorbidities, and procedural variables from the 10 hospitals across the province that perform TAVR. These data elements have been validated through selected chart abstractions and core laboratory analyses.24, 25, 26

Data from the CorHealth Ontario TAVR Registry were linked using encoded unique patient identifiers to population‐based administrative databases housed at ICES. We used the Canadian Institute for Health Information Discharge Abstract Database for data on short‐term hospitalizations and in‐hospital complications and to supplement baseline comorbidities.27, 28 Dementia diagnoses were determined through linkage with any of the following 3 administrative databases: Ontario Health Insurance Program physician claims database, Ontario Drug Benefit database, or Canadian Institute for Health Information Discharge Abstract Database.29 Validated ICES‐derived databases were used to identify diabetes mellitus,30, 31 HF,32, 33 hypertension,34 and chronic obstructive pulmonary disease.35 Medical frailty was determined using The Johns Hopkins Adjusted Clinical Group Case‐Mix adjustment system (The Johns Hopkins ACG System, version 10).36 Mortality was ascertained via the Registered Persons Database, as were additional demographic variables, such as neighborhood income quintile and rural residence.

Patient Selection

We included all patients who underwent TAVR in Ontario from April 1, 2012, to March 31, 2016. We excluded episodes with data quality issues (ie, patients with missing income, rurality, and access site). For patients with >1 TAVR record, we included only the first record.

Outcome Variables

Patients were followed up from the date of procedure until March 31, 2017. Our primary clinical outcomes of interest were all‐cause readmission within 30 days and 1 year after TAVR procedure. Secondary outcomes were all‐cause mortality, as well as postprocedural complications, including pacemaker implantation, stroke, myocardial infarction, bleeding, and acute renal injury events that occurred during the index hospitalization. We also examined wait times and length of hospital stay.

Statistical Analysis

In general, there are 2 ways of reporting standardized results. The first is risk‐standardized rates, which is based on the ratio of predicted/expected results. The second is risk‐adjusted rates, which is based on the ratio of observed/expected results. The former requires the incorporation of cluster‐specific random effects, which the latter does not. In contrast, the latter can be calculated using a Fine‐Gray model, accounting for competing risks (eg, death), which is especially relevant in TAVR, in which postprocedural mortality is nontrivial. However, a Fine‐Gray model cannot incorporate random effects and, therefore, cannot be used to calculate risk‐standardized rates. We studied both approaches to standardization to understand which is preferable for hospital profiling in TAVR.

Demographics and clinically relevant patient‐level variables were selected on the basis of a thorough review of previous studies.9, 17, 18, 19, 37, 38, 39, 40, 41 Given that there is no gold‐standard approach for variable selection, we used a variety of methods. First, we used a backwards variable elimination process. We began by assessing the statistical significance of the univariate association between each covariate and the outcome. All covariates whose univariate statistical significance was <0.1 were forced into a multivariable model. Backwards variable elimination was then used to develop a parsimonious regression model. Those variables whose adjusted statistical significance was <0.1 were retained in the final model. The second approach was to force all 28 clinically relevant variables into the models, with no subsequent model simplification. For the Fine‐Gray models, given the computation complexity, we only used the clinically relevant approach.

Validation

Internal validation was performed using bootstrapping, with model development repeated in each bootstrap sample, and then discrimination was assessed by optimism‐corrected estimates of the C‐statistic. Details of these full processes have been previously described in prior work from our group.23 Calibration was examined by plotting observed versus predicted readmission rates across the deciles of predicted risk.

Calculation of Risk‐Standardized and Risk‐Adjusted Readmission Rates

Risk standardization is less sensitive to the effects of small‐volume hospitals.42, 43, 44 We calculated a hospital‐specific risk‐standardized readmission rate (RSRR; 30‐day and 1‐year RSRR) using the estimated hospital‐specific parameters from the hierarchical logistic models. RSRRs are calculated as the ratio of “predicted” (including the average intercept and hospital‐specific random effect)/“expected” (in an “average” center with the same case‐mix distribution, but without the hospital‐specific random effect [ie, a hospital whose random effect was equal to 0]), multiplied by the provincial unadjusted readmission rate.43, 45, 46

We calculated the risk‐adjusted readmission rates (RARRs), defined as the ratio of the “observed” rates/“expected” rates, multiplied by the overall provincial unadjusted readmission rates. For each hospital, the observed probability of the outcome (readmission) was determined using a cumulative incidence function to estimate the incidence of readmission at 30 days and 1 year after accounting for the competing risk of death. The expected probability of admission for each patient was computed by fitting a Fine‐Gray competing risk regression model to the entire sample (using the appropriately selected covariates). Using the fitted model, we obtained an expected probability of readmission within 30 days and 1 year for each patient. The hospital‐specific expected rate of the outcome was computed as the mean of these patient‐specific probabilities for the patients undergoing the procedure at that hospital.

We calculated the 95% CIs for each hospital's RSRR and RARR using bootstrapping. If an individual hospital's entire 95% CI was above versus below the provincial mean, that hospital performance was categorized as worse versus better than expected, respectively. If the 95% CI included the provincial mean, the hospital performance was classified as expected.

All data analyses were performed using SAS, version 9.4 (SAS Institute Inc, Cary, NC). Statistical significance was considered to be a 2‐sided P<0.05.

Results

Study Cohort

After applying the exclusion criteria, our TAVR cohort included 2129 patients who underwent TAVR procedures between the years 2012 and 2016 at 10 hospitals across Ontario (Figure 1). For the purpose of model derivation, we excluded an additional 16 patients because of missing data, for a final modeling cohort of 2113. There was a substantial difference in the volume of TAVR procedures performed at each hospital, with an interhospital range (IHR) of 60 to 376. As seen in Table 1, 80% (IHR, 58.2%–87.3%) of the procedures were performed in elective outpatients and 82% (IHR, 56.4%–94.4%) were performed via transfemoral access. In the total cohort, the median age was 83 years (interquartile range, 78–87 years), and 46% of the patients were women. Patient characteristics varied substantially across hospitals, in particular for comorbidities such as frailty, coronary artery disease, peripheral vascular disease, hypertension, dyslipidemia, HF, renal disease, and dementia (Table 1).

Figure 1.

Figure 1

Cohort selection. TAVR indicates transcatheter aortic valve replacement.

Table 1.

Baseline Characteristics

Variable Label Total Cohort Range Across Centersa P Value
TAVR procedure, N 2129 60–376
Demographic characteristics
Age, median (IQR), y 83 (78–87) 82 (77–86)–85 (80–89) <0.001
Sex, women, N (%) 975 (45.8) 23.3–51.3 0.03
Income quintile, N (%)
1 341 (16.0) 10.7–25.0 0.15
2 444 (20.9) 14.3–22.9
3 442 (20.8) 17.6–25.0
4 451 (21.2) 16.3–29.6
5 439 (20.6) 16.4–24.7
Rural resident, N (%) 257 (12.1) 2.7–31.7 <0.001
Medical comorbidities
Charlson score, mean±SD 1.95±1.91 1.54±1.75–2.59±2.09 <0.001
Frailty, N (%) 462 (21.7) 14.4–32.4 <0.001
DM, N (%) 986 (46.3) 38.3–53.1 0.21
Dyslipidemia, N (%) 1468 (69.0) 36.8–80.1 <0.001
Hypertension, N (%) 2015 (94.6) 82.4–97.3 <0.001
CHF, N (%) 1606 (75.4) 57.7–86.2 <0.001
COPD, N (%) 770 (36.2) 29.3–50.0 0.12
Dementia, N (%) 156 (7.3) 4.0–14.2 0.001
Malignancy, N (%) 143 (6.7) 2.9–11.2 0.17
Renal disease, N (%) 237 (11.1) 5.1–25.0 0.003
Dialysis, N (%) 77 (3.6) 1.0–5.3 0.27
Liver disease, N (%) 29 (1.4) 0.0–2.7 0.14
ILD, N (%) 30 (1.4) 0.0–2.4 0.60
Cardiac arrhythmia/AF, N (%) 558 (26.2) 19.4–33.8 0.15
CAD, N (%) 1526 (71.7) 62.0–86.7 <0.001
CVD, N (%) 116 (5.4) 4.0–10.0 0.45
PVD, N (%) 117 (5.5) 0.0–16.0 <0.001
Previous cardiosurgery procedure, N (%)
Previous PCI 775 (36.4) 17.7–55.0 <0.001
Previous CABG 509 (23.9) 17.3–37.0 <0.001
Previous valve surgery 303 (14.2) 2.9–21.7 <0.001
TAVR valve in valve, N (%) 208 (9.8) 1.5–15.2 <0.001
TAVR access site, N (%)
Nontransfemoral 388 (18.2) 1.5–43.6 <0.001
Transfemoral 1737 (81.6) 56.4–94.4
TAVR procedure status, N (%)
Elective 1702 (79.9) 58.2–87.3 <0.001
Urgent/emergent 427 (20.1) 12.7–41.8
Fiscal year, N (%)
2012 309 (14.5) 0.0–18.7 <0.001
2013 462 (21.7) 14.3–25.1
2014 633 (29.7) 26.2–41.7
2015 725 (34.1) 29.5–51.0

AF indicates atrial fibrillation; CABG, coronary artery bypass graft; CAD, coronary artery disease; CHF, congestive heart failure; COPD, chronic obstructive lung disease; CVD, cerebrovascular disease; DM, diabetes mellitus; ILD, interstitial lung disease; IQR, interquartile range; PCI, percutaneous coronary intervention; PVD, peripheral vascular disease; TAVR, transcatheter aortic valve replacement.

a

Data are given as percentage, unless otherwise indicated.

Unadjusted Outcomes

The observed, unadjusted, 30‐day readmission rate for the province was 15.4%, whereas the observed, unadjusted, 1‐year readmission rate was 44.2% (Table 2). There was a substantial range in unadjusted readmission rates across hospitals (30‐day IHR, 10.9%–21.7% [P=0.11]; 1‐year IHR, 38.8%–55.0% [P=0.005]). The causes of admissions at both 30 days and 1 year are found in Table S1. As seen, most admissions were noncardiac. The most frequent cardiac readmission was for HF (3.9% of admissions for 30 days and 11.2% of admissions for 1 year). Unadjusted mortality rates at 30 days and 1 year were 7.0% and 16.4%, respectively. There was also substantial variation between hospitals in the rates of in‐hospital complications, TAVR wait times, and the hospital length of stay (Table 2).

Table 2.

TAVR Wait Times and Unadjusted Outcome

Variable Label Total Cohort Range Across Centersa P Value
TAVR procedure, N 2129 60–376
Readmission after TAVR procedure, N (%)
Within 30 d 327 (15.4) 41 (10.9)–13 (21.7) 0.113
Within 1 y 924 (44.2) 38 (38.8)–33 (55.0) 0.005
In‐hospital complication, secondary outcome, N (%)
Pacemaker insertion 279 (13.1) 6.3–20.5 <0.01
Stroke 38 (1.8) 0.0–3.3 0.05
Dialysis 65 (3.1) 2.7–4.8 0.05
Myocardial infarction 12 (0.6) 0.0–1.3 0.77
Acute kidney injury 46 (2.2) 0.0–3.6 0.12
Bleeding <0.001
Major 139 (6.5) 2.6–15.6
Minor 77 (3.6) 0.0–6.6
Wait time, mean±SD, d
From referral to TAVR procedure 131.7±117.0 66.6±39.5–210.7±163.2 <0.001
From eligible decision to TAVR procedure 57.6±62.1 23.0±24.4–80.5±90.0 <0.001
Length of stay, mean±SD, d
From TAVR admission to discharge 12.0±22.7 7.7±16.8–22.2±20.7 <0.001
From TAVR procedure to discharge 9.1±20.1 6.4±9.4–15.3±16.8 <0.001

TAVR indicates transcatheter aortic valve replacement.

a

Data are given as percentage, unless otherwise indicated.

Multivariable Case‐Mix Adjustment Models

The clinically relevant and backward logistic models for 30‐day readmission are found in Tables S2 and S3, whereas the Fine‐Gray model is found in Table S4. The corresponding models for 1‐year readmission are found in Tables S5 through S7. We found that the clinically relevant logistic model had greater discrimination compared with the clinically relevant Fine‐Gray model, with an optimism‐corrected C‐statistic of 0.64 versus 0.60, respectively, for the 1‐year outcomes and the same discrimination (ie, 0.62) for the 30‐day outcomes (Table S8). All models had excellent calibration on the basis of observed to predicted plots; that said, the 1‐year clinically relevant Fine‐Gray model appeared to perform better than the clinically relevant logistic model (Figure S1). On the basis of these factors, we concluded that the clinically relevant Fine‐Gray models were the preferred approach for both 30‐day and 1‐year outcomes, both in terms of performance and the theoretical issues for competing risk of 1‐year death, which was relatively frequent at ≈15%.

In the clinically relevant Fine‐Gray models, the factors with the strongest association with 30‐day readmission were nonfemoral access (odds ratio [OR], 2.33) and HF (OR, 1.37). The factors with the strongest association with 1‐year readmission were arrhythmia/atrial fibrillation (OR, 1.47), nonfemoral access (OR, 1.46), peripheral vascular disease (OR, 1.44), HF (OR, 1.30), frailty (OR, 1.24), and chronic obstructive pulmonary disease (OR, 1.22). Given the time span of our cohort, we included fiscal year in the models; although the point estimates suggest improved outcomes with procedures done in the more recent years, these did not reach statistical significance (Tables S4 and S7).

Risk‐Standardized and Risk‐Adjusted Readmission

The observed, predicted, and expected readmission rates are found in Tables S9 and S10. In Figure 2A, on the basis of these quantities and the respective 95% CI, we plotted the 30‐day RARR of each hospital against the provincial mean, using the preferred clinically relevant Fine‐Gray model. For comparison, in Figure 2B and 2C, we show the same plot when calculating 30‐day RSRR using the clinically relevant and backward selection hierarchical logistic models, respectively. In all cases, the hospitals performed as expected, with 95% CIs that included the provincial mean.

Figure 2.

Figure 2

Risk‐standardized 30‐day and 1‐year all‐cause readmission. A, Risk‐standardized 30‐day readmission rate for clinically variables. B, Risk‐standardized 30‐day readmission rate for backward variables. C, Fine‐Gray competing model, 30‐day readmission, accounting for the competing risk of death. D, Risk‐standardized 1‐year readmission rate for clinically variables. E, Risk‐standardized 1‐year readmission rate for backward variables. F, Fine‐Gray competing model, 1‐year readmission, accounting for the competing risk of death.

In Figure 2D through 2F, similar plots for 1‐year RARR and RSRR are shown. In contrast with the 30‐day results, we found qualitatively different results among the approaches. With the Fine‐Gray model 1‐year RARR, we found that there was substantial variation, with one hospital performing better than expected and one hospital performing worse than expected. In contrast, when using the 1‐year RSRR as the metric of reporting, regardless of variable selection method, all hospitals performed as expected.

Discussion

In this population‐based study of all TAVRs in Ontario, we found substantial variation in 1‐year readmission rates among hospitals after case‐mix adjustment using the preferred Fine‐Gray models. More important, we found that the approach taken to case‐mix adjustment had meaningful impact in how hospitals were profiled and the subsequent conclusions. The competing risk of death is a crucial consideration for TAVR, and our recommendation is to use methods that account for this when developing models for adjusted readmission rates. We identified important variation in 1‐year readmission rates, suggesting that this should be the focus of quality improvement initiatives in TAVR.

The National Quality Strategy and the Partnership for Patients initiative include reduction in readmissions as a national goal.15 Historically, 20% of all Medicare discharges had a readmission within 30 days.16 The Medicare Payment Advisory Commission has estimated that 12% of readmissions are potentially avoidable.14 Hospital readmissions are associated with worse patient outcomes and high financial costs.22, 47 Given this, there have been efforts to initiate programs aiming to reduce the readmission rate.48, 49 Causes of readmissions are multifactorial, and rates vary substantially by institution.50, 51 Reducing readmission rates is a cost‐reduction goal, but more important, it is a patient‐centric goal and a target for quality improvement.

As TAVR has continued to evolve, there have been extensive efforts to define, track, and improve outcomes, including reducing length of stay, readmission rates, and mortality.19, 37, 52 All have been aimed at improving healthcare efficiency, healthcare quality, and value of care delivered. Appropriate risk‐adjusted models in the TAVR are necessary to support these efforts. In TAVR, there has been previous work done to profile hospitals on the basis of mortality.19, 37, 52 The more contemporary literature suggests that there is no important variation in mortality among TAVR hospitals. In contrast, there is a lack of similar work on readmission. Murugiah and colleagues demonstrated wide interhospital variation with respect to 30‐day all‐cause readmission after TAVR.19 Our study builds on this previous work. We found no significant interhospital variation on 30‐day readmission rates. However, we did find a substantial difference in 1‐year readmission. To keep in context, of the 10 TAVR hospitals, only 2 were significantly different, with one performing worse than expected and the other performing better than expected.

Several notable points about this observation merit discussion. First, although most drivers of readmission were nonmodifiable patient comorbidities, there were discretionary drivers, such as nonfemoral access, that are modifiable. Second, we did not include complications, such as pacemaker need, in our models. Such complications are likely related to subsequent readmission; and given that they are in the causal pathway, we did not adjust for them in our modelling. Indeed, to do so would potentially dilute important differences between hospitals.

Third, to date, there is no gold standard with regard to the correct statistical method of profiling hospitals. Hierarchical models are powerful statistical tools that have been the focus of much development in recent years.53 However, they do not account for the competing risk of death, which is important in TAVR. To account for this risk, we developed the Fine‐Gray model and found qualitatively different conclusions on variation in readmission rates.

Finally, given the variation in 1‐year readmission despite accounting for case‐mix, it suggests that there are factors between hospitals that are different that may account for this variation. Our study was not designed to identify the process or system differences associated with lower readmission, but we hypothesize it may be related to better transition services, from in‐hospital care to out‐of‐hospital care and infrastructure, to support ambulatory care for these complex patients, which may obviate the need for hospital admission (Figure 3). This is an important area for further study; and given the small number of hospitals involved, it is an opportunity to understand the specific process differences among the hospitals that underperformed, versus those that overperformed, versus the majority that performed as expected.

Figure 3.

Figure 3

Conceptual framework for readmission intervention. TAVR indicates transcatheter aortic valve replacement.

Our study must be interpreted in the context of several limitations that merit discussion. First, to account for case‐mix, we used variables that are available in our data set. Our models performed adequately, with excellent calibration and moderate to good discrimination. Broadly, these are consistent with other case‐mix adjustment models for hospital profiling in other conditions, including acute myocardial infarction and HF.44, 54 However, we did not have potentially important elements, such as left ventricular ejection fraction and Society of Thoracic Surgeons and Euro scores, as these are not available in the data set. As such, we cannot rule out residual confounding. Second, we incorporated year of implant in our models, to account for temporal improvements in care delivery. Moving forward, if this model was to be used for annual reporting, the year of implant would be excluded from subsequent models. Finally, we did not include cause‐specific admission in our modeling, but rather all‐cause readmission, given that this is the major driver of healthcare costs.

In conclusion, using RSRR in a contemporary cohort of TAVR hospitals in Ontario, we found that all 10 hospitals in the province performed as expected on short‐term readmission, but that there was substantial variation on 1‐year readmission. Furthermore, nonfemoral access as well as patient characteristics, such as HF, atrial fibrillation, peripheral vascular disease, lung disease, and frailty, had a strong association with readmission. These findings highlight potential areas of focus for quality improvement efforts.

Sources of Funding

This study was supported by ICES, which is funded by an annual grant from the Ontario Ministry of Health and Long‐Term Care (MOHLTC). The opinions, results, and conclusions reported in this article are those of the authors and are independent from the funding sources. No endorsement by ICES or the Ontario MOHLTC is intended or should be inferred. Parts of this material are based on data and/or information compiled and provided by the Canadian Institute for Health Information (CIHI). However, the analyses, conclusions, opinions, and statements expressed in the material are those of the authors, and not necessarily those of CIHI. Dr Wijeysundera is supported by a Distinguished Clinical Scientist Award from the Heart and Stroke Foundation of Canada. Dr Austin is supported by a Mid‐Career Investigator award from the Heart and Stroke Foundation.

Disclosures

Dr Wijeysundera receives research funding from Medtronic Inc and Edwards Life Sciences. The remaining authors have no disclosures to report.

Supporting information

Table S1. Causes of Readmission

Table S2. Final Hierarchical Model for Clinically Relevant Variables; 30‐Day All Cause Readmission Following TAVR

Table S3. Final Hierarchical Model for Backward Selected Variables: 30‐Day All Cause Readmission Following TAVR

Table S4. Fine Gray Model, 30‐Day All Cause Readmission Rate for Hospital

Table S5. Final Hierarchical Model for Clinically Relevant Variables; 1‐Year All Cause Readmission Following TAVR

Table S6. Final Hierarchical Model for Backward Selected Variables: 1‐Year All Cause Readmission Following TAVR

Table S7. Fine Gray Model, 1‐Year All Cause Readmission Rate for Hospital

Table S8. Models Discrimination Parameters

Table S9. Observed and Expected 30‐Day Readmission—Fine Gray Model

Table S10. Observed and Expected 1‐Year Readmission‐Fine Gray Model

Figure S1. Calibration models.

Acknowledgments

The authors acknowledge that the clinical registry data used in this publication are from participating hospitals through CorHealth Ontario, which serves as an advisory body to the Ontario Ministry of Health and Long‐Term Care (MOHLTC), is funded by the MOHLTC, and is dedicated to improving the quality, efficiency, access, and equity in the delivery of the continuum of adult cardiac, vascular, and stroke services in Ontario, Canada. We thank IMS Brogan for use of its Drug Information Database. The corresponding author affirms that he has listed everyone who contributed significantly to the work. The authors had access to all the study data, take responsibility for the accuracy of the analysis, and had authority over manuscript preparation and the decision to submit the manuscript for publication. The corresponding author confirms that all authors read and approve the manuscript.

(J Am Heart Assoc. 2019;8:e012355 DOI: 10.1161/JAHA.119.012355.)

References

  • 1. Leon MB, Smith CR, Mack M, Miller DC, Moses JW, Svensson LG, Tuzcu EM, Webb JG, Fontana GP, Makkar RR, Brown DL, Block PC, Guyton RA, Pichard AD, Bavaria JE, Herrmann HC, Douglas PS, Petersen JL, Akin JJ, Anderson WN, Wang D, Pocock S; PARTNER Trial Investigators . Transcatheter aortic‐valve implantation for aortic stenosis in patients who cannot undergo surgery. N Engl J Med. 2010;363:1597–1607. [DOI] [PubMed] [Google Scholar]
  • 2. Smith CR, Leon MB, Mack MJ, Miller DC, Moses JW, Svensson LG, Tuzcu EM, Webb JG, Fontana GP, Makkar RR, Williams M, Dewey T, Kapadia S, Babaliaros V, Thourani VH, Corso P, Pichard AD, Bavaria JE, Herrmann HC, Akin JJ, Anderson WN, Wang D, Pocock SJ; PARTNER Trial Investigators . Transcatheter versus surgical aortic‐valve replacement in high‐risk patients. N Engl J Med. 2011;364:2187–2198. [DOI] [PubMed] [Google Scholar]
  • 3. Leon MB, Smith CR, Mack MJ, Makkar RR, Svensson LG, Kodali SK, Thourani VH, Tuzcu EM, Miller DC, Herrmann HC, Doshi D, Cohen DJ, Pichard AD, Kapadia S, Dewey T, Babaliaros V, Szeto WY, Williams MR, Kereiakes D, Zajarias A, Greason KL, Whisenant BK, Hodson RW, Moses JW, Trento A, Brown DL, Fearon WF, Pibarot P, Hahn RT, Jaber WA, Anderson WN, Alu MC, Webb JG; PARTNER 2 Investigators . Transcatheter or surgical aortic‐valve replacement in intermediate‐risk patients. N Engl J Med. 2016;374:1609–1620. [DOI] [PubMed] [Google Scholar]
  • 4. Jones DA, Tchetche D, Forrest J, Hellig F, Lansky A, Moat N. The SURTAVI study: TAVI for patients with intermediate risk. EuroIntervention. 2017;13:e617–e620. [DOI] [PubMed] [Google Scholar]
  • 5. Waksman R, Rogers T, Torguson R, Gordon P, Ehsan A, Wilson SR, Goncalves J, Levitt R, Hahn C, Parikh P, Bilfinger T, Butzel D, Buchanan S, Hanna N, Garrett R, Asch F, Weissman G, Ben‐Dor I, Shults C, Bastian R, Craig PE, Garcia‐Garcia HM, Kolm P, Zou Q, Satler LF, Corso PJ. Transcatheter aortic valve replacement in low‐risk patients with symptomatic severe aortic stenosis. J Am Coll Cardiol. 2018;72:2095–2105. [DOI] [PubMed] [Google Scholar]
  • 6. Carroll JD, Vemulapalli S, Dai D, Matsouaka R, Blackstone E, Edwards F, Masoudi FA, Mack M, Peterson ED, Holmes D, Rumsfeld JS, Tuzcu EM, Grover F. Procedural experience for transcatheter aortic valve replacement and relation to outcomes: the STS/ACC TVT registry. J Am Coll Cardiol. 2017;70:29–41. [DOI] [PubMed] [Google Scholar]
  • 7. Mylotte D, Osnabrugge RLJ, Windecker S, Lefevre T, de Jaegere P, Jeger R, Wenaweser P, Maisano F, Moat N, Sondergaard L, Bosmans J, Teles RC, Martucci G, Manoharan G, Garcia E, Van Mieghem NM, Kappetein AP, Serruys PW, Lange R, Piazza N. Transcatheter aortic valve replacement in Europe: adoption trends and factors influencing device utilization. J Am Coll Cardiol. 2013;62:210–219. [DOI] [PubMed] [Google Scholar]
  • 8. Krasopoulos G, Falconieri F, Benedetto U, Newton J, Sayeed R, Kharbanda R, Banning A. European real world trans‐catheter aortic valve implantation: systematic review and meta‐analysis of European national registries. J Cardiothorac Surg. 2016;11:159–167. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 9. Hermiller JB Jr, Yakubov SJ, Reardon MJ, Deeb GM, Adams DH, Afilalo J, Huang J, Popma JJ; CoreValve United States Clinical Investigators . Predicting early and late mortality after transcatheter aortic valve replacement. J Am Coll Cardiol. 2016;68:343–352. [DOI] [PubMed] [Google Scholar]
  • 10. Khera S, Kolte D, Gupta T, Goldsweig A, Velagapudi P, Kalra A, Tang GHL, Aronow WS, Fonarow GC, Bhatt DL, Aronow HD, Kleiman NS, Reardon M, Gordon PC, Sharaf B, Abbott JD. Association between hospital volume and 30‐day readmissions following transcatheter aortic valve replacement. JAMA Cardiol. 2017;2:732–741. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 11. Agabiti N, Ancona C, Forastiere F, Arca M, Perucci CA. Evaluating outcomes of hospital care following coronary artery bypass surgery in Rome, Italy. Eur J Cardiothorac Surg. 2003;23:599–606; discussion 607–608. [DOI] [PubMed] [Google Scholar]
  • 12. Dimick JB, Staiger DO, Hall BL, Ko CY, Birkmeyer JD. Composite measures for profiling hospitals on surgical morbidity. Ann Surg. 2013;257:67–72. [DOI] [PubMed] [Google Scholar]
  • 13. Peterson ED, Dai D, DeLong ER, Brennan JM, Singh M, Rao SV, Shaw RE, Roe MT, Ho KK, Klein LW, Krone RJ, Weintraub WS, Brindis RG, Rumsfeld JS, Spertus JA; NCDR Registry Participants . Contemporary mortality risk prediction for percutaneous coronary intervention: results from 588,398 procedures in the National Cardiovascular Data Registry. J Am Coll Cardiol. 2010;55:1923–1932. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 14. 30‐Day unplanned readmission and death measures. 2016. Medicare.gov. https://www.medicare.gov/hospitalcompare/Data/30-day-measures.html. Accessed August 7, 2018.
  • 15. Overview of key readmission measures and methods report 2012‐04. https://www.hcup-us.ahrq.gov/reports/methods/2012_04.pdf. Accessed November 12, 2018.
  • 16. Nombela‐Franco L, del Trigo M, Morrison‐Polo G, Veiga G, Jimenez‐Quevedo P, Abdul‐Jawad Altisent O, Campelo‐Parada F, Biagioni C, Puri R, DeLarochelliere R, Dumont E, Doyle D, Paradis JM, Quiros A, Almeria C, Gonzalo N, Nunez‐Gil I, Salinas P, Mohammadi S, Escaned J, Fernandez‐Ortiz A, Macaya C, Rodes‐Cabau J. Incidence, causes, and predictors of early (</=30 days) and late unplanned hospital readmissions after transcatheter aortic valve replacement. JACC Cardiovasc Interv. 2015;8:1748–1757. [DOI] [PubMed] [Google Scholar]
  • 17. Kolte D, Khera S, Sardar MR, Gheewala N, Gupta T, Chatterjee S, Goldsweig A, Aronow WS, Fonarow GC, Bhatt DL, Greenbaum AB, Gordon PC, Sharaf B, Abbott JD. Thirty‐day readmissions after transcatheter aortic valve replacement in the United States: insights from the Nationwide Readmissions Database. Circ Cardiovasc Interv. 2017;10:e004472. [DOI] [PubMed] [Google Scholar]
  • 18. Holmes DR Jr, Brennan JM, Rumsfeld JS, Dai D, O'Brien SM, Vemulapalli S, Edwards FH, Carroll J, Shahian D, Grover F, Tuzcu EM, Peterson ED, Brindis RG, Mack MJ; STS/ACC TVT Registry . Clinical outcomes at 1 year following transcatheter aortic valve replacement. JAMA. 2015;313:1019–1028. [DOI] [PubMed] [Google Scholar]
  • 19. Murugiah K, Wang Y, Desai NR, Nuti SV, Krumholz HM. Hospital variation in outcomes for transcatheter aortic valve replacement among Medicare beneficiaries, 2011 to 2013. J Am Coll Cardiol. 2015;66:2678–2679. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 20. Hannan EL, Samadashvili Z, Jordan D, Sundt TM III, Stamato NJ, Lahey SJ, Gold JP, Wechsler A, Ashraf MH, Ruiz C, Wilson S, Smith CR. Thirty‐day readmissions after transcatheter aortic valve implantation versus surgical aortic valve replacement in patients with severe aortic stenosis in New York State. Circ Cardiovasc Interv. 2015;8:e002744. [DOI] [PubMed] [Google Scholar]
  • 21. Society of Thoracic Surgeons; American College of Cardiology . STS/ACC TVT Registry. https://www.ncdr.com/webncdr/tvt/publicpage/data-collection. Accessed April 17, 2018.
  • 22. Jencks SF, Williams MV, Coleman EA. Rehospitalizations among patients in the Medicare fee‐for‐service program. N Engl J Med. 2009;360:1418–1428. [DOI] [PubMed] [Google Scholar]
  • 23. Elbaz‐Greener G, Qiu F, Masih S, Fang J, Austin PC, Cantor WJ, Dvir D, Asgar AW, Webb JG, Ko DT, Wijeysundera HC. Profiling hospital performance based on mortality after transcatheter aortic valve replacement in Ontario, Canada. Circ Cardiovasc Qual Outcomes. 2018;11:e004947. [DOI] [PubMed] [Google Scholar]
  • 24. Bennell MC, Qiu F, Kingsbury KJ, Austin PC, Wijeysundera HC. Determinants of variations in initial treatment strategies for stable ischemic heart disease. CMAJ. 2015;187:E317–E325. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 25. Ko DT, Tu JV, Austin PC, Wijeysundera HC, Samadashvili Z, Guo H, Cantor WJ, Hannan EL. Prevalence and extent of obstructive coronary artery disease among patients undergoing elective coronary catheterization in New York State and Ontario. JAMA. 2013;310:163–169. [DOI] [PubMed] [Google Scholar]
  • 26. Tu JV, Ko DT, Guo H, Richards JA, Walton N, Natarajan MK, Wijeysundera HC, So D, Latter DA, Feindel CM, Kingsbury K, Cohen EA; Cardiac Care Network of Ontario's Variations in Revascularization Practice in Ontario Working Group . Determinants of variations in coronary revascularization practices. CMAJ. 2012;184:179–186. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 27. Deyo RA, Cherkin DC, Ciol MA. Adapting a clinical comorbidity index for use with ICD‐9‐CM administrative databases. J Clin Epidemiol. 1992;45:613–619. [DOI] [PubMed] [Google Scholar]
  • 28. Quan H, Sundararajan V, Halfon P, Fong A, Burnand B, Luthi JC, Saunders LD, Beck CA, Feasby TE, Ghali WA. Coding algorithms for defining comorbidities in ICD‐9‐CM and ICD‐10 administrative data. Med Care. 2005;43:1130–1139. [DOI] [PubMed] [Google Scholar]
  • 29. Jaakkimainen RL, Bronskill SE, Tierney MC, Herrmann N, Green D, Young J, Ivers N, Butt D, Widdifield J, Tu K. Identification of physician‐diagnosed Alzheimer's disease and related dementias in population‐based administrative data: a validation study using family physicians’ electronic medical records. J Alzheimers Dis. 2016;54:337–349. [DOI] [PubMed] [Google Scholar]
  • 30. Guttmann A, Nakhla M, Henderson M, To T, Daneman D, Cauch‐Dudek K, Wang X, Lam K, Hux J. Validation of a health administrative data algorithm for assessing the epidemiology of diabetes in Canadian children. Pediatr Diabetes. 2010;11:122–128. [DOI] [PubMed] [Google Scholar]
  • 31. Hux JE, Ivis F, Flintoft V, Bica A. Diabetes in Ontario: determination of prevalence and incidence using a validated administrative data algorithm. Diabetes Care. 2002;25:512–516. [DOI] [PubMed] [Google Scholar]
  • 32. Schultz SE, Rothwell DM, Chen Z, Tu K. Identifying cases of congestive heart failure from administrative data: a validation study using primary care patient records. Chronic Dis Inj Can. 2013;33:160–166. [PubMed] [Google Scholar]
  • 33. Tu K, Chen Z, Lipscombe LL; Canadian Hypertension Education Program Outcomes Research Taskforce . Prevalence and incidence of hypertension from 1995 to 2005: a population‐based study. CMAJ. 2008;178:1429–1435. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 34. Tu K, Campbell NR, Chen ZL, Cauch‐Dudek KJ, McAlister FA. Accuracy of administrative databases in identifying patients with hypertension. Open Med. 2007;1:e18–e26. [PMC free article] [PubMed] [Google Scholar]
  • 35. Gershon AS, Wang C, Guan J, Vasilevska‐Ristovska J, Cicutto L, To T. Identifying individuals with physician diagnosed COPD in health administrative databases. COPD. 2009;6:388–394. [DOI] [PubMed] [Google Scholar]
  • 36. Weiner JP. The Johns Hopkins Adjusted Clinical Group Technical Reference Guide, version 10.0. 2011. https://studylib.net/doc/8365333/the-johns-hopkins-acg%C2%AE-system-technical-reference-guide. Accessed April 15, 2017.
  • 37. O'Brien SM, Cohen DJ, Rumsfeld JS, Brennan JM, Shahian DM, Dai D, Holmes DR, Hakim RB, Thourani VH, Peterson ED, Edwards FH; STS/ACC TVT Registry . Variation in hospital risk‐adjusted mortality rates following transcatheter aortic valve replacement in the United States: a report from the Society of Thoracic Surgeons/American College of Cardiology Transcatheter Valve Therapy Registry. Circ Cardiovasc Qual Outcomes. 2016;9:560–565. [DOI] [PubMed] [Google Scholar]
  • 38. Arnold SV, Reynolds MR, Lei Y, Magnuson EA, Kirtane AJ, Kodali SK, Zajarias A, Thourani VH, Green P, Rodes‐Cabau J, Beohar N, Mack MJ, Leon MB, Cohen DJ; PARTNER Investigators . Predictors of poor outcomes after transcatheter aortic valve replacement: results from the PARTNER (Placement of Aortic Transcatheter Valve) trial. Circulation. 2014;129:2682–2690. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 39. Arnold SV, Afilalo J, Spertus JA, Tang Y, Baron SJ, Jones PG, Reardon MJ, Yakubov SJ, Adams DH, Cohen DJ; U. S. CoreValve Investigators . Prediction of poor outcome after transcatheter aortic valve replacement. J Am Coll Cardiol. 2016;68:1868–1877. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 40. Seiffert M, Sinning JM, Meyer A, Wilde S, Conradi L, Vasa‐Nicotera M, Ghanem A, Kempfert J, Hammerstingl C, Ojeda FM, Kim WK, Koschyk DH, Schirmer J, Baldus S, Grube E, Mollmann H, Reichenspurner H, Nickenig G, Blankenberg S, Diemert P, Treede H, Walther T, Werner N, Schnabel RB. Development of a risk score for outcome after transcatheter aortic valve implantation. Clin Res Cardiol. 2014;103:631–640. [DOI] [PubMed] [Google Scholar]
  • 41. Iung B, Laouenan C, Himbert D, Eltchaninoff H, Chevreul K, Donzeau‐Gouge P, Fajadet J, Leprince P, Leguerrier A, Lievre M, Prat A, Teiger E, Laskar M, Vahanian A, Gilard M; FRANCE 2 Investigators . Predictive factors of early mortality after transcatheter aortic valve implantation: individual risk assessment using a simple score. Heart. 2014;100:1016–1023. [DOI] [PubMed] [Google Scholar]
  • 42. Normand SL, Shahian DM. Statistical and clinical aspects of hospital outcomes profiling. Stat Sci. 2007;22:206–226. [Google Scholar]
  • 43. Nicholl J, Jacques RM, Campbell MJ. Direct risk standardisation: a new method for comparing casemix adjusted event rates using complex models. BMC Med Res Methodol. 2013;13:133–137. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 44. Yale New Haven Health Services Corporation/Center for Outcomes Research and Evaluation (YNHHSC/CORE) . 2017 Condition‐specific measures updates and specifications report hospital level 30‐day risk standardized mortality measures. 2017. https://medicine.yale.edu/core/current_projects/quality_measurement/. Access November 12, 2018.
  • 45. Normand SL, Glickman ME, Gatsonis CA. Statistical methods for profiling providers of medical care: issues and applications. J Am Stat Assoc. 1997;92:803–814. [Google Scholar]
  • 46. Krumholz HM, Wang Y, Mattera JA, Wang Y, Han LF, Ingber MJ, Roman S, Normand SL. An administrative claims model suitable for profiling hospital performance based on 30‐day mortality rates among patients with heart failure. Circulation. 2006;113:1693–1701. [DOI] [PubMed] [Google Scholar]
  • 47. Bueno H, Ross JS, Wang Y, Chen J, Vidan MT, Normand SL, Curtis JP, Drye EE, Lichtman JH, Keenan PS, Kosiborod M, Krumholz HM. Trends in length of stay and short‐term outcomes among Medicare patients hospitalized for heart failure, 1993–2006. JAMA. 2010;303:2141–2147. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 48. McIlvennan CK, Eapen ZJ, Allen LA. Hospital readmissions reduction program. Circulation. 2015;131:1796–1803. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 49. Zuckerman RB, Sheingold SH, Orav EJ, Ruhter J, Epstein AM. Readmissions, observation, and the hospital readmissions reduction program. N Engl J Med. 2016;374:1543–1551. [DOI] [PubMed] [Google Scholar]
  • 50. Krumholz HM, Merrill AR, Schone EM, Schreiner GC, Chen J, Bradley EH, Wang Y, Wang Y, Lin Z, Straube BM, Rapp MT, Normand SL, Drye EE. Patterns of hospital performance in acute myocardial infarction and heart failure 30‐day mortality and readmission. Circ Cardiovasc Qual Outcomes. 2009;2:407–413. [DOI] [PubMed] [Google Scholar]
  • 51. Zhang W, Watanabe‐Galloway S. Ten‐year secular trends for congestive heart failure hospitalizations: an analysis of regional differences in the United States. Congest Heart Fail. 2008;14:266–271. [DOI] [PubMed] [Google Scholar]
  • 52. Arnold SV, O'Brien SM, Vemulapalli S, Cohen DJ, Stebbins A, Brennan JM, Shahian DM, Grover FL, Holmes DR, Thourani VH, Peterson ED, Edwards FH; STS/ACC TVT Registry . Inclusion of functional status measures in the risk adjustment of 30‐day mortality after transcatheter aortic valve replacement: a report from the Society of Thoracic Surgeons/American College of Cardiology TVT Registry. JACC Cardiovasc Interv. 2018;11:581–589. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 53. Christiansen CL, Morris CN. Improving the statistical approach to health care provider profiling. Ann Intern Med. 1997;127:764–768. [DOI] [PubMed] [Google Scholar]
  • 54. Yale New Haven Health Services Corporation/Center for Outcomes Research and Evaluation (YNHHSC/CORE) . 2017 Condition‐specific measures updates and specifications report hospital‐level 30‐day risk‐standardized readmission measures. 2017. https://medicine.yale.edu/core/current_projects/quality_measurement/. Access November 12, 2018.

Associated Data

This section collects any data citations, data availability statements, or supplementary materials included in this article.

Supplementary Materials

Table S1. Causes of Readmission

Table S2. Final Hierarchical Model for Clinically Relevant Variables; 30‐Day All Cause Readmission Following TAVR

Table S3. Final Hierarchical Model for Backward Selected Variables: 30‐Day All Cause Readmission Following TAVR

Table S4. Fine Gray Model, 30‐Day All Cause Readmission Rate for Hospital

Table S5. Final Hierarchical Model for Clinically Relevant Variables; 1‐Year All Cause Readmission Following TAVR

Table S6. Final Hierarchical Model for Backward Selected Variables: 1‐Year All Cause Readmission Following TAVR

Table S7. Fine Gray Model, 1‐Year All Cause Readmission Rate for Hospital

Table S8. Models Discrimination Parameters

Table S9. Observed and Expected 30‐Day Readmission—Fine Gray Model

Table S10. Observed and Expected 1‐Year Readmission‐Fine Gray Model

Figure S1. Calibration models.


Articles from Journal of the American Heart Association: Cardiovascular and Cerebrovascular Disease are provided here courtesy of Wiley

RESOURCES