Collaborative quality improvement vs public reporting for percutaneous coronary intervention: A comparison of percutaneous coronary intervention in New York vs Michigan

Abstract

Introduction

Public reporting (PR) is a policy mechanism that may improve clinical outcomes for percutaneous coronary intervention (PCI). However, prior studies have shown that PR may have an adverse impact on patient selection. It is unclear whether alternatives to PR, such as collaborative quality improvement (CQI), may drive improvements in quality of care and outcomes for patients receiving PCI without the unintended consequences seen with PR.

Methods

Using National Cardiovascular Data Registry CathPCI Registry data from January 2011 through September 2012, we evaluated patients who underwent PCI in New York (NY), a state with PR (N = 51,983), to Michigan, a state with CQI (N = 53,528). We compared patient characteristics, the quality of care delivered, and clinical outcomes.

Results

Patients undergoing PCI in NY had a lower-risk profile, with a lower proportion of patients with ST-segment elevation myocardial infarction, non–ST-segment elevation myocardial infarction, or cardiogenic shock, compared with Michigan. Quality of care was broadly similar in the 2 states; however, outcomes were better in NY. In a propensity-matched analysis, patients in NY were less likely to be referred for emergent, urgent, or salvage coronary artery bypass surgery (odds ratio [OR] 0.67, 95% CI 0.51–0.88, P < .0001) and to receive blood transfusion (OR 0.7, 95% CI 0.61–0.82, P < .0001), and had lower in-hospital mortality (OR 0.72, 95% CI 0.63–0.83, P < .0001).

Conclusions

Public reporting of PCI data is associated with fewer high-risk patients undergoing PCI compared with CQI. However, in comparable samples of patients, PR is also associated with a lower risk of mortality and adverse events. The optimal quality improvement method may involve combining these 2 strategies to protect access to care while still driving improvements in patient outcomes.

Public reporting (PR) of mortality rates was first introduced in the late 1980s as a means to improve quality of care by incentivizing hospitals and physicians to “compete” against each other to achieve low mortality rates.^1–3 Public reporting may also improve quality by allowing informed decision making when patients choose a physician or health system. Early analyses examining the effect of PR demonstrated reduced mortality for both coronary artery bypass grafting (CABG) and percutaneous coronary intervention (PCI).^4–9 However, PR may also have unintended consequences. Prior studies have shown that high-risk patients are less likely to undergo revascularization in states with PR of outcomes, thus denying potentially lifesaving therapy to patients who may benefit from it the most.^10–13 As a result, there is a growing need to identify alternative strategies to improve access to care and patient outcomes after PCI.

Quality improvement (QI) systems that do not publicly report data may result in similar improvement in adherence to quality performance metrics.^14,15 One such QI system, termed collaborative quality improvement (CQI), provides cross-institutional, peer-reviewed analysis and promotes accountability through sharing of information to institutions and providers.^14,16,17 This information is shared among individual practitioners within the health systems participating in CQI, but does not include practitioner-level PR.

However, little is known regarding how PR and CQI may compare in terms of their effects on practice and outcome. Therefore, we set out to compare patient selection, quality of care, and patient outcomes in 2 US states with very different approaches to the use and publication of quality data: New York (NY), a pioneer in PR, vs Michigan, a leader in CQI implementation.

Methods

Data source

The analytic cohorts for this study were derived from the National Cardiovascular Data Registry (NCDR) CathPCI Registry. Details of the NCDR participants and data collection methods have been previously described.^18–21 The NCDR is an initiative of the American College of Cardiology Foundation and the Society for Cardiovascular Angiography and Interventions. Hospitals participating in the CathPCI Registry provide patient, procedure, and outcome data on all PCI cases performed in their facilities.²² All index PCIs performed at NCDR reporting centers in NY (N = 51,983) and Michigan (N = 53,528) between January 2011 and September 2012 were included in this analysis. All hospitals in Michigan participate in NCDR; however, only 43 of 59 nonfederal hospitals in NY participate in the registry.²³ Federal hospitals in NY do not participate in NCDR. Index PCI includes the initial PCI performed on a patient during their hospitalization.

Data elements collected in the registry include demographic characteristics (age, gender, race, and insurance status), cardiovascular risk factors (hypertension, dyslipidemia, family history of premature coronary artery disease, diabetes mellitus, end-stage renal disease), cardiovascular disease history (prior myocardial infarction, congestive heart failure [CHF], prior PCI, cerebrovascular disease, peripheral vascular disease), and clinical presentation (asymptomatic, atypical chest pain, stable angina, unstable angina, non–ST-segment elevation myocardial infarction [NSTEMI], or ST-segment elevation myocardial infarction [STEMI]). Procedure-related information includes indication (acute coronary syndrome, evaluation of cardiomyopathy, preoperative evaluation for noncardiac surgery, or cardiogenic shock within 24 hours prior to procedure), and presence and location (native coronary arteries vs bypass grafts) of coronary stenosis of ≥50%.

Outcomes

The primary predictor in this study was state. All analyses compare patients in NY with those in Michigan. Our primary outcomes had 3 components: patient mix (proportion of patients with NSTEMI, STEMI, and cardiogenic shock), quality of care (PCI appropriateness, periprocedural assessment, referral to cardiac rehabilitation, and discharge on optimal medical therapy), and outcomes (contrast-induced nephropathy, renal failure, need for urgent, emergent or salvage CABG, cardiogenic shock/CHF/cerebral vascular accident/tamponade, vascular complications including bleeding within 72 hours of PCI, access site bleeding, access site hematoma, retroperitoneal bleeding, gastrointestinal bleeding, need for blood transfusion, and in-hospital mortality).

Statistical analysis

We performed a baseline, unadjusted analysis to assess for differences in patient characteristics (including demographics, medical history, risk factors, presenting diagnosis, and baseline risk of mortality), procedural characteristics (including diagnostic catheterization procedure, estimate of coronary anatomy, PCI procedure, type of lesions, and devices), quality of care, and outcomes. Categorical variables were presented as frequencies (percentages), and differences between the CQI and PR states were assessed using the χ² test when the sample size was sufficient; otherwise, an exact test was used. Continuous variables were presented as median and were compared using the Wilcoxon rank sum test. Baseline risk was estimated using logistic regression with generalized estimating equations to account for within-hospital clustering.

In order to account for the baseline differences between the 2 patient populations, a propensity-matched analysis using the gmatch macro was performed.²⁴ The propensity-matched analysis was adjusted for all precatheterization variables in the NCDR CathPCI mortality model, version 4, as well as the prespecified outcomes measured and matched on the logit of the propensity score to undergo PCI. We used a caliper with a width of 0.2 times the standard deviation of the logit of the propensity score, which has been shown to result in estimates of the treatment effect with lower mean squared error.²⁵ We assessed for balance of the covariates between the 2 groups using methods previously described,^26,27 and then assessed PCI outcomes, performance measures, and appropriateness within this cohort. Percutaneous coronary intervention appropriateness was evaluated based on the American College of Cardiology Foundation, American Heart Association, and Society of Cardiovascular Angiography and Interventions appropriateness criteria.^28,29

To estimate the effect of our primary predictor (state) on quality of care and clinical outcomes among the propensity-matched cohorts, we developed a logistic regression model stratified by matched pair. Matched pairs of patients had similar propensity scores and were more likely to have similar outcomes. This method is a generalization of McNemar test for matched pairs which is expected to reduce most of the observed differences in patient case mix between the 2 groups. We used Bonferroni correction to account for multiple comparisons³⁰ and assessed for the presence of unmeasured confounding and its impact on mortality by performing a sensitivity analysis.³¹ The Duke Clinical Research Institute performed all statistical analyses using SAS software (version 9.3; SAS Institute, Cary, NC). A P value b.05 was considered significant.

No extramural funding was used to support this work. The authors are solely responsible for the design and conduct of this study, all study analyses, the drafting and editing of the manuscript, and its final contents.

Results

Patient characteristics and risk profile

Baseline characteristics including patient demographics, comorbidities, and procedure indications are listed in Table I. There was no difference in age among the 2 cohorts; however, there were significant baseline differences with regard to patient and procedure characteristics. Patients in NY were less likely to be female or white. They were also less likely to have a history of myocardial infarction, CHF, hypertension, dyslipidemia, cerebrovascular disease, peripheral vascular disease, and chronic lung disease. Procedural characteristics are presented in Table II. New York patients were less likely to undergo PCI for STEMI or NSTEMI and had lower rates of cardiogenic shock and cardiac arrest at the time of PCI. The baseline differences resulted in a significantly lower percentage of patients with extremely high (>20%) predicted risk of mortality in NY compared with Michigan (Figure 1). Following matching based on the propensity to undergo PCI in either state, the patient and procedure-related variables were well balanced (online Appendix Supplementary Figures 1 and 2, and Tables I and II).

Table I.

Baseline, unadjusted characteristics for patient’s undergoing PCI: NY (PR) vs Michigan (CQI)

	NY (PR; N = 51,983)	Michigan (CQI; N = 53,528)	P
Age (y)	64.8 ± 11.8	64.9 ± 12.1	0.50
Female	29.8	33.7	<0.0001
White	79.0	86.2	<0.0001
Medicaid	18.0	10.5	<0.0001
No insurance	3.9	5.4	<0.0001
BMI (kg/m2)	29.7 ± 6.2	30.6 ± 6.6	<0.0001
Previous MI	28.9	35.8	<0.0001
Previous CHF	10.3	15.9	<0.0001
Previous PCI	43.2	45.6	<0.0001
Previous CABG	15.8	18.8	<0.0001
Diabetes mellitus	39.2	37.9	<0.0001
Insulin-dependent diabetes mellitus	13.5	15.9	<0.0001
Cerebrovascular disease	10.5	15.4	<0.0001
Peripheral vascular disease	9.8	16.4	<0.0001
Hypertension	84.4	85.2	<0.0001
Current smoker	23.2	29.5	<0.0001
Dyslipidemia	81.5	82.7	<0.0001
Chronic lung disease	9.6	18.5	<0.0001
Presentation
Asymptomatic	6.10	5.33
Atypical chest pain	1.04	2.41
Stable angina	19.65	14.90	<0.0001
Unstable angina	43.60	41.23
NSTEMI	16.07	20.51
STEMI	13.49	15.59
Thrombolytics (if STEMI) before PCI	9.7	6.5	<0.0001
Cardiomyopathy	8.5	11.0	<0.0001
Cardiogenic shock within 24 h of PCI	1.6	2.4	<0.0001
Cardiac arrest within 24 h of PCI	1.2	1.8	<0.0001
Intra-aortic balloon pump	2.3	2.6	0.003

Abbreviations: BMI, Body mass index; MI, myocardial infarction.

All variables expressed as percent, except age and BMI which are expressed as mean.

Table II.

Baseline, unadjusted PCI characteristics: NY (PR) vs Michigan (CQI)

	NY (PR; N = 51,983)	Michigan (CQI; N = 53,528)	P
PCI appropriate	81.5	82.7	<0.0001
PCI status
Elective	48.6	40.4
Urgent	37.3	43.1	<0.0001
Emergent	14.0	16.3
Salvage	0.1	0.2
PCI indication
Immediate STEMI	11.2	13.9
Unstable STEMI (>12 h after onset)	1.0	0.7
Stable STEMI (>12 h after onset)	0.3	0.3
Stable STEMI	0.5	0.3	<0.0001
after thrombolysis
STEMI after	0.6	0.6
failed thrombolysis
High-risk NSTEMI/USA	47.9	52.2
Staged PCI	6.5	4.9
Other	31.8	27.1
Lesion
Left main	1.7	2.1
Proximal LAD	14.9	15.6	<0.0001
Mid-LAD or Proximal	34.8	34.5
RCA/Cx
Other	47.3	46.8
Lesion risk
Non–high risk	49.0	46.1	<0.0001
High risk	50.4	53.5
Lesion length (mm)	19.4 ± 10.9	22.5 ± 12.7	<0.0001
Thrombus present	13.9	15.2	<0.0001
Bifurcation lesion	11.2	8.7	<0.0001
Device deployed	97.4	97.7	<0.0001
Dissection	0.7	1.3	<0.0001
Perforation	0.4	0.4	0.3

Abbreviations: USA, Unstable angina; LAD, left anterior descending; RCA, right coronary artery; Cx, circumflex.

All variables expressed as percent, except lesion length which is expressed as mean.

Figure 1.

Distribution of patients with predicted risk of in-hospital mortality greater than 20% among patients undergoing PCI in NY (PR) and Michigan (CQI).

Quality of care delivered

Measures of PCI quality (Table III) varied significantly between the 2 states. Specifically, those in NY were more likely to undergo appropriate PCI; however, there was an increased number of PCI of uncertain appropriateness in NY with no difference in inappropriate PCI between the 2 states. In NY, patients were more likely to have markers of myonecrosis assessed but less likely to undergo pre-PCI renal function assessment. At the time of discharge, the NY cohort was much less likely to refer patients to cardiac rehabilitation, but there was no difference between the 2 states with regard to discharging patients on optimal medical therapy.

Table III.

PCI quality measures using Bonferroni correction based on the propensity to undergo PCI

Outcome	OR	Corrected LCL	Corrected UCL	Corrected P value
Appropriate PCI	1.10	1.04	1.17	<.0001
Inappropriate PCI	1.04	0.91	1.20	1.0000
Uncertain appropriateness of PCI	1.24	1.14	1.35	<.0001
Pre–renal function testing	0.27	0.24	0.31	<.0001
Post–renal function testing	1.55	1.41	1.71	<.0001
Pre–renal function testing and post–renal function testing	0.86	0.80	0.93	<.0001
Markers of myonecrosis	1.60	1.44	1.79	<.0001
Cardiac rehabilitation referral	0.15	0.14	0.16	<.0001
Post-PCI length of stay	1.14	1.07	1.21	<.0001
Optimal medical therapy	1.10	0.90	1.36	1.0000
Transfer to another facility post-PCI	0.88	0.74	1.05	.8277

OR > 1 suggests an increased likelihood of the outcome in NY (PR).

Abbreviations: LCL, lower confidence limit; UCL, upper confidence limit.

Clinical outcomes

Adverse events were generally lower in the NY cohort (Table IV). Patients in NY were less likely to have an access site complication, bleeding, or need a blood transfusion following PCI, and had a lower incidence of contrast-induced nephropathy. In addition, the NY cohort was less likely to refer a patient for an emergent, urgent, or salvage CABG. There was a lower likelihood of mortality associated with PCI performed in NY compared with Michigan which persisted after propensity matching (344 [0.84%] vs 478 [1.17%]; odds ratio [OR] 0.72, 95% CI 0.63–0.83) (Figure 2), including fewer deaths in NY the day of the procedure (Table IV).

Table IV.

Clinical outcomes among propensity-matched cohorts using Bonferroni correction

Outcome	OR	Corrected LCL	Corrected UCL	Corrected P value
Contrast-induced nephropathy	0.91	0.80	1.05	1.0000
Renal failure	0.68	0.43	1.08	.2622
CABG (urgent, emergent, or salvage)	0.67	0.51	0.89	.0002
Cardiogenic shock	0.33	0.26	0.42	<.0001
CHF	0.34	0.28	0.42	<.0001
Cerebrovascular accident	0.95	0.59	1.53	1.0000
Tamponade	1.42	0.67	3.00	1.0000
Any vascular complication	0.66	0.46	0.95	.0092
Bleeding event ≤ 72 h of PCI	0.62	0.52	0.74	<.0001
Bleeding at access site	0.69	0.48	0.98	.0287
Hematoma at access site	0.67	0.51	0.89	.0003
Retroperitoneal bleeding	0.72	0.42	1.23	1.0000
Gastrointestinal bleeding	0.42	0.27	0.66	<.0001
Blood transfusion	0.70	0.61	0.82	<.0001
Death day of procedure	0.60	0.37	0.96	.0175
Death in catheterization laboratory	0.69	0.37	1.31	1.0000

OR > 1 suggests an increased likelihood of the outcome in NY (PR).

Abbreviation: MI, Myocardial infarction; LCL, lower confidence limit; UCL, upper confidence limit.

Figure 2.

Unadjusted and propensity-matched in-hospital mortality in NY (PR) vs Michigan (CQI).

Sensitivity analyses

A sensitivity analysis was performed to assess for the effect of unmeasured confounding and its impact on mortality. The lower likelihood of mortality observed for patients undergoing PCI in NY was nullified when the estimated prevalence of the hypothetical unmeasured confounder was ≥40% in the Michigan cohort and ≤10% in the NY cohort. Mortality was more likely in patients undergoing PCI in NY once the estimated prevalence of the hypothetical unmeasured confounder in Michigan patients was ≥80% and the prevalence of the unmeasured confounder in NY was ≤10% (online Supplementary material). As neither of these scenarios is clinically realistic, the results were considered to be robust to the presence of an unmeasured confounder.

Discussion

We examined 2 states with alternate QI systems and found substantial differences in patient and procedure-related characteristics, baseline risk distribution, quality measures, and PCI-related outcomes. Patients in NY, a state with PR, demonstrated a significantly lesser burden of comorbidities and high-risk features such as STEMI, NSTEMI, cardiac arrest, and cardiogenic shock at the time of PCI, suggesting a degree of risk aversion in patient selection in NY compared with MI. On the other hand, patients in NY received similar or higher-quality care, and had fewer adverse events and lower mortality even after adjusting for these clinical differences, suggesting that the PR strategy may be more powerful than the CQI strategy in improving hard clinical outcomes.

Our observations confirm prior studies showing that extremely high-risk patients are less likely to undergo PCI in states with PR, which may be related to risk avoidance.^10–12,16 This phenomenon may also explain the lower use of urgent, emergent, or salvage CABG in this population, as CABG outcomes are also publicly reported in NY, and a high expected complication rate may influence a surgeon’s decision to offer the procedure. There are a growing number of potential ways in which these unintended consequences of PR may be mitigated. For example, one strategy is the introduction of compassionate use criteria, as is used in the Massachusetts PCI reporting program. As described by Resnic and Welt,¹² compassionate use PCI impacts both the predicted and observed mortality particularly in cases which may be considered futile. Expanding PR may be another promising strategy. Including procedure-related processes and outcomes, such as access to PCI and adverse clinical events, may help highlight other important related performance measures which may be neglected by too narrow a focus on mortality alone.

However, we also demonstrated a significantly lower mortality rate in NY compared with Michigan even after matching based on the propensity to undergo PCI in either state. Because patients were matched on a wide array of variables including disease severity, presentation, and lesion type, this suggests that the difference in mortality cannot easily be attributed to case mix alone. In addition, similar or better quality of care and fewer adverse events were demonstrated in NY, suggesting that PR may have a meaningful impact on clinical care and outcomes. This suggests that despite its potential unintended consequences, PR may be a strategy that still warrants consideration for improving outcomes for PCI and other procedures. Further study is warranted to understand what specific changes cardiac catheterization laboratory directors and other clinical leaders made in response to PR, to determine whether these positive strategies can be generalized to a wider group of hospitals.

Our findings have several important implications. As the American College of Cardiology moves toward publically reporting mortality statistics for PCI, and as states and the federal government increase the use of PR for more procedures and conditions, it will be important to closely follow use patterns to ensure that this and other reported procedures are being appropriately offered and performed. It is unlikely in this era of increasing transparency that PR will cease; instead, we should focus on leveraging its upsides and limiting its downsides. Having both clinical leaders and policymakers engaged in discussions around this topic is thus critical.

Our study should be interpreted with certain key caveats. This analysis is an observational study that carries the inherent risk of unmeasured confounding that cannot be fully mitigated. We attempted to mitigate unmeasured confounding through propensity matching, which included demographic, socioeconomic, health severity, and procedure-related variables. In addition, when testing for unmeasured confounding through our sensitivity analysis, our findings appeared robust to a moderate level of confounding. Selection bias may also affect the observed benefits associated with PR in our analysis because not all hospitals and health systems in NY provide data to the NCDR while every hospital in Michigan does. It is possible that NY hospitals that participate in NCDR self-select; therefore, they may represent institutions that have more resources or provide better care compared with those who do not participate. In addition, all hospitals in Michigan undergo a rigorous audit of their data on an ongoing basis in addition to the more selective audit used by NCDR. The audit process used by the CQI system in the state of Michigan may lead to more accurate identification of adverse events and clinical outcomes than reliance on self-reporting alone, thus biasing the results. Because the data submitted by participating institutions are not linked to primary clinical documentation, we are limited by the accuracy of the data reported to the CathPCI Registry. This makes the data susceptible to ascertainment bias for outcomes as well as PCI appropriateness. Finally, although our analysis alludes to the possibility of risk aversion in patients with the highest likelihood of mortality, this is not a variable included in the analysis and its role in patient selection and clinical outcome can only be extrapolated from the data.

Conclusion

Public reporting of PCI data is associated with fewer high-risk patients undergoing PCI compared with CQI. However, in comparable samples of patients, PR is also associated with a lower risk of mortality and adverse events. The optimal QI method may involve combining these 2 strategies to protect access to care while still driving improvements in patient outcomes.