Improvement in Mortality Risk Prediction Following Percutaneous Coronary Intervention Through Addition of a “Compassionate Use” Variable to the National Cardiovascular Data Registry CathPCI® Dataset: A Study from the Massachusetts Angioplasty Registry

Abstract

Objectives

This study investigated the impact of adding novel elements to models predicting in-hospital mortality following percutaneous coronary interventions (PCIs).

Background

Massachusetts (MA) mandated public reporting of hospital-specific PCI mortality in 2003. In 2006, a physician advisory group recommended adding to the prediction models three attributes not collected by the National Cardiovascular Data Registry instrument. These “compassionate use” (CU) features included coma on presentation, active hemodynamic support during PCI, and cardiopulmonary resuscitation at PCI initiation.

Methods

From October 2005 through September 2007, PCI was performed during 29,784 admissions in MA non-federal hospitals. Of these, 5,588 involved patients with ST segment elevation myocardial infarction or cardiogenic shock. Cases with CU criteria identified were adjudicated by trained physician reviewers. Regression models with and without the CU composite variable (presence of any of the 3 features) were compared using areas under the receiver operator characteristic curves (AUC).

Results

Unadjusted mortality in this high-risk subset was 5.7%. Among these admissions, 96 (1.7%) had at least one CU feature, with 69.8% mortality. The adjusted odds ratio for in-hospital death for CU PCIs (vs. no CU criteria) was 27.3 (95% CI 14.5–47.6). Discrimination of the model improved after including CU, with AUC increasing from 0.87 to 0.90 (p<0.01), while goodness of fit was preserved.

Conclusions

A small proportion of patients at extreme risk for post-PCI mortality can be identified using pre-procedural factors not routinely collected, but that heighten predictive accuracy. Such improvements in model performance may result in greater confidence in reporting of risk-adjusted PCI outcomes.

Background

Beginning in 2003, Massachusetts (MA) followed New York, becoming the second state in the nation to require public release of risk-adjusted mortality statistics following percutaneous coronary interventional (PCI) procedures. The program was established to monitor the quality of invasive cardiac procedures performed within the state and to provide rigorous statistical assessments of any differences in the risk-adjusted mortality among hospitals and individual PCI operators. The Massachusetts program utilizes Bayesian hierarchical logistic regression models to evaluate the quality of cardiac care as assessed by the in-hospital, risk-adjusted, all-cause mortality following PCI(1). The hierarchical model methodology has been shown to be more specific in the detection of performance outliers than non-hierarchical methods(2,3).

The public release of risk-adjusted mortality reports is controversial(4–8) and can have significant impact on the hospitals and operators through adverse publicity associated with being identified as a negative outlier in quality performance(8). From the outset of the PCI reporting effort, the Massachusetts program actively engaged interventional cardiologists in the case review and adjudication process for all patients with high risk clinical features. In addition, clinicians were directly involved in the process of developing the risk-prediction models, based on the covariates available in the National Cardiovascular Data Registry (NCDR®) CathPCI Registry® (version 3.04) in an effort to establish the maximum confidence by the clinical community in the risk adjustment methodology. Based on physician input, Massachusetts divided the population of patients undergoing PCI into two groups: those presenting with ST segment elevation myocardial infarction (STEMI) or cardiogenic shock (the shock or STEMI, or “SOS” group), and all other, typically less acute PCI admissions (non-SOS group).

While the NCDR CathPCI Registry dataset is broadly used as a quality monitoring tool throughout the United States, it had not been originally intended for nor designed to be applied to mandatory public reporting as utilized in Massachusetts. Recognizing limitations in the breadth of the variables collected in the existing CathPCI Registry dataset, interventional cardiologists in Massachusetts became concerned that the risk adjustment models were limited in their ability to account fully for the clinical acuity of the most critically ill patients. There was a specific concern that clinicians could become increasingly risk-averse due to fears that the risk model methodology would not adequately adjust for the highest-risk patient populations being treated. The concerns centered on the possibility that uncaptured (and thus unmodeled) covariates, such as impaired neurologic status on presentation, might be independently predictive of in-hospital mortality. Exclusion of such potentially strong, independent predictors of mortality that could vary in prevalence across hospitals would reduce the clinical credibility of the models and might lead to inaccurate prediction of post-PCI mortality.

The concerns raised by Massachusetts clinicians was based, in part, on the observation that there had been a gradual decline in the proportion of patients treated with PCI for cardiogenic shock in New York in the years since that state began publishing reports of clinician-specific risk adjusted outcomes(4). Supporting this view, several surveys of New York State PCI operators and cardiac surgeons indicated an increasing unwillingness to treat the most severely ill patients after the introduction of public reporting of mortality outcomes; despite validated risk adjustment models(7). Moreover, it had been observed that New York interventionalists and cardiac surgeons were unique among geographic locales in providing lower rates of revascularization for patients included in a national cardiogenic shock registry(9), despite mounting evidence of the clinical benefits of early revascularization of most patients presenting in cardiogenic shock(10). In 2008, New York State began excluding patients presenting in cardiogenic shock from the publicly released risk adjusted mortaility reports. In light of these findings, clinical advisors to the Massachusetts public reporting program recommended inclusion of additional high-risk clinical factors not routinely collected in the CathPCI Registry dataset in an effort to improve the performance of the risk adjustment models as well as bolster clinician acceptance of the risk adjustment process.

We sought to examine the feasibility of collecting a set of novel factors likely predictive of post-PCI demise and their impact in the Massachusetts risk adjusted models of in-hospital mortality.

Methods

Patient and Hospital Populations

We studied all patients aged 18 years or older presenting with cardiogenic shock or ST segment elevation myocardial infarction and who underwent PCI at any of the Massachusetts, non-federal, acute care hospitals between October 1, 2005 through September 30, 2007. Because it was possible for patients to undergo more than one PCI during a single admission, we selected the first PCI as the unit of observation in accordance with NCDR risk adjustment methods and to reflect the clinical status of the patient on presentation. Patients were assigned to the hospital where the initial PCI was performed, even if transferred elsewhere for subsequent procedures.

Data Sources

We used clinical and billing data submitted to the Massachusetts Data Analysis Center (Mass-DAC), the data coordinating center located in Harvard Medical School under contract with the Department of Public Health of the Commonwealth of Massachusetts to collect, clean, and analyze the PCI data. Three distinct data sources were used to assemble information. Detailed clinical information obtained during the hospital admission was collected prospectively by trained data managers using variables defined in the NCDR CathPCI Registry dataset(11) and submitted electronically every 3 months. Massachusetts hospitals were also mandated by regulation to supplement the NCDR data with additional elements incorporating patient identification information and details regarding the identity of hospitals from which and to which patients may have been transferred. To ensure completeness of the admissions submitted to Mass-DAC, inpatient discharge billing data available from the Division of Health Care Finance and Policy in the Massachusetts Department of Public Health were linked to the Mass-DAC data using patient and hospital identifiers. The billing data included patient demographic information, as well as diagnoses and procedures performed during the hospitalization. Finally, while data managers report patient vital status at discharge, Mass-DAC confirmed this information by linking to the Massachusetts Registry of Vital Records and Statistics.

Compassionate Use

Beginning October 1, 2005, Mass-DAC added three unique covariates to those available in the NCDR CathPCI dataset and collected this information electronically at regular intervals. The new covariates were termed “compassionate use” (CU) criteria and were intended to identify extremely high risk clinical scenarios not currently characterized by the existing NCDR data instrument. The CU criteria included: coma on presentation for emergent PCI, use of a percutaneous ventricular assist device or extracorporeal bypass, and cardiopulmonary resuscitation (CPR) at the initiation of the procedure. Of note, both coma on presentation and CPR at initiation of procedure required that the case also be coded as an emergent or salvage procedure, while a planned or emergent ventricular assist device-supported procedure would also qualify as CU. Table 1 summarizes the definitions of the additional compassionate use covariates.

Table 1.

Definition of Compassionate Use Criteria

Criteria	Definition	Notes
Coma on presentation	The patient presents to the Emergency Room or the Cardiac Catheterization Laboratory with a Glasgow Coma Score <7 and is coded as emergent status	The coma must not be medication induced.
Use of ventricular assist device	The medical record must indicate the use of cardiopulmonary bypass (CPB), extra-corporeal membrane oxygenation (ECMO), or percutaneous ventricular assist device (PVAD) prior to the start of the PCI	Use of ventricular support must be justified in the medical record. Use of CPB/ECMO or PVAD to rescue a diagnostic case complication would not be a criteria for CU.
Cardiopulmonary resuscitation (CPR) at start of procedure	The medical record must reflect that spontaneous circulation was not restored prior to the start of the PCI and, therefore, the patient was receiving active CPR at the start of the PCI	Excludes patients successfully resuscitated in the field without the need for ongoing CPR. Utilizing CPR to rescue a diagnostic case complication would not be a criteria for “compassionate use.”

All CU cases were independently reviewed, adjudicated, and confirmed by physician reviewers trained in the definitions of items in the NCDR data instrument (e.g., cardiogenic shock, PCI status as emergent or salvage) as well as the CU criteria. All reviewers underwent training in research on human subjects and were approved by the Harvard Medical School’s Institutional Review Board. The CU case adjudication process scrutinized every case coded as CU using supplemental clinical documentation forwarded to Mass-DAC. This documentation included emergency department and emergency medical services records (when appropriate), cardiac catheterization laboratory reports, and clinical documentation for the inpatient hospital course. If a physician reviewer was uncertain about a particular adjudication, the case was discussed with the other members of the review committee, and a decision was reached by consensus. An appeal process was established so that medical centers could challenge the adjudication of any particular case which was originally ruled to not meet the CU criteria. All appeals required additional clinical documentation explicitly justifying the use of CU designation. Appeal adjudications were made by board certified interventional cardiologists with extensive experience in the adjudication process.

Primary Outcome

The primary patient outcome was mortality, regardless of cause, during the hospitalization (or outpatient procedure, if applicable) in which the PCI was performed. Secondary outcomes included survival at 30 days and 12 months as assessed by linked data analysis with Massachusetts death index, and assessment of impact of inclusion of compassionate use adjustment on hospital quality classification.

Statistical Analysis

Univariate analyses were performed using Chi-squared tests or Fisher Exact tests were implemented using SAS 9 (SAS Institute, Cary, NC)(12). Hierarchical logistic-normal regression models that included random hospital-specific intercepts were used to account for within-hospital clustering and estimated using the WinBugs application (13). The CU covariate was operationalized as a binary composite variable that assumed a value of 1 if any of the specified conditions was observed to be true; otherwise it assumed a value of 0. In-hospital mortality prediction models were developed using hierarchical regression models, including and excluding the compassionate use covariate.

The discriminating ability of the model was assessed by the area under the receiver operator characteristic (AUC) curve. We calculated the difference in the AUC between a model with and without the CU variables using a paired test(14). Because we were also interested in how the addition of the variables would impact risk prediction, we undertook a reclassification analysis to assess the proportion of patients whose predicted mortality changed by at least one quintile of risk(15). This was accomplished by calculating risk quintile for each patient using both the predictive model excluding and including the compassionate risk covariate. The Net Reclassification Improvement (NRI) measure that reflects the overall gain in prediction due to the inclusion of the compassionate use covariate was also calculated (16).

Results

Between October 1, 2005 and September 30, 2007, at least one PCI occurred in a total of 29,784 admissions in Massachusetts licensed (non-Federal) hospitals. Among these admissions, 5,588 patients (18.8%) underwent a PCI after presenting with shock or ST segment elevation myocardial infarction (SOS cases). Within the SOS cohort of patients, a total of 96 patients (1.7% of SOS cases) were adjudicated as having qualified for compassionate use.

Compassionate use admissions were significantly more likely to have presented in cardiogenic shock (Table 2: 65.6% vs. 8.2%, p<0.001), more likely to have received an intra-aortic balloon pump (49.0% vs. 12.5%, p<0.001), and more likely to have pre-existing renal insufficiency on presentation (15.6% vs. 4.1%, p<0.001) as compared with SOS patients without compassionate use clinical features.

Table 2.

Patient presenting factors in the cardiogenic shock or ST segment myocardial infarction PCI cohort, overall and stratified by compassionate use.

Characteristic	Compassionate Use		All (%)	p-value§
	No	Yes
Number of Admissions	5492	96	5588
Female	1587	32	1619 (29)	0.364
Male	3905	64	3969 (71)
Age < 60 years	2813	37	2850 (51)	0.11
61 – 70 years	1127	25	1152 (21)
71 – 80 years	976	22	998 (18)
> 80 years	576	12	588 (11)
Race: White	4816	77	4893 (88)	0.10
African American	140	3	143 (2.6)
Hispanic	161	5	166 (3.0)
Other	346	11	357 (6.4)
Previous MI	937	24	961 (17)	0.055
No Previous MI	4555	72	4627 (83)
Diabetes mellitus	1097	33	1130 (20)	0.0012
No Diabetes mellitus	4395	63	4458 (80)
History of Renal Failure	226	15	241 (4)	0.00001
No History of Renal Failure	5266	81	5347 (96)
Previous MI	937	24	961 (17)	0.055
No Previous MI	4555	72	4627 (83)
History of CHF	311	14	325 (5.8)	0.0012
No History of CHF	5181	82	5263 (94)
Cerebrovascular Disease	374	17	391 (7)	0.0003
No Cerebrovascular Disease	5118	79	5197 (93)
Peripheral Vascular Disease	482	17	499 (9)	0.0057
No Peripheral Vascular Disease	5010	79	5089 (91)
Chronic Lung Disease	586	16	602 (11)	0.0672
No Chronic Lung Disease	4906	80	4986 (89)
Hypertension	3354	58	3412 (61)	0.9161
No Hypertension	2138	38	2176 (39)
Current Smoker	2059	33	2092 (37)	0.5952
Not a Current Smoker	3433	63	3496 (63)
Dyslipidemia	3478	48	3526 (63)	0.0101
No Dyslipidemia	2013	48	2061 (37)
Previous PCI	813	9	822 (15)	0.1479
No Previous PCI	4679	87	4766 (850
Previous CABG	248	10	258 (5)	0.0126
No Previous CABG	5244	86	5330 (95)
Current CHF	834	48	882 (16)	< 0.00001
No Current CHF	4658	48	4706 (84)
Renal Failure/Dialysis	39	4	43 (1)	0.0060
No Renal Failure/Dialysis	5453)	92	5545 (99)
Left Ventricular Ejection Fraction (LVEF)				< 0.00001
Not Measured	2858	53	2911 (52)
LVEF < 30%	433	25	458 (8.2)
31% < LVEF ≤ 40%	510	8	518 (9.3)
LVEF > 40%	1691	10	1701 (30)
Cardiogenic Shock	394	63	457 (8)	< 0.00001
No Cardiogenic Shock	5098	33	5131 (92)
STEMI	5175	80	5255 (94)	0.00015
No STEMI	317	16	333 (6)
Status of PCI
Elective/Urgent	332	1	333 (6)	< 0.00001
Emergent	5143	60	5203 (93)
Salvage	17	35	52 (1)
Presence of LMCA lesion	268	76	5300 (95)	< 0.00001
No LMCA lesion	5224	20	288 (5.2)
Intra-Aortic Balloon Pump	652	47	699 (13)	< 0.00001
No Intra-Aortic Balloon Pump	4840	49	4889 (87)
Successful Procedure	5176	76	5252 (94)	< 0.00001
Unsuccessful Procedure	316	20	336 (6)

^§All p-values use Fisher’s exact test with the exception of race, ejection fraction, and PCI status which use χ² tests.

Notes:

MI = myocardial infarction

CHF = congestive heart failure

LMCA = left main coronary artery

In-hospital outcomes for compassionate use cases were dramatically worse than for the high risk SOS cohort of patients without compassionate use clinical features as a whole (Table 3). The likelihood of procedural success was significantly lower for CU cases as compared with non-CU SOS cases (79.2% vs. 94.2%, p<0.001). The CU cases were significantly more likely to suffer new post-procedural cardiogenic shock and more likely to experience bleeding and renal complications following the index procedure than non-CU cases. The unadjusted in-hospital mortality rate was 15.6 times higher for CU cases than for non CU SOS cases (69.8% vs. 4.5%, p<0.001). Although CU cases represented only 1.7% of overall SOS cases, these cases accounted for more than 21% of the overall mortality following PCI in the SOS cohort. Post-procedural death from neurologic causes was significantly more frequent in the compassionate use population as compared with the standard risk SOS cases, likely driven by the concentration of patients presenting with coma in the CU cohort. The observed mortality for CU cases with use of percutaneous ventricular support, rescucitation at the start of the procedure and coma on presentation were 50%, 84% and 70% respectively. Due to the relatively small individual sample sizes, there was no significant difference in the mortality rates between these three categories.

Table 3.

In-hospital complications and death, overall and stratified by compassionate use.

Complication	Compassionate Use		All	p-value§
	No	Yes
Number of Admissions	5492	96	5588
New Cardiogenic Shock	148	6	154 (3)	0.0484
No New Cardiogenic Shock	5344	90	5434 (97)
New Renal Failure	68	7	75 (1)	0.00027
No New Renal Failure	5424	89	5513 (99)
Any Bleeding Complication	417	14	431 (8)	0.0185
No Bleeding Complication	5075	82	5157 (92)
Any Vascular Complication	48	2	50 (1)	0.2118
No Vascular Complication	5444	94	5538 (99)
Any Vascular or Bleeding Complication	447	15	462 (8)	0.0139
No Vascular or Bleeding Complication	5045	81	5126 (92)
Blood Products Used	643	25	668 (12)	0.00017
No Blood Products Used	4849	71	4920 (88)
In-Hospital Survival	5247	29	5276 (94)	<0.00001
In-Hospital Death	245	67	312 (6)
Cause of In-Hospital Death (N = 312)
Cardiac	185	46	231 (74)	0.21
Neurologic	12	15	27 (9)
Renal	3	0	3 (1)
Vascular	1	1	2 (1)
Infection	9	0	9 (3)
Pulmonary	16	3	19 (6)
Other/Unknown	19	2	21 (7)

^§All p-values use Fisher’s exact test with the exception of cause of death which is based on a χ² test.

After adjusting for all other known predictors of in-hospital mortality, compassionate use designation was associated with an odds ratio for in-hospital death of 27.3 (95% CI:14.5–47.6) relative to the non-CU SOS patients. The final multiple logistic regression model included: patient age, pre-procedure renal insufficiency, documented pre-procedure left ventricular ejection fraction <30%, emergent or salvage procedure status, presence of left main coronary artery disease (of severity >50%), presentation with cardiogenic shock and the compassionate use indicator (Table 4).

Table 4.

Adjusted odds ratios of risk of in-hospital all-cause mortality following PCI in the Commonwealth of Massachusetts. Based on 5,588 PCI admissions from October 2005 through September 2007 and 312 deaths.

Risk Factor	Compassionate Use Excluded		Compassionate Use Included
	Adjusted Odds Ratio	95% Posterior Interval	Adjusted Odds Ratio	95% Posterior Interval
AGE: 60 – 70 years	1.47	(0.97, 2.15)	1.66	(1.04, 2.50)
70–80 years	2.48	(1.68, 3.57)	2.94	(1.90, 4.30)
>80 years	5.37	(3.63, 7.65)	6.90	(4.49, 10.1)
Renal insufficiency	3.19	(2.10, 4.64)	3.11	(2.01, 4.58)
Ejection Fraction < 30%	1.74	(1.12, 2.57)	1.64	(1.02, 2.46)
Presence of LMCA lesion	1.94	(1.29, 2.83)	1.73	(1.09, 2.55)
Emergent or Salvage PCI	2.51	(1.23, 4.42)	2.06	(1.09, 3.69)
Cardiogenic Shock	14.0	(10.6, 18.4)	9.91	(7.07, 13.4)
Compassionate Use	Excluded		27.28	(14.5, 47.6)

Among those CU patients who survived to hospital discharge, 83% (24 of 29) were alive at 30 days, while 76% (22 of 29) were alive at one year.

Inclusion of the compassionate use covariates significantly improved the inhospital mortality risk prediction model performance. The discrimination of the hierarchical mortality prediction model significantly improved from an ROC of 0.87 to 0.90 (p<0.001) with preserved goodness of fit. Individual cases were assigned to five risk strata according to predicted risk before and after incorporation of compassionate use criteria. Incorporation of the compassionate use covariate led to the reclassification of the risk by at least one risk strata for 347 SOS cases (6.2%). The NRI was 8.7% (p = 0.43), indicating that 8.7% more patients who died appropriately moved up a category of risk than down when compared with survivors. Most of the impact of inclusion of the new covariate was observed for those who died within the hospitalization (Figure 1).

Figure 1. Impact of inclusion of compassionate use (CU) on predicted mortality.

Impact of inclusion of compassionate use (CU) indicator on predicted mortality for patients in the high risk (cardiogenic shock or STEMI) cohort. The horizontal axis represents the predicted mortality excluding CU, while the vertical axis represents predicted mortality after inclusion of the CU indicator covariate. The diagonal line indicates no change in predicted mortality for individual cases. The cases with CU features had significant increases in their predicted mortality, which further discriminated patients who survived versus those who suffered an in hospital fatality.

In this analysis, overall classification of hospital performance did not change significantly with the inclusion of the CU indicator variable (Figure 2). However, there were measurable changes in the width and range of the estimated posterior intervals which made certain centers “closer” to a change in classification as an outlier institution. For example, the posterior interval for Hospital 1 was shifted to the left after including CU indicating a reclassification of cases which results in improved estimated quality relative to the overall State performance, as compared with the method that excluded the compassionate use indicator. In contrast, the posterior interval estimate for Hospital 12 was shifted to the right after inclusion of the CU indicator, indicating that reclassification of cases resulted in a less favorable estimate of risk adjusted mortality performance. Therefore, Hospital 12 was closer to being identified as an outlier for poorer performance after incorporation of CU indicators, than for the models that did not include CU. However, there were no changes in the posterior interval estimates that led to a change in overall classification by institution as either above, below or within expectations for risk adjusted mortality, though the observed changes in posterior intervals confirms the potential impact o inclusion of CU indicators on assessment of hospital quality.

Figure 2. Hospital risk adjusted mortality with and without compassionate use variable.

Posterior mean risk-standardized mortality rates and corresponding 95% intervals (x-axis) for each MA hospital (y-axis) based on a hierarchical model excluding the compassionate use variable (solid line) and on a hierarchical model including the compassionate use variable (dashed line).

While the impact on physician perception of the risk adjustment methods was not directly measured in this study, a temporal association was apparent after the introduction of the CU indicator in October 2005, with a substantial increase in the prevalence of cardiogenic shock among PCI admissions (Figure 3). The proportion of overall PCI patients presenting in cardiogenic shock monotonically declined from 2.3% in 2003 to 1.3% in 2005. Beginning with the inclusion of compassionate use criteria in late 2005, this trend reversed with an increasing proportion of patients treated with PCI for cardiogenic shock for both 2006 and 2007, to a level of 1.7%. Of note, there was no concomitant change to the risk adjustment methodologies used for predicting mortality after coronary artery bypass graft (CABG) surgery, and the proportion of CABG cases with shock has declined steadily since the inception of public reporting of post-revascularization outcomes (Figure 3).

Figure 3. Emergent revascularization for cardiogenic shock in MA, 2003–2007.

Temporal trends of the prevalence of treatment for cardiogenic shock for both the PCI (maroon columns) and isolated CABG (blue columns) cohorts in MA, 2003–2007. The red arrow indicates the time of introduction of compassionate use indicator covariates in the PCI mortality prediction models. There was no change in the risk prediction methodology for isolated CABG during the study period. A change in the prevalence of cardiogenic shock as an indication for PCI is noted to be temporally associated with the introduction of the CU indicator covariate.

Discussion

The Massachusetts experience demonstrates that a small proportion of patients at extremely high risk for in-hospital mortality can be identified using objective, pre-procedural clinical factors that had not been previously collected as part of traditional quality monitoring efforts. Incorporation of these compassionate use covariates in risk adjustment models lead to significant improvements in model performance as well as reclassification of predicted risk in a substantial proportion of cases. The NCDR dataset was supplemented with compassionate use covariates without significant changes in data collection methods, and these additional variables predictive of mortality were conducive to comprehensive case level review and adjudication.

In Massachusetts between 2005–2007, compassionate use cases represented 1.72% of all high-risk cardiogenic shock or STEMI cases and experienced an in-hospital mortality rate of 69.8% (vs. 4.45% for other shock or STEMI patients). The adjusted odds ratio for compassionate use was 27.3 (95% CI: 14.5–47.6) when compared to the non-compassionate use shock or STEMI patients. Nearly two thirds of compassionate use cases were already coded as presenting in cardiogenic shock; indicating the extreme instability of these CU patients on arrival to the cardiac catheterization laboratory. Importantly, the presence of compassionate use covariate was highly, and independently, predictive of in-hospital mortality, conferring substantial additional risk of in-hospital death beyond that of cardiogenic shock alone. Despite a prevalence of compassionate use criteria in only 1.7% of the analyzed cohort, the inclusion of the covariate significantly improved model discrimination and risk prediction.

Beyond improving the performance of the risk prediction models, the incorporation of compassionate use covariates was intended to increase the confidence of participating physicians in the risk adjustment methodology utilized in Massachusetts. While the potential association between inclusion of compassionate use covariate and changes in physician willingness to treat high risk patients with PCI cannot be directly inferred from the available data, the observed change in proportion of patients treated for cardiogenic shock (Figure 3) may indicate a positive influence on physician acceptance of the risk adjustment methodology.

There was debate within the medical and regulatory communities within MA as to the best approach to handling cases identified as CU. While many physicians advocated excluding such cases altogether from the risk prediction models, the regulator community held that CU was an appropriately “behaved” predictor variable for in-hospital mortality, which added to the discrimination of the prediction models and should therefore be included in the final model. The current study confirms the utility of including CU as an independent predictor of in-hospital mortality, and the temporal association of inclusion of CU in the model with the increasing proportion of the sickest patients in Cardiogenic shock being treated with PCI may demonstrate a physician acceptance of the improvement of modeling performance and therefore patient selection behavior.

This analysis was based on clinical data from a mandatory, state-wide observational registry with a well-publicized and rigorous audit program, and its generalizability is therefore limited. Specifically, each fatal, high risk and compassionate use case was individually reviewed and key variables adjudicated by volunteer, practicing interventional cardiologists trained in use of definitions from both the NCDR data instrument and the CU variables, resulting in high credibility of the final dataset. Not all PCI cases and not all variables in the risk prediction models could be uniformly reviewed, however. Post-PCI mortality occurring after transfer to another facility was also not evaluated.

The generalizability of incorporation of CU variables into voluntary PCI registries such as the NCDR CathPCI registry hinges on the accuracy of institutionally reported CU status, as compared with the central adjudication process utilized in MA. In the first year of use of the CU criteria, 69 cases were coded by the participating hospitals as CU of which 49 (71%) were adjudicated as meeting criteria for CU. This proportion of adjudication rejection for a high risk clinical factor is similar to the MA experience for the institutional coding of cardiogenic shock, and may therefore be representative of the accuracy of institutionally reported high risk features. Therefore, it appears that incorporation of CU in a voluntary registry may be reasonable, given the current reliance on non-centrally adjudicated high risk factors with similar adjudication rejection rates.

In conclusion, incorporation of compassionate use covariates in a statewide clinical outcomes registry is feasible, significantly improves mortality risk prediction model performance, and was temporally associated with an increase in the prevalence of cardiogenic shock among patients undergoing PCI in Massachusetts. Improvements to risk model performance with regards to extremely high risk cases may provide clinicians, regulators and health care consumers with increased confidence regarding the results of mandatory public reporting of risk adjusted PCI outcomes.