New York's Statistical Model Accurately Predicts Mortality Risk for Veterans Who Obtain Private Sector CABG

Abstract

Objective

To determine whether patients' use of the Veterans Health Administration health care system (VHA) is an independent risk factor for mortality following coronary artery bypass grafting (CABG) in the private sector in New York.

Data Sources

VHA administrative and New York Department of Health Cardiac Surgery Reporting System (CSRS) databases for surgeries performed in 1999 and 2000.

Study Design

Prospective cohort study comparing observed, expected, and risk-adjusted mortality rates following private sector CABG for 2,326 male New York State residents aged 45 years and older who used the VHA (VHA users) and 21,607 who did not (non-VHA users).

Data Collection Methods

We linked VHA administrative databases to New York's CSRS to identify VHA users who obtained CABG in the private sector in New York in 1999 and 2000. Using CSRS risk factors and previously validated risk-adjustment model, we compared patient characteristics and expected and risk-adjusted mortality rates of VHA users to non-VHA users.

Principal Findings

Compared with non-VHA users, patients undergoing private sector CABG who had used the VHA were older, had more severe cardiac disease, and were more likely to have the following comorbidities associated with increased risk of mortality: diabetes, chronic obstructive pulmonary disease, cerebrovascular disease, peripheral vascular disease, and history of stroke (p<.001 for all); a calcified aorta (p=.009); and a high creatinine level (p=.003). Observed (2.28 versus 1.80 percent) and expected (2.48 versus 1.78 percent) mortality rates were higher for VHA users than for non-VHA users. The risk-adjusted mortality rate for VHA users (1.70 percent; 95 percent confidence interval [CI]: 1.27–2.22) was not statistically different than that for the non-VHA users (1.87 percent; 95 percent CI: 1.69–2.06). Use of the VHA was not an independent risk factor for mortality in the risk-adjustment model.

Conclusions

Although VHA users had a greater illness burden, use of the VHA was not found to be an independent risk factor for mortality following private sector CABG in New York. The New York Department of Health risk adjustment model adequately applies to veterans who obtain CABG in the private sector in New York.

Since its reorganization in 1996, the Veterans Health Administration (VHA) has adopted an aggressive quality improvement strategy (Kizer 1996) and has achieved dramatic improvements on many performance measures (Jha et al. 2003). Although the VHA has demonstrated improvement in coronary artery bypass graft (CABG) surgery outcomes over the past decade (Grover et al. 2001), recent reports have questioned the overall quality of cardiac care provided in the VHA (McNeil 2003) and found that risk- and volume-adjusted in-hospital mortality rates were 40–60 percent higher in the VHA than in the private sector (Rosenthal, Vaughan Sarrazin, and Hannan 2003).

Efforts to assess the quality of VHA cardiac care by comparing risk- adjusted outcomes between VHA hospitals and private sector hospitals face a substantial methodologic challenge. Although VHA to private sector comparisons have used well-validated adjustment models that incorporate both disease severity and comorbidity measures, concern remains that they may not account for possible differences between veterans and the general population, such as socioeconomic characteristics, insurance status, existence of multiple disabilities, or other factors associated with poor medical outcomes (Berkman and Kawachi 2000). Because veterans who use the VHA system are likely to be older, sicker, poorer, and more disabled than the general population (Wilson and Kizer 1997), it is possible that risk-adjustment models do not adequately capture mortality risks for veterans who use the VHA.

To explore this possibility, we sought to identify veterans who used the VHA system and who obtained CABG through the private sector in New York State, where the Department of Health's Cardiac Surgery Reporting System (CSRS) provided both a well-validated risk adjustment model and a patient identifier that could be uniquely linked to VHA identifiers. To determine whether private sector risk-adjustment models adequately apply to veterans who use the VHA system, we compared their raw and risk-adjusted private sector CABG mortality outcomes to those of the non-VHA using a population undergoing CABG surgery in New York.

Methods

We conducted a prospective cohort study of New York veterans who obtained coronary artery bypass graft surgery in New York State during 1999 and 2000. We obtained data from two major sources: VHA administrative databases and the New York Department of Health's CSRS. VHA administrative databases include individual patient-level information including social security number, date of birth, demographics, zip code of residence, patient enrollment information, and health services utilization data. The CSRS is a cardiac registry that was established in 1989 and has been described in detail elsewhere (Hannan et al. 1994). CSRS collects patient-level data including social security number, date of birth, county of residence, demographics, procedural information, preoperative risk factors, and discharge status for every CABG procedure provided in all New York cardiac facilities, except for those operated by the VHA.

Identification of the Cohort

From VHA administrative databases, we identified all veterans who had used any VHA inpatient or outpatient health services during 1997–2000 (VHA users). We excluded females from the analysis because females represent a small portion of the veteran user population, and we were concerned that their inclusion would confound comparisons with non-VHA users. In addition, we limited our analysis to those who were 45 years of age or older because the large majority of CABG surgeries are obtained by this age group both historically (Tu et al. 1997) and during this study (only 3 percent of all CABG surgeries performed on males in New York in 1999 and 2000 were on patients younger than 45 years old). Using zip code of residence, we identified 280,180 male residents of New York state who were 45 years of age or older and who had used the VHA health system during 1997–2000.

We used probabilistic matching of social security numbers, gender, and date of birth to identify VHA users within CSRS database, initially seeking perfect matching on all three identifiers, then progressively relaxing criteria to improve match rates, as described elsewhere (Fleming et al. 1992). Of our final cohort, 91.5 percent matched perfectly. We used CSRS codes to identify isolated CABGs—CABGs without valvuloplasties—performed in non-VHA, private sector New York facilities for VHA users, and non-VHA users.

To determine whether our matching algorithm had a high sensitivity and specificity, we identified 1,867 VHA users with New York residence who were also Medicare eligible and who had received a CABG through Medicare during the two years of our study (based on claims records in Medicare's MedPAR file) and tested whether they appeared in the CSRS data set. Ninety-eight percent (1,828) of these New York veterans with a CABG appeared in the CSRS data set. To test specificity, we identified 400 Medicare eligible VHA users with Washington State residence who had also obtained CABG in the private sector in Washington State (who should therefore not match to the CSRS data set); none matched.

In-Hospital Mortality Model

We used the multivariate logistic-regression risk-adjustment model and methods developed by the New York State Department of Health to assess adjusted in-hospital mortality rates for CABG (Hannan et al. 1990; New York State Department of Health 2002). Developed over the last 14 years, the model incorporates the severity of each patient's cardiac illness as well as existing comorbid factors. The model is validated annually. Similar models have been used in comparisons of in-hospital mortality rates for CABG between the VHA and the private sector (Rosenthal, Vaughan Sarrazin, and Hannan 2003) and to track mortality rates for CABG within VHA (Shroyer et al. 1995).

The CSRS model includes a number of risk factors for CABG-related mortality including measures of disease severity (ejection fraction, time since myocardial infarction, previous cardiac surgery, patient age, and 12 comorbidities). Although New York State occasionally changes the variables included in the model we used the specific variables that contributed to the model during the time period of our analysis. These are listed in the table. Because others (Jones et al. 1996; Block et al. 1998) have included whether left main coronary stenosis is greater than 50 percent and the number of diseased vessels in risk-adjustment models, we also obtained data on those measures of disease severity—but because they neither differed when comparing VHA to non-VHA users nor statistically contributed to the New York model, we did not include them in our logistic regression modeling.

Statistical Analysis

We compared patient characteristics and outcomes of male VHA and non-VHA users who were residents of New York State, 45 years of age and older, and who had obtained CABG in New York in 1999–2000. Our dependent variable was in-hospital death, defined as a patient who died subsequent to CABG surgery during the same admission, or was discharged to hospice care (New York State Department of Health 2002).

Through the CSRS, we obtained measures of patient demographics, comorbidities, and cardiac disease severity. We used the χ² test to compare the proportion of patients with specific comorbidities or levels of cardiac disease severity. We used counts of in-hospital deaths in the two groups to calculate observed mortality rates. We used logistic regression to calculate the expected and risk adjusted mortality rates for VHA and non-VHA users by summing the predicted probabilities of death for each patient group (VHA user and non-VHA user) and dividing the respective sums by the number of patients in that group. We then generated 95 percent confidence intervals for expected- and risk-adjusted mortality rates to determine the significance of the difference for the two groups. We also included hospital effects in the model which allowed us to account for the possibility that VHA patients received care in higher or lower quality private sector hospitals.

To assess whether “VHA user status” was an independent risk factor for in-hospital CABG mortality, we used a second logistic regression model in which veteran user status was entered into the model as a fixed effect. We then calculated the risk-adjusted odds of mortality for VHA users relative to non-VHA users. Because our definition of VHA user was broad, we also performed a sensitivity analysis that restricted the VHA population to those who had used the VHA within 6 months of CABG surgery.

Results

In the CSRS data set, we identified 2,326 VHA users and 21,607 non-VHA users who were male, New York State residents, 45 years of age or older, and had obtained isolated CABG in a non-VHA New York State cardiac care center. VHA users were older, sicker, and had more severe cardiac disease than the non-VHA users who obtained CABG in the private sector in New York in 1999–2000 (Table 1). They had substantially worse cardiac disease as measured by the proportion that had experienced a previous myocardial infarction (57.8 versus 54.0 percent, p=.0005), the proportion with an ejection fraction less than 40 percent (28.7 versus 25.6 percent, p=.0013), and the proportion that had experienced previous cardiac surgery (8.9 versus 5.8 percent, p<.0001). VHA users were more likely to have the following comorbidities associated with higher CABG mortality: diabetes, chronic obstructive pulmonary disease, cerebrovascular disease, aortoiliac disease, and history of stroke (p<.0001 for all); femoral or popliteal disease (p=.0006); a calcified aorta (p=.009); and a high creatinine level (p=.003). VHA users did not differ from non-VHA users in the measures of cardiac disease severity that were excluded from the CSRS model.

Table 1.

Comparison of Risk Factors for Mortality Following CABG in VHA Users and Non-VHA Users

Risk Factors	VHA User N=2,326	Non-VHA User N=21,607	p-Value
Age (years)	70.4	65.4	<.0001
Disease severity (%)
Included in CSRS model
Previous myocardial infarction	57.8	54.0	.0005
Ejection fraction <40	28.7	25.6	.0013
Previous cardiac surgery	8.9	5.8	<.0001
Excluded from CSRS model
Left main disease (>50% stenosis)	26.2	25.9	.80
Three vessels with >70% stenosis	60.1	58.5	.16
Comorbidities (%)
Diabetes	32.3	28.3	<.0001
COPD	20.6	15.8	<.0001
Cerebrovascular disease	19.4	12.8	<.0001
Femoral/popliteal disease	9.7	7.7	.0006
History of stroke	9.6	6.3	<.0001
Aortoiliac disease	7.7	5.0	<.0001
Calcified aorta	6.3	5.0	.009
Creatinine >2.5	2.6	1.7	.003
Arrhythmia	2.3	1.8	.11
Dialysis	1.5	1.2	.20
Unstable	1.1	1.1	.92
Shock	0.3	0.4	.47
CPR	0.1	0.2	.43
Hepatic failure	0.1	0.1	.76

All patients were male and aged 45 and older.

CABG, coronary artery bypass grafting; VHA, Veterans Health Administration; CSRS, cardiac surgery reporting system; COPD, chronic obstructive pulmonary disease.

Observed mortality rates were higher for VHA users (2.28 percent, 95 percent CI: 1.67–2.89) than for non-VHA users (1.80 percent, 95 percent CI: 1.62–1.98) as were expected mortality rates, (2.48 percent, for VHA users, 95 percent CI: 1.87–3.09 versus 1.78 percent for non-VHA users, 95 percent CI: 1.61–1.95) (Figure 1). The risk-adjusted mortality rates, however, were similar: 1.70 percent for VHA users (95 percent CI: 1.27–2.22) and 1.87 percent for non-VHA users (95 percent CI: 1.69–2.06). Use of the VHA was not an independent risk factor for mortality in the CSRS risk-adjustment model (C-statistic for model=0.826): VHA users had a slightly, but not significantly, lower mortality risk when compared with non-VHA users (OR=0.897, 95 percent CI: 0.662–1.216, p=.48).

Figure 1.

Observed, Expected, and Risk-Adjusted Mortality Rates from Coronary Artery Bypass Grafting

The risk-adjustment model applied equally well when restricting the VHA population to those who had used the VHA in the 6 months prior to CABG surgery. Although VHA patients who had recently used the VHA system had significantly higher observed mortality rates (3.42 percent, 95 percent CI: 2.15–4.69), neither their higher expected mortality rates (3.08 percent, 95 percent CI: 1.92–1.98), nor their higher risk-adjusted mortality rates (2.06 percent, 95 percent CI: 1.36–3.00) were statistically different from non-VHA users. Recent use of the VHA was not an independent risk factor for mortality in the CSRS risk-adjustment model: VHA users had a slightly, but not significantly, higher mortality risk when compared with non-VHA users (OR=1.139, 95 percent CI: 0.748–1.735, p=.54).

Discussion

In New York State, resident patients who obtain CABG in the private sector and who use the VHA system to obtain some of their health care are older and sicker than those who do not use the VHA system. VHA user status, however, is not an independent risk factor for CABG mortality in the private sector in New York. The risk-adjustment model used in New York appears to adequately account for the greater disease burden and inherently higher mortality risk of VHA users who obtain their CABG surgery in the private sector.

Our findings have important implications for efforts to improve the care of patients in both the VHA and private sector. First, it is important to recognize that a substantial fraction of private sector patients also receive care in the VHA, especially among males (Fleming et al. 1992). In our analysis, VHA users represented 9.7 percent of the total male New York resident population that received CABG in New York between 1999 and 2000. This dual-system utilization raises important concerns about continuity of care. Second, our findings suggest that direct comparisons between the VHA and the private sector would be enriched by separate analyses of the subset of VHA users who receive their care in the private sector. Although VHA to private sector outcomes comparisons can help policymakers understand the magnitude of system differences (Rosenthal, Vaughan Sarrazin, and Hannan 2003), the methods that we propose may be more helpful in effecting change for the population of interest, by more specifically pinpointing whether differences in outcomes are because of patient characteristics, dual-system use, or processes of care within each system. Indeed, similar studies could be conducted from existing data sets by comparing outcomes of potentially vulnerable populations—the elderly, the uninsured, those who live in rural settings—to populations without those characteristics. Finally, our findings suggest that researchers who are interested in studying outcomes of cardiac surgery among VHA users—or for comparing outcomes across systems—should be confident in their use of current risk-adjustment models.

Our study has several limitations. First, we were inherently limited by our use of administrative databases to identify our VHA study population; in addition, because we linked a number of datasets, it is possible that we missed some cases. Our own analysis of the sensitivity and specificity of our linking methods, however, suggests that we missed few cases. Second, our results may not apply to other risk-adjustment models. Although the New York model uses most of the same data elements as the VHA (Shroyer et al. 1995) and the American Society for Thoracic Surgeons (Jones et al. 1996), among others (Block et al. 1998), New York State has much more experience with its data collection (in operation since 1989) and has lower overall mortality rates from CABG (Hannan et al. 2003). Risk-adjustment models used by other states may not apply as equally well to veterans. Finally, we were not able to directly test the model on veterans who obtain CABG within the VHA. A prior analysis in a small rural setting suggests that veterans who obtain CABG within the VHA have lower insurance coverage and more severe cardiac disease than those VHA users who obtain care in the private sector (Weeks et al. 2002). Although this analysis found that mortality rates were similar to those in the current study, we cannot rule out the possibility of unmeasured differences between veterans receiving care in the VHA and VHA users receiving care in the private sector.

Despite these limitations, our findings suggest that currently used risk-adjustment models adequately apply to patient populations with substantially greater illness burdens than experienced by the general population. VHA should consider using an identical data collection methodology and merging this data with private sector databases in order to directly compare mortality risks and outcomes of CABGs performed within VHA to those performed in the private sector on VHAs service population.