Abstract
OBJECTIVE
To compare severity-adjusted in-hospital mortality and length of stay (LOS) in a Veterans Administration (VA) hospital and private sector hospitals serving the same health care market.
DESIGN
Retrospective cohort study.
SETTING
A large VA hospital and 27 private sector hospitals in the same metropolitan area.
PATIENTS
Consecutive VA (N = 1,960) and private sector (N = 157,147) admissions in 1994 to 1995 with 9 high-volume diagnoses.
MEASUREMENTS
Severity of illness was measured using validated multivariable models that were based on data abstracted from medical records. Outcomes were adjusted for severity and compared in VA and private sector patients using multiple logistic or linear regression analysis.
MAIN RESULTS
Unadjusted mortality was similar in VA and private sector patients (5.0% vs 5.6%, respectively; P = .26), although mean LOS was longer in VA patients (12.7 vs 7.0 days; P < .001). Adjusting for severity, the odds of death in VA patients was similar (odds ratio [OR] 1.07; 95% confidence interval [95% CI], 0.74 to 1.54; P = .73). However, a larger proportion of deaths in VA patients occurred later during hospitalization (P < .001), and the odds of death in VA patients were actually lower (P < .05) in analyses limited to deaths during the first 7 (OR, 0.56) or 14 (OR, 0.63) days. Adjusted LOS was longer (P < .001) in VA patients for all 9 diagnoses.
CONCLUSIONS
If the current findings generalizable to other markets, hospital mortality, a widely used performance measure, may be similar or lower in VA and private sector hospitals serving the same markets. The longer LOS of VA patients may reflect differences in practice patterns and may be an important source of bias in comparisons of VA and private sector hospitals.
Keywords: length of stay; hospital mortality; severity of illness; hospitals, veterans; risk adjustment
During the late 1980s to mid 1990s, an increasing emphasis was placed on the use of outcomes data to evaluate the quality and efficiency of hospital care.1,2 More recently, concerns about medical errors and patient safety have renewed demands on health care providers for accountability and have stimulated interest in collection and reporting of provider-specific data.3,4
As the nation's largest organized health care system, the Veterans Administration (VA) has developed several programs to measure outcomes within individual VA facilities. Most notable are programs examining outcomes of general surgery5 and cardiac surgery.6 Although these programs provide comparative data with other VA hospitals, the VA has not participated in similar regional or statewide efforts involving private sector hospitals.7–9 Thus, while it is possible to benchmark the performance of individual VA hospitals against the best performing VA hospitals, little information exists on performance, relative to the larger world of private sector care. Such information might be particularly valuable as a means to identify best practices and improve the quality of VA care.
To date, empirical comparisons of VA and private sector care are largely limited to several studies using administrative (i.e., claims) data10,11 and a few studies using clinical data abstracted from medical records.12–15 While administrative data may be reliable for examining issues related to access and utilization, the validity of administrative data in comparing outcomes may be suspect, given the well-documented difficulties in measuring severity of illness.16,17 In addition, most of the studies based on clinical data only examined a single diagnosis, such as acute myocardial infarction.12,15 Furthermore, no prior studies have systematically compared outcomes of VA and private sector hospitals in a defined geographic region.
The primary objective of this research was to compare in-hospital mortality and length of stay (LOS) in a VA hospital and in private sector hospitals serving a large metropolitan area. The study capitalized on a unique regional program to measure hospital performance in 27 private sector hospitals that organized the collection of clinical data from the medical records of patients with 9 high-volume medical and surgical diagnoses. These data permitted adjustments for admission severity of illness using validated multivariable models. Similar data on a concurrent cohort of patients in a VA hospital serving the same health care market were then collected, and severity-adjusted outcomes in the VA hospital compared to outcomes in the private sector hospitals. As a secondary objective, the study explored whether comparisons of in-hospital mortality may be biased by the typically longer LOS in VA hospitals.
METHODS
Hospitals
The study was conducted at the Cleveland VA Medical Center, a 392-acute care bed system affiliated with Case Western Reserve University School of Medicine, and 27 acute care nongovernmental hospitals in Northeast Ohio that participated in Cleveland Health Quality Choice, a regional initiative to compare hospital performance.7 The 27 private sector hospitals included 21 private nonprofit, 5 church-affiliated, and 1 public (county) hospital. Five hospitals were members of the Council of Teaching Hospitals of the Association of America Medical Colleges during the period of data collection, and were considered major teaching hospitals. Other characteristics of participating hospitals have been described previously.18 The study protocol was reviewed and approved by the Institutional Review Board and the Research and Development Committee of the Cleveland VA Medical Center.
Patients
The eligible sample consisted of consecutive admissions who were: 1) 18 years or older; 2) discharged from the study hospitals during the 24-month period from January 1994 to December 1995; and 3) had 1 of the following 9 medical and surgical diagnoses: acute myocardial infarction, congestive heart failure, stroke, pneumonia, obstructive airway disease, gastrointestinal hemorrhage, lower bowel resection, peripheral vascular surgery, and coronary artery bypass surgery. Patients were identified from computerized hospital databases by qualifying principal International Classification of Diseases, 9th revision, Clinical Modification (ICD-9-CM) diagnosis codes for medical diagnoses, and by both a qualifying ICD-9-CM procedure code and qualifying ICD-9-CM diagnoses code for surgical diagnoses.19
A total of 2,084 eligible VA admissions were identified. Of these, medical records were available for 1,960 (94.0%). Admissions for whom medical records were available were similar (P > .2) to the 124 admissions for whom medical records were unavailable in mean age (68.2 vs 67.6 years, respectively), gender (98.0% vs 97.6% male, respectively), and race (64.6% vs 67.7% white, respectively). In addition, the 2 groups had similar rates of in-hospital mortality (5.0% vs 7.3%, respectively), although mean hospital LOS was somewhat lower in patients for whom medical records were available (12.7 vs 16.0 days, respectively; P = .07). During the study period, data for 157,147 private sector patients admitted to the 27 nongovernmental hospitals were obtained from Cleveland Health Quality Choice. Distributions of VA and private sector patients according to diagnosis are shown in Table 1. The 3 most common diagnoses, congestive heart failure, pneumonia, and obstructive airway disease, accounted for 58% and 57% of VA and private sector patients, respectively.
Table 1.
Distributions of VA and Private Sector Patients by Diagnosis
| Diagnosis | VA (N = 1,960), % (n) | Private Sector (N = 157,147), % (n) |
|---|---|---|
| Congestive heart failure | 23.0 (450) | 22.5 (35,343) |
| Pneumonia | 19.8 (388) | 18.5 (29,066) |
| Obstructive airway disease | 15.7 (307) | 15.6 (24,470) |
| Coronary artery bypass surgery | 10.2 (200) | 7.9 (12,451) |
| Gastrointestinal hemorrhage | 9.2 (180) | 7.9 (12,482) |
| Stroke | 8.6 (168) | 9.4 (14,740) |
| Peripheral vascular surgery | 6.1 (120) | 3.6 (5,611) |
| Acute myocardial infarction | 4.7 (92) | 10.8 (16,903) |
| Lower bowel resection | 2.8 (55) | 3.9 (6,081) |
Data
Trained reviewers abstracted data for eligible patients from medical records using standard forms. Abstracted data elements include: 1) sociodemographic characteristics; 2) admission source; 3) comorbid conditions and substance abuse history; 4) use of specific medications prior to admission; 5) dates of all “do not resuscitate” orders; 6) ICD-9 primary and secondary diagnosis and procedure codes; 7) admission (defined as the first 2 days of hospitalization) laboratory, radiological, electrocardiographic, and echocardiographic findings; 8) patient disposition (discharge vital status, discharge location, hospital LOS); and 9) selected laboratory and clinical findings during hospitalization to determine occurrence of hospital-acquired complications.
Protocols for ensuring the reliability of Cleveland Health Quality Choice have been reported previously.7,19 These protocols included required semiannual training sessions for all data abstractors, development of data dictionaries and explicit definitions of each data element, double keystroke data entry, identification and correction of variables with missing or out-of-range values, and independent evaluation of the reliability of data abstracted at each hospital. Data collection at the Cleveland VA Medical Center followed identical protocols, including independent audits of the reliability of abstracted data that were performed by data abstractors employed by Cleveland Health Quality Choice.
Assessment of Severity of Illness
Admission severity of illness was determined using multivariable risk-adjustment models for each diagnosis that were developed and validated by Cleveland Health Quality Choice and that were previously used to compare outcomes in hospitals participating in the Cleveland Health Quality Choice program.18,19 Separate risk-adjustment models developed for mortality and LOS. Each model included 18 to 35 variables that were independently associated with mortality, as determined by stepwise logistic regression, or LOS, as determined by stepwise linear regression. Variables included age, admission source, comorbid conditions, admission vital signs and diagnostic tests collected during the first 48 hours of admission. Individual variables in each of the models and variable weights are available from the authors upon request.
Performance of the mortality risk-adjustment models in VA and private sector patients was determined by the c statistic.20 The c statistic is numerically equivalent to the area under the receiver operating characteristic curve and represents the proportions of times that patients who died had a higher predicted risk of death than each patient who was discharged alive. Performance of the LOS models was determined by the explained variance (R2).
Analysis
Characteristics and observed in-hospital mortality and LOS of patients in the VA hospital and the 27 private sector hospitals were compared using the χ2, t, or Wilcoxon signed rank test, as appropriate.
To adjust mortality rates in the VA and private sector hospitals for differences in severity of illness, additional logistic regression analyses were conducted. Independent variables in these analyses included the individual variables from the severity of illness risk-adjustment variables and an indicator variable for VA hospitalization. The antilog of the regression coefficient associated with the VA indicator variable was used to estimate the odds of death in VA patients, relative to patients in private sector hospitals. Separate analyses were conducted for each of the 9 diagnoses. We conducted an additional logistic regression analysis that compared mortality for all 9 diagnoses, in aggregate. Independent variables in this analysis were an indicator variable for VA hospitalization and a predicted risk of death for each patient. Predicted risks of death were derived from the multivariable severity of illness models, which did not include a VA indicator variable. To improve model fit, the log of the predicted risk of death was used, rather than the actual predicted risk of death. The logistic regression analyses compared mortality in the VA hospital to all 27 private sector hospitals and to the 5 major teaching hospitals separately. The latter hospitals were found, in a prior study,18 to have lower mortality and LOS than the 22 minor teaching and nonteaching hospitals. We conducted additional logistic regression analyses that examined only those deaths that occurred within 4, 7, 14, or 21 days of admission. These analyses were conducted to examine the potential impact of differences in LOS on in-hospital mortality. Finally, because VA patients were predominantly male, we conducted analyses in men only. These analyses yielded similar findings as analyses in all patients and are not reported.
All logistic regression analyses were conducted using hierarchical modeling21 that accounted for clustering of patients (i.e., correlations among patients in individual hospitals) by including both fixed and random hospital effects.
To adjust LOS for differences in severity of illness, linear regression analyses were conducted, using a similar analytic strategy as was used for the mortality analyses. Because of the skewed distribution of LOS, analyses were conducted using log transformed data. Because of different relationships between risk variables and LOS of patients who died and who were discharged alive, LOS analyses excluded patients who died. In the LOS analyses, the antilog of the regression coefficient associated with the indicator variable for VA hospitalization was used to estimate the relative difference (i.e., percent difference) in LOS of VA and private sector patients.
RESULTS
Characteristics of study patients are shown in Table 2. VA patients were younger (P < .001), more likely to be male, and less likely to be white. In addition, VA patients were less likely to be admitted from home, but more likely to be admitted from another acute care hospital. VA patients also were more likely to be admitted through the emergency room. Several preexisting comorbid conditions were more common in VA patients, including diabetes mellitus, peripheral vascular disease, ischemic heart disease, cirrhosis, and cerebral vascular disease. Asthma was less common in VA patients.
Table 2.
Demographic and Clinical Characteristics of VA and Private Sector Study Patients
| VA | Private Sector | P Value | |
|---|---|---|---|
| Mean age, y ± SD | 66.2 ± 10.4 | 68.2 ± 14.7 | <.001 |
| White, % | 64.6 | 78.1 | <.001 |
| Male, % | 98.0 | 48.6 | <.001 |
| Admission source, % | <.001 | ||
| Home | 77.6 | 84.9 | |
| Nursing home or rehabilitation facility | 11.5 | 10.5 | |
| Other acute care hospital | 9.4 | 3.8 | |
| Other | 1.1 | 0.2 | |
| Not documented | 0.5 | 0.7 | |
| Admission through emergency room, % | 73.1 | 67.9 | <.001 |
| Comorbid conditions prior to admission, % | |||
| Asthma | 3.6 | 4.9 | <.001 |
| Cancer, metastatic | 2.2 | 2.2 | .98 |
| End-stage renal dialysis | 1.4 | 1.6 | .53 |
| Cerebral vascular disease | 18.3 | 16.0 | .005 |
| Ischemic heart disease | 45.1 | 40.5 | <.001 |
| Cirrhosis | 1.8 | 0.7 | <.001 |
| Peripheral vascular disease | 18.0 | 11.8 | <.001 |
| Restrictive lung disease | 1.1 | 1.0 | .72 |
| Diabetes mellitus | 31.3 | 25.5 | <.001 |
Unadjusted Mortality and LOS
The overall unadjusted in-hospital mortality rate was similar in VA and private sector patients (5.0% vs 5.6%, respectively; P = .26). In analyses of individual diagnoses (Table 3), mortality was significantly (P < .05) lower for 1 diagnosis (acute myocardial infarction) and higher for 1 diagnosis (lower bowel resection). Among the 98 VA patients and the 8,478 private sector patients who died, VA deaths occurred later in the hospital stay (P < .001; Table 4). For example, only 8% of VA deaths occurred during the first or second hospital day, compared to 23% of private sector deaths, while 37% of VA deaths occurred on Hospital Day 22 or later, compared to 8% of private sector deaths.
Table 3.
Unadjusted (i.e., Observed) Mortality Rates in VA and Private Sector Patients and Odds of Death in VA Patients, Relative to Private Sector Patients, Adjusting for Admission Severity of Illness Using Validated Multivariable Models
| Observed Mortality Rates, % (Number of deaths) | Adjusted Odds of Death | |||||
|---|---|---|---|---|---|---|
| Diagnosis | VA | Private Sector | P Value | Odds Ratio | 95% CI | P Value |
| Congestive heart failure | 3.8 (17) | 4.4 (1,555) | .48 | 0.93 | 0.50 to 1.74 | .83 |
| Pneumonia | 9.5 (37) | 8.7 (2,529) | .63 | 1.24 | 0.77 to 2.01 | .38 |
| Obstructive airway disease | 1.3 (4) | 1.9 (465) | .57 | 0.43 | 0.14 to 1.35 | .14 |
| Coronary artery bypass surgery | 3.0 (6) | 2.6 (324) | .54 | 1.19 | 0.42 to 3.32 | .75 |
| Gastrointestinal hemorrhage | 3.9 (7) | 3.4 (424) | .60 | 1.15 | 0.51 to 2.58 | .73 |
| Stroke | 6.5 (11) | 9.2 (1,356) | .29 | 1.57 | 0.80 to 3.10 | .19 |
| Peripheral vascular surgery | 6.7 (8) | 4.5 (252) | .38 | 1.44 | 0.56 to 3.68 | .45 |
| Acute myocardial infarction | 1.1 (1) | 10.1 (1,707) | .01 | 0.09 | 0.01 to 0.69 | .02 |
| Lower bowel resection | 10.9 (6) | 3.7 (225) | .01 | 3.50 | 1.16 to 10.60 | .03 |
Table 4.
Number and Percent of Deaths in VA and Private Sector Patients by Hospital Day
| VA (N = 98 Deaths)* | Private Sector (N = 8,478 Deaths) | |||||
|---|---|---|---|---|---|---|
| Hospital Day | Number of Deaths | Percent of All Deaths | Cumulative Percent | Number of Deaths | Percent of All Deaths | Cumulative Percent |
| 1–2 | 8 | 8.4 | 8.4 | 1,963 | 23.2 | 23.2 |
| 3–4 | 10 | 10.7 | 19.1 | 1,489 | 17.5 | 40.7 |
| 5–7 | 11 | 11.3 | 30.4 | 1,628 | 19.2 | 59.9 |
| 8–10 | 9 | 9.2 | 39.6 | 1,029 | 12.1 | 72.1 |
| 11–14 | 11 | 11.2 | 50.8 | 908 | 10.7 | 82.8 |
| 15–21 | 13 | 13.3 | 63.2 | 743 | 8.7 | 91.5 |
| 22 and beyond | 36 | 36.7 | 100.0 | 718 | 8.5 | 100.0 |
Differences in the proportions of VA and private sector patients who died during different intervals was significant (P < .001).
Mean LOS for all 9 diagnoses in aggregate was considerably higher in VA patients than in private sector patients (12.7 vs 7.0 days, respectively; P < .001). In analyses of individual diagnoses (Table 5), mean LOS was higher (P < .05) in VA patients for all 9 diagnoses.
Table 5.
Unadjusted (i.e., Observed) Length of Stay in VA and Private Sector Patients and the Percent Difference in Length of Stay in VA Patients, Relative to Private Sector Patients, Adjusting for Admission Severity of Illness Using Validated Multivariable Models
| Observed Mean Length of Stay, d ± SD | Adjusted Difference in Length of Stay | |||||
|---|---|---|---|---|---|---|
| Diagnosis | VA | Private Sector | P Value | Difference, % | 95% CI, % | P Value |
| Congestive heart failure | 9.6 ± 12.7 | 6.3 ± 5.1 | <.001 | 31 | 23 to 40 | <.001 |
| Pneumonia | 11.0 ± 12.1 | 7.5 ± 6.0 | <.001 | 20 | 12 to 28 | <.001 |
| Obstructive airway disease | 8.7 ± 13.8 | 5.2 ± 4.6 | <.001 | 18 | 9 to 28 | <.001 |
| Coronary artery bypass surgery | 17.2 ± 16.4 | 9.4 ± 7.3 | <.001 | 76 | 66 to 87 | <.001 |
| Gastrointestinal hemorrhage | 6.8 ± 7.7 | 5.1 ± 5.1 | .04 | 21 | 9 to 34 | <.001 |
| Stroke | 19.2 ± 27.6 | 7.3 ± 5.9 | <.001 | 115 | 94 to 139 | <.001 |
| Peripheral vascular surgery | 31.0 ± 24.4 | 9.4 ± 7.6 | <.001 | 156 | 130 to 186 | <.001 |
| Acute myocardial infarction | 10.5 ± 6.9 | 7.1 ± 4.6 | <.001 | 45 | 29 to 64 | <.001 |
| Lower bowel resection | 17.3 ± 17.8 | 11.0 ± 8.7 | <.001 | 36 | 20 to 54 | <.001 |
Risk-adjustment Model Performance
Table 6 shows the performance characteristics of the mortality (as measured by the c statistic) and LOS (as measured by explained variance[R2]) risk-adjustment models in VA and private sector patients. The mortality models generally exhibited good discrimination in VA patients. For 7 of the 9 models, c statistics were 0.80 or higher. As would be expected, for most diagnoses c statistics tended to be higher in private sector patients, the group in which the models had been originally developed. The mean (weighted according to numbers of patients) c statistic across the 9 diagnoses was somewhat lower in VA than in private sector patients (0.84 vs 0.87, respectively). Relative differences in performance of the LOS models in VA and private sector patients tended to be greater, particularly for the 3 surgical diagnoses in which R2 values were 2- to 3-fold higher in private sector patients. Mean weighted R2 values for the 9 diagnoses were .13 and .21 in VA and private sector patients, respectively.
Table 6.
Performance of Risk-adjustment Models for Mortality (as Measured by the c Statistic) and Length of Stay (as Measured by the Explained Variance) in VA and Private Sector Patients
| Mortality Models (c Statistic) | Length of Stay Models (Explained Variance) | |||
|---|---|---|---|---|
| Diagnosis | VA | Private Sector | VA | Private Sector |
| Congestive heart failure | 0.86 | 0.86 | 0.05 | 0.14 |
| Pneumonia | 0.85 | 0.87 | 0.20 | 0.23 |
| Obstructive airway disease | 0.80 | 0.90 | 0.12 | 0.19 |
| Coronary artery bypass surgery | 0.93 | 0.83 | 0.15 | 0.30 |
| Gastrointestinal hemorrhage | 0.84 | 0.88 | 0.23 | 0.21 |
| Stroke | 0.85 | 0.91 | 0.11 | 0.19 |
| Peripheral vascular surgery | 0.69 | 0.79 | 0.09 | 0.26 |
| Acute myocardial infarction | 0.94 | 0.90 | 0.19 | 0.20 |
| Lower bowel resection | 0.76 | 0.87 | 0.14 | 0.32 |
Risk-adjusted Mortality and LOS
Mortality and LOS in VA patients were adjusted for severity of illness by introducing a coefficient for VA hospitalization directly into the risk-adjustment models. Risk-adjusted in-hospital mortality was similar in VA patients, relative to private sector patients (odds ratio [OR], 1.07; 95% confidence interval [95% CI], 0.74 to 1.54; P = .73) for all 9 diagnoses, in aggregate (Table 3). In analyses of individual diagnoses, mortality in VA patients was lower (P < .05) for acute myocardial infarction and higher (P < .05) for lower bowel resection. Additional comparisons to patients in the 5 major teaching hospitals found that mortality for all diagnoses in aggregate was similar in VA and private sector patients (OR, 1.25; 95% CI, 0.92 to 1.72; P = .16). In analyses of individual diagnoses, VA mortality again was lower (P < .05) for acute myocardial infarction and higher (P < .05) for lower bowel resection.
However, adjusting for severity of illness did not attenuate differences that were observed in unadjusted LOS. Risk-adjusted LOS was longer (P < .001) in VA patients, relative to patients in private sector hospitals for each of the 9 diagnoses (Table 5). Percent differences in risk-adjusted LOS ranged from 18% longer for obstructive airway disease to 156% longer for peripheral vascular surgery. These results were again similar in comparisons to patients in the 5 major teaching hospitals. Risk-adjusted LOS was longer (P < .001) in VA patients for each of the 9 diagnoses. Percent differences ranged from 31% longer for pneumonia to 129% longer for peripheral vascular disease.
Finally, to examine the potential impact of differences in LOS on mortality, a series of analyses were conducted that only considered deaths occurring within the fourth, seventh, 14th, and 21st hospital day. Odds ratios for these analyses were 0.54 (95% CI, 0.27 to 1.07; P = .08), 0.56 (95% CI, 0.33 to 0.96; P = .04), 0.63 (95% CI, 0.41 to 0.98; P = .04), and 0.72 (95% CI, 0.48 to 1.08; P = .12), respectively. Notably, the odds ratios were less than 1.0 for each of the endpoints and significantly lower than 1.0 for analyses of deaths within the seventh and 14th hospital days. In analyses limited to comparisons of mortality in the 5 major teaching hospitals, odds ratios were less than 1.0 for all endpoints, although none of the odds ratios was statistically significant. Odds ratios for analyses of deaths within the fourth, seventh, 14th, and 21st hospital day were 0.76 (95% CI, 0.35 to 1.63; P = .47), 0.74 (95% CI, 0.44 to 1.24; P = .25), 0.79 (95% CI, 0.53 to 1.17; P = .24), and 0.88 (95% CI, 0.62 to 1.27; P = .50), respectively.
DISCUSSION
The current study represents one of the first comparisons of outcomes in a VA hospital and private sector hospitals serving the same health care market. Evaluating consecutive admissions over a 24-month period with 9 common medical and surgical conditions, the study found that in-hospital mortality was similar in VA and private sector patients, after adjusting for admission severity of illness using previously validated risk-adjustment models. This finding was consistent in comparisons to the 5 major teaching hospitals. However, VA patients had considerably longer length of stay, and among patients who died, a higher proportion of VA patients died later during the hospital stay. To examine the potential bias of the difference in lengths of stay and the longer period of observation of VA patients, additional comparisons were done that only examined deaths occurring during fixed intervals after admission. In contrast to the results based on all in-hospital deaths, analyses limited to deaths within 4, 7, 14, and 21 days of admission found that mortality was lower in VA patients when compared to all hospitals and similar when compared to the 5 major teaching hospitals.
Although these results exhibited some variability across individual diagnoses, the findings suggest that the VA hospital had lower, or at least similar, mortality rates as private sector hospitals in the same geographic market. The findings also highlight the potential methodological challenges in using in-hospital mortality as an endpoint when comparing health care systems that may have very different patterns of hospital utilization.
Our finding of longer hospital LOS in VA patients is consistent with several prior studies.22–24 Although the current study was not designed to examine factors leading to the differences in LOS or the clinical appropriateness of such differences, it is possible that the longer LOS and greater number of late deaths in VA patients reflect higher rates of hospital-acquired complications. A further possibility is that the VA hospital had less access to nursing homes, hospice, or other post-acute care facilities.25 Such differences in access may reflect differences in mechanisms for reimbursements in VA and private sector systems, greater pressures on private sector hospitals to decrease hospital LOS and shift care to other settings, or lower social and family support in VA patients.
Our findings add to the growing body of literature comparing outcomes in VA and private sector hospitals. An initial series of studies used administrative records to compare outcomes of patients undergoing more than 100 different surgical procedures. These analyses found no systematic differences in postsurgical mortality in VA and private sector hospitals.10,11 A major limitation of these analyses was their reliance on administrative data, which are limited in their ability to account for differences in severity of illness.
More recently, several studies have used clinical data abstracted from patients' medical records to adjust for severity of illness. A study of 385 patients with acute myocardial infarction in a VA and an affiliated university hospital staffed by similar physicians found no differences in in-hospital mortality.12 A subsequent analysis of 5,016 medical and surgical patients at the same VA hospital found similar mortality to that observed in a normative database of patients in private sector hospitals, after adjusting for severity of illness using MedisGroups.26 A further study of patients with acute myocardial infarction, based on a nationally representative sample of VA hospitals, found that 30-day severity-adjusted mortality rates were similar in VA and private sector patients.15
The results of the current study are also consistent with the results of a concurrent study of patients admitted to intensive care units in the same VA hospital.27 That study also found that in-hospital mortality was similar in VA and private sector and that a higher proportion of deaths in VA patients occurred later in the hospitalization. After accounting for the longer period of observation in VA patients using survival analysis, mortality was actually 30% lower in VA patients after adjusting for admission severity of illness.27
In interpreting our findings, it is important to consider several potential limitations. First, the study involved a single health care market. Generalizability to other VA hospitals and to other health care markets needs to be established. However, additional analyses using VA administrative data of the 5 most prevalent diagnoses examined in the current study found that mortality was similar (P≥.10) in the study hospital and other VA hospitals nationally for 4 of the 5 diagnoses (congestive heart failure, pneumonia, coronary artery bypass graft, and gastrointestinal hemorrhage), and was lower only for chronic obstructive lung disease. Length of stay was similar for 3 of the 5 diagnoses. Other data suggest that the efficiency of care in private sector hospitals in the Cleveland metropolitan area may be similar to hospitals in other markets. For example, American Hospital Association data indicate that in 1995 nonfederal hospitals in the Cleveland metropolitan statistical area had an average LOS of 6.2 days and a 62% average daily census, compared to 6.5 days and 63%, respectively, in U.S. hospitals nationally.28
Second, although the study used previously validated multivariable risk-adjustment models that were based on clinical data from patients' medical records, the results may be confounded by unmeasured severity of illness. For example, our models did not consider biopsychosocial factors, such as functional status, cognition, mood, or social support29–31 that may impact outcomes. The results also may be confounded by differences in patient preferences for care or other aspects of patient or physician decision making.32 Our findings also may be affected by systematic differences between hospitals in the recording of clinical information used in risk-adjustment models. Similarly, the models incorporated physiologic data and the results of diagnostic tests from the first 48 hours of hospitalization. Although this period of observation is commonly used in severity adjustment methods,33 it may inappropriately incorporate the adverse effects of inadequate or deleterious care that may be provided during this period into the assessment of severity.
Third, our results may be confounded by selection bias due to systematic differences in admission practices between VA and private sector hospitals or differences in hospital referral patterns.34 Such selection biases may result in the preferential admission (or exclusion) of patients with different underlying prognoses, independent of severity of illness. Assessment of such bias would be extremely difficult and would require information about all possible hospital admissions (e.g., patients presenting for evaluation to outpatient clinics, emergency rooms, or physicians' offices or all patients who are candidates for possible hospital transfer).
Fourth, we were unable to directly examine circumstances surrounding the cause of death. Thus, it is unclear if the higher proportion of VA patients who died late during hospitalization reflected differences in care or differences in the ability to discharge patients to home or to skilled care settings. Such differences in discharge practices may merely shift the site of death away from the hospital, and may bias comparisons based on in-hospital outcomes. Thus, a preferable study endpoint would have been mortality occurring within a fixed time from admission (e.g., 30 days). However, the lack of unique patient identifiers in the private sector database made it extremely difficult to determine this endpoint. Moreover, recent efforts to protect patient confidentiality that limit the use of administrative data may make it increasingly difficult to determine post-hospital outcomes for large population-based studies.
Fifth, the relatively small numbers of VA patients with individual diagnoses may have limited our ability to detect statistically significant differences in mortality that may have been of clinical significance. Finally, we did not have physician identifiers and were unable to account for physician-level effects in our modeling.
Despite the above potential limitations, the current findings have several potential important implications for national health policy and for health services research. If the findings are generalizable to other health care markets, the quality of VA care, as measured by severity-adjusted mortality, may be at least similar in VA hospitals. While quality of care is a multidimensional concept, outcomes are an important attribute of quality35 and have been shown in several studies to be related to other indicators of quality.36–39
In addition, the current study represents one of the only direct comparisons of VA care and private sector at the level of an individual health care market. Such direct comparisons may be particularly relevant in light of proposals to outsource certain VA health care services to private sector health care providers or provide VA patients with vouchers that would enable them to obtain care from providers of their choice in a given health care market.40,41 In the absence of differences in quality or in patient outcomes between VA and private sector facilities in individual markets, these policy options will largely be debated on the basis of economic considerations.
Finally, our findings demonstrate the need for further evaluation of the utilization of acute care beds in VA and private sector hospitals. The differences in LOS suggest that there may be substantial opportunities for cost savings in VA hospitals by developing lower cost skilled nursing units or improving access to private facilities.
Acknowledgments
The views expressed in this article are those of the authors and do not necessarily represent the views of the Department of Veterans Affairs.
Dr. Rosenthal is currently a Senior Quality Scholar, Office of Academic Affiliation, Veterans Health Administration, Department of Veterans Affairs. This research was supported by an investigator-initiated award (IIR 94-093) and a Career Development Award to Dr. Rosenthal from the Health Services Research and Development Service, Veterans Health Administration, Department of Veterans Affairs.
REFERENCES
- 1.Ellwod PM. Shattuck lecture. Outcomes management: a technology of patient experience. N Engl J Med. 1988:1549–56. doi: 10.1056/NEJM198806093182327. [DOI] [PubMed] [Google Scholar]
- 2.Roper WL, Winkenwerder W, Hackbarth GM, Krakauer H. Effectiveness in health care: an initiative to evaluate and improve medical practice. N Engl J Med. 1988;319:1197–202. doi: 10.1056/NEJM198811033191805. [DOI] [PubMed] [Google Scholar]
- 3.Doing what counts for patient safety: Federal actions to reduce medical errors and their impact. Washington, DC: Quality Interagency Coordination Task Force; QuIC Report to the President. 1999;February 21. Available at: http://www.quic.gov. Accessed June 24, 2003. [Google Scholar]
- 4.Committee on Quality Health Care in America. Crossing the Quality Chasm: A New Health Care System for the 21st Century. Washington, DC: National Academy Press; 2001. [Google Scholar]
- 5.Khuri SF, Daley J, Henderson W, et al. The National Veterans Administration Surgical Risk Study: risk adjustment for the comparative assessment of the quality of surgical care. J Am Coll Surg. 1995;180:519–31. [PubMed] [Google Scholar]
- 6.Hammermeister KE, Johnson R, Marshall G, Grover FL. Continuous assessment and improvement in quality of care. A model from the Department of Veterans Affairs Cardiac Surgery. Ann Surg. 1994;219:281–90. doi: 10.1097/00000658-199403000-00008. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7.Rosenthal GE, Harper DL. Cleveland Health Quality Choice: a model for community-based outcomes assessment. Jt Comm J Qual Improv. 1994;20:425–44. doi: 10.1016/s1070-3241(16)30088-8. [DOI] [PubMed] [Google Scholar]
- 8.Hannan EL, Kilburn H, O'Donnell JF, Lukacik G, Shields EP. Adult open heart surgery in New York state. JAMA. 1990;264:2768–74. [PubMed] [Google Scholar]
- 9.Hospital Effectiveness Report. Harrisburg, Pa: Pennsylvania Health Care Containment Council; 1991. [Google Scholar]
- 10.Stremple JF, Bross DS, Davis CL, McDonald GO. Comparison of postoperative mortality and morbidity in VA and nonfederal hospitals. J Surg Res. 1994;56:405–16. doi: 10.1006/jsre.1994.1065. [DOI] [PubMed] [Google Scholar]
- 11.Stremple JF, Bross DS, Davis CL, McDonald GO. Comparison of postoperative mortality in VA and private hospitals. Ann Surg. 1993;217:277–85. doi: 10.1097/00000658-199303000-00010. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12.Rosenthal GE, Larimer DJ, Owens KE. Treatment of patients with acute myocardial infarction at a Veterans Administration (VA) hospital and a non-VA hospital. J Gen Intern Med. 1994;9:455–8. doi: 10.1007/BF02599064. [DOI] [PubMed] [Google Scholar]
- 13.Steinmiller AM. Management of uncomplicated acute myocardial infarction: are there differences in patterns of care between VAMC physicians and private practice physicians? Mil Med. 1991;3:104–7. [PubMed] [Google Scholar]
- 14.Heck DA, Maar DC, Lowdermilk GA, Kalasinki LA, Mesko JW. Comparative analysis of total knee arthroplasty in two health care delivery systems. J Arthroplasty. 1992;7:93–100. doi: 10.1016/0883-5403(92)90038-r. [DOI] [PubMed] [Google Scholar]
- 15.Petersen LA, Normand SLT, Daley J, McNeil BJ. Outcome of myocardial infarction in veterans health administration patients compared with Medicare patients. N Engl J Med. 2000;343:1934–41. doi: 10.1056/NEJM200012283432606. [DOI] [PubMed] [Google Scholar]
- 16.Greenfield S, Aronow HU, Elashoff RM, Watanabe D. Flaws in mortality data: the hazards of ignoring comorbid disease. JAMA. 1988;260:2253–5. [PubMed] [Google Scholar]
- 17.Iezzoni LI. Data sources and implications: Administrative databases. In: Iezzoni LI, editor. Risk Adjustment for Measuring Health Care Outcomes. 2nd ed. Chicago, Ill: Health Administration Press; 1997. pp. 169–242. [Google Scholar]
- 18.Rosenthal GE, Harper DL, Quinn LM, Cooper GS. Severity-adjusted mortality and length of stay in teaching and non-teaching hospitals. Results of a regional study. JAMA. 1997;278:485–90. [PubMed] [Google Scholar]
- 19.Rosenthal GE, Quinn L, Harper DL. Declines in hospital mortality associated with a regional initiative to measure hospital performance. Am J Med Qual. 1997;12:103–12. doi: 10.1177/0885713X9701200204. [DOI] [PubMed] [Google Scholar]
- 20.Ash AS, Shwartz M. Evaluating the performance of risk-adjustment methods: dichotomous methods. In: Iezzoni LI, editor. Risk Adjustment for Measuring Health Care Outcomes. 2nd ed. Chicago, Ill: Health Administration Press; 1997. pp. 427–70. [Google Scholar]
- 21.Mixed Models Analyses Using the SAS System Course Notes. Cary, NC: SAS Institute; 2002. [Google Scholar]
- 22.Thomas JW, Bates EW, Hofer T, Perkins A, Foltz-Murphy N, Webb C. Interpreting risk-adjusted length of stay patterns for VA hospitals. Med Care. 1998;36:1660–75. doi: 10.1097/00005650-199812000-00004. [DOI] [PubMed] [Google Scholar]
- 23.Rogers JL, Feinglass J, Martin GJ, Hughes RL, Handler I, Stoms GB. Longer hospitalization at Veterans Administration hospitals than private hospitals. Verification and additional insights. Med Care. 1989;27:928–36. doi: 10.1097/00005650-198910000-00003. [DOI] [PubMed] [Google Scholar]
- 24.Wolinsky FD, Coe RM, Mosely RR. Length of stay in the VA. Long-term care in short-term hospitals. Med Care. 1987;25:250–3. doi: 10.1097/00005650-198703000-00008. [DOI] [PubMed] [Google Scholar]
- 25.Better Patient Management Practices Could Reduce Length of Stay in VA Hospitals. Washington, DC: General Accounting Office; 1985. [Google Scholar]
- 26.Gordon HS, Fuehrer SM, Aron DC, Montague WD, Rosenthal GE. Using severity-adjusted mortality to compare performance in a Veterans Affairs (VA) hospital and in private sector hospitals. Am J Med Qual. 2000;15:207–11. doi: 10.1177/106286060001500505. [DOI] [PubMed] [Google Scholar]
- 27.Kaboli PJ, Barnett MJ, Fuehrer SM, Rosenthal GE. Length of stay as a source of bias in comparing performance in VA and private sector facilities: lessons learned from a regional evaluation of intensive care outcomes. Med Care. 2001;39:1014–24. doi: 10.1097/00005650-200109000-00011. [DOI] [PubMed] [Google Scholar]
- 28.Hospital Statistics. 1996–97 Based on 1995 Data. Chicago, Ill: American Hospital Association; 1998. [Google Scholar]
- 29.Covinsky KE, Justice AC, Rosenthal GE, Palmer RM, Landefeld CS. Measuring prognosis and case-mix in hospitalized elders: the importance of functional status. J Gen Intern Med. 1997;12:203–8. doi: 10.1046/j.1525-1497.1997.012004203.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 30.Rumsfeld JS, Whinney SM, McCarthy M, Jr, et al. Health-related quality of life as a predictor of mortality following coronary artery bypass graft surgery. JAMA. 1999;281:1298–303. doi: 10.1001/jama.281.14.1298. [DOI] [PubMed] [Google Scholar]
- 31.Gordon HS, Rosenthal GE. Impact of marital status on outcomes in hospitalized patients: evidence from an academic medical center. Arch Intern Med. 1995;155:2465–71. [PubMed] [Google Scholar]
- 32.Shepardson LB, Youngner S, Speroff T, Harper DL, Rosenthal GE. Increased risk of death in patients with do-not-resuscitate orders. Med Care. 1999;37:727–37. doi: 10.1097/00005650-199908000-00003. [DOI] [PubMed] [Google Scholar]
- 33.Daley J, Shwartz M. Developing risk-adjustment methods. In: Iezzoni LI, editor. Risk Adjustment for Measuring Health Care Outcomes. 2nd ed. Chicago, Ill: Health Administration Press; 1997. pp. 279–329. [Google Scholar]
- 34.Miller MG, Miller LS, Fireman B, Black SB. Variation in practice for discretionary admissions: impact on estimates of quality of hospital care. JAMA. 1994;271:1493–8. [PubMed] [Google Scholar]
- 35.Donabedian A. The role of outcomes in quality assessment and assurance. QRB Qual Rev Bull. 1992;18:356–60. doi: 10.1016/s0097-5990(16)30560-7. [DOI] [PubMed] [Google Scholar]
- 36.Dubois RW, Rogers WH, Moxley JH, III, Draper D, Brook RH. Special report: hospital inpatient mortality: is it a predictor of quality? N Engl J Med. 1987;317:1674–80. doi: 10.1056/NEJM198712243172626. [DOI] [PubMed] [Google Scholar]
- 37.Kahn KL, Rogers WH, Rubenstein LV, et al. Measuring quality of care with explicit process criteria before and after implementation of the DRG-based prospective payment system. JAMA. 1990;264:1969–73. [PubMed] [Google Scholar]
- 38.Hartz AJ, Gottlieb MS, Kuhn EM, Rimm AA. The relationship between adjusted hospital mortality and the results of peer review. Health Serv Res. 1993;27:765–77. [PMC free article] [PubMed] [Google Scholar]
- 39.Thomas JW, Holloway JJ, Guire KE. Validating risk-adjusted mortality as an indicator for quality of care. Inquiry. 1993;30:6–22. [PubMed] [Google Scholar]
- 40.Shinkman R. Outsourcing on the upswing. Health providers are farming out more services to spend less money. Mod Healthc. 2000;30:46–60. [PubMed] [Google Scholar]
- 41.Corder MP. Veteran's health care: time for a change? Physician Exec. 1998;24:48–51. [PubMed] [Google Scholar]