Effect of Critical Access Hospital Conversion on Patient Safety

Abstract

Background

The Medicare Rural Hospital Flexibility Program of the 1997 Balanced Budget Act allowed hospitals meeting certain criteria to convert to critical access hospitals (CAH) and changed their Medicare reimbursement mechanism from prospective payment system (PPS) to cost-based.

Objective

To examine the impact of CAH conversion on hospital patient safety.

Data Source

Secondary data on hospital patient safety indicators (PSIs), hospital CAH status, patient case-mix, and market variables, for 89 Iowa rural hospitals during 1997–2004.

Study Design

We employed quasi-experimental designs that use both control groups and pretests. The hospital-year was the unit of analysis. We used generalized estimating equations logit and random-effects Tobit models to assess the effects of CAH conversion on hospital patient safety. The models were adjusted for patient case-mix and market variables. Sensitivity analyses, which varied by sample and statistical model, were used to examine the robustness of our findings.

Data Extraction Methods

PSIs were computed from Iowa State Inpatient Databases (SIDs) using Agency for Healthcare Research and Quality indicators software. Hospital CAH status was extracted from Iowa Hospital Association. Patient case-mix variables were extracted from Iowa SIDs. Market variables came from Area Resource File (ARF).

Principal Findings

CAH conversion in Iowa rural hospitals was associated with better performance of risk-adjusted rates of iatrogenic pneumothorax, selected infections due to medical care, accidental puncture or laceration, and composite score of four PSIs, but had no significant impact on the observed rates of death in low-mortality diagnosis-related groups (DRGs), foreign body left during procedure, risk-adjusted rate of decubitus ulcer, or composite score of six PSIs.

Conclusion

CAH conversion is associated with enhanced performance of certain PSIs.

The Institute of Medicine (IOM) definition of patient safety is “freedom from accidental injury due to medical care, or medical errors” (Kohn, Corrigan, and Donaldson 1999). More than 44,000 Americans die as a result of preventable medical errors each year and national costs of preventable adverse events were approximately $17 billion in 1996 (Kohn, Corrigan, and Donaldson 1999).

Rural hospitals play a key role in providing health services in rural areas. Quality of care is one of the main issues that rural hospitals face. In improving quality of care, rural hospitals face several problems, including financial constraints (Moscovice, Gregg, and Klingner 2002), limited resources and knowledge (Moscovice 1989), and staffing (Moscovice and Stensland 2002). It was difficult for many of the smallest rural hospitals to recover their Medicare costs under the prospective payment system (PPS) rates (Moscovice and Stensland 2002; Dalton et al. 2003; Stensland, Davidson, and Moscovice 2004). In order to protect these small, financially vulnerable rural hospitals, the Medicare Rural Hospital Flexibility Program (Flex Program) of 1997 allowed some small hospitals to convert to critical access hospitals (CAH) and changed their Medicare reimbursement mechanism from prospective to cost based (Cameron, Zelman, and Stewart 2001). One objective of the Flex Program is to improve quality of care for those financially vulnerable rural hospitals.

The Rural Hospital Flexibility Program Tracking Team conducted a phone survey of 217 CAHs in 2001 and found that most CAHs were involved in a broad range of quality assurance (QA) or quality improvement (QI) activities. Almost all CAHs (99–100 percent) used continuing education programs for staff, medical error reporting policy systems, and hospital QA or QI training initiatives. Eighty-seven percent of CAHs reported the use of data feedback to staff and 38–52 percent of CAHs reported that they strengthened QA/QI activities after converting to CAH status. Only 1–2 percent of CAHs reported that these activities became weaker (Moscovice and Gregg 2001). In their second year survey, they found that all of these activities were stronger over time after CAH conversion (Moscovice, Gregg, and Klingner 2002). Compared with preconversion, 52 percent of hospitals increased pooling or coordinating resources; 31 percent of hospitals increased the number of available QA or QI staff; 19 percent of hospitals increased appropriateness of credentials; 35 percent of hospitals increased use of critical pathway protocols; 38 percent of hospitals increased use of admission protocols; and 43 percent of hospitals increase the use of transfer protocols. Less than 5 percent of hospitals decreased QA/QI-related characteristics (Moscovice and Gregg 2001). While these reports document an increase in QA and QI activities after conversion, no studies have been published to date that examine the effect of conversion on outcomes. The aim of our study is to examine indicators of patient safety over time in Iowa rural hospitals to explore changes that occurred after hospitals converted from PPS to CAH status.

METHODS

Sample and Data Sources

We employed quasi-experimental designs that use both control groups and pretests. Iowa has 96 nonfederal hospitals, which are located in non-MSA areas. These hospitals are classified as Rural Referral Hospitals, Rural PPS Hospitals, or CAHs. The seven Rural Referral Hospitals are located in non-MSA areas, but have similar operating characteristics to urban hospitals (Iowa Hospital Association [IHA] 2005). The other 89 rural hospitals (rural PPS hospitals and CAHs) are the sample of the study. All 89 rural hospitals were rural PPS in 1997. By the end of 2005, 81 hospitals had converted to CAHs and only eight hospitals remained as rural PPS hospitals. The study period from 1997 to 2004 includes 712 observations (hospital-years).

We developed a longitudinal database for 89 Iowa rural hospitals, spanning the 8-year period, which merged together: (1) patient safety indicators (PSIs) computed from Iowa State Inpatient Databases (SIDs) using Agency for Healthcare Research and Quality (AHRQ) PSIs software (AHRQ 2006a, AHRQ 2006c), (2) Hospital CAH status extracted from IHA records, (3) market variables extracted from the Area Resource File (ARF), and (4) other variables computed from Iowa SIDs. The definitions and sources of the variables are described in Table 1. Ordinary least squares regression techniques were used to impute temporal gaps in the ARF data (Alexander, D'Aunno, and Succi 1996; Little and Rubin 2002). Information on observed years in a county's time series was used to estimate data in missing years.

Table 1.

Sources of Variables and Means for 1997 and 2004

Variable	Definition	Data Sources	1997	2004
PSI-2	Observed rate of in-hospital deaths per 1,000 patients in DRGs with <0.5% mortality*, †	Iowa SID, AHRQ	0.46	0.95
PSI-3	Risk-adjusted cases of decubitus ulcer per 1,000 discharges with a length of stay >4 days	Iowa SID, AHRQ	6.90	11.12
PSI-5	Observed rate of foreign body accidentally left in during procedure per 1,000 medical and surgical discharges*,†	Iowa SID, AHRQ	0.043	0.073
PSI-6	Risk-adjusted rate of iatrogenic pneumothorax per 1,000 discharges*	Iowa SID, AHRQ	0.47	0.18
PSI-7	Risk-adjusted rate of selected infections due to medical care per 1,000 discharges*	Iowa SID, AHRQ	0.83	0.27
PSI-15	Risk-adjusted rate of accidental puncture or laceration during procedures per 1,000 discharges*	Iowa SID, AHRQ	2.4	1.8
Composite score of 4PSIs	Weighted sum of PSI-5, PSI-6, PSI-7, and PSI-15, weighted by numerator of each indicator	Iowa SID, AHRQ	1.93	1.34
Composite score of 6PSIs	Weighted sum of PSI-2, PSI-3, PSI-5, PSI-6, PSI-7, and PSI-15, weighted by numerator of each indicator	Iowa SID, AHRQ	4.74	5.92
CAH	CAH status=1, rural PPS status=0	IHA	0	0.74
CAHmv	The moving average of CAH status of the year t, t−1, and t−2 lags	IHA	0	0.62
% Medicare days	Medicare patient days/total patient days × 100%	Iowa SID	66.68%	65.71%
% Medicaid days	Medicaid patient days/total patient days × 100%	Iowa SID	6.72%	7.20%
% surgical discharges	Surgical discharges‡/total discharges × 100%	Iowa SID	12.56%	11.07%
Charlson comorbidity index	Sum of Charlson comorbidity scores for all inpatient patients/total discharges	Iowa SID	0.80	0.80
Market concentration (HHI)	Sum of the squares of each hospital's market share in the county	Iowa SID	0.89	0.89
Per capita income	County-level per capita income (using CPI to adjusted to 1997 dollar)	ARF	$21,180	$22,432
Population density	Number of population in the county/area of the county (square miles)	ARF	31.80	31.80

^*Please refer to Guide to Patient Safety Indicators 2003 Version 3.0 (AHRQ 2006a) for the detailed definition of each indicator.

^†No risk-adjusted rate is available for PSI-2 or PSI-5 in AHRQ PSI V3.0.

^‡Surgical discharges were identified based on the DRG variable in SIDs.

ARF, Area Resource File; SID, State Inpatient Database; IHA, Iowa Hospital Association; CAH, critical access hospitals; AHRQ, Agency for Healthcare Research and Quality; PSI, patient safety indicators; DRG, diagnosis-related group; HHI, Herfindahl–Hirschman Index.

Patient Safety Measurement

AHRQ provides software to compute quality indicators for hospitals using inpatient discharge data. The quality indicators include three modules: prevention quality indicators (PQIs), inpatient quality indicators (IQIs), and PSIs. PSIs screen for complications or adverse events “that patients experience as a result of exposure to the healthcare system and that are likely amenable to prevention by changes at the system or provider level” (AHRQ 2006a, AHRQ 2006c). Using PSI software, 29 provider-level PSIs can be computed. Each observed rate of PSI can be defined as “outcome of interest/population at risk.” The variations in observed rate of PSIs come not only from the quality of care of hospitals, but also from the different patient case mix in hospitals. Hospitals with more severe cases are more likely to have higher observed rates. Risk-adjusted PSIs are “the estimated performance of providers on the PSIs if those providers had an ‘average’ case mix” (AHRQ 2006a, AHRQ 2006c). The average case mix reflects the distribution in “age, sex, modified DRG, and co-morbidity categories” among the providers in the 35 states with SID databases. Risk-adjusted PSIs can be used for comparing patient safety among different providers and across different years. To eliminate unstable estimates based on too few cases, AHRQ recommends suppressing the estimates if fewer than 30 cases are in the denominator (AHRQ 2006a, AHRQ 2006c). Only five PSIs (PSI-5, foreign body left during procedure; PSI-6, iatrogenic pneumothorax; PSI-7, selected infections due to medical care; PSI-15, accidental puncture or laceration; and PSI-16, transfusion reaction), which use almost all medical and surgical discharges defined by specific diagnosis-related groups (DRGs) as denominators, are able to generate PSI measures for all rural Iowa hospitals without being subject to the small denominator issue. The observed rate of PSI-16 (transfusion reaction) is too rare to provide variability to differentiate hospitals in Iowa. Thus, we identify four PSIs (PSI-5, PSI-6, PSI-7, and PSI-15) as our measures of patient safety for all hospitals. In addition, to measure a broader scope of patient safety, we include two more PSIs (PSI-2, death in low-mortality DRGs and PSI-3, decubitus ulcer), which have fairly large denominators for most hospitals. Suitable PSI-2 measures are available for 630 observations among 64 hospitals. Suitable PSI-3 measures are available for 624 observations among 86 hospitals. AHRQ PSI software version 3 (AHRQ 2006a, AHRQ 2006c) was used to compute the risk-adjusted rates for PSI-3, PSI-6, PSI-7, and PSI-15, and observed rates1 for PSI-2 and PSI-5 for each hospital in our sample for each year from 1997 to 2004. We built two composite PSIs, “composite score of four PSIs” and “composite score of six PSIs.” Composite score of four PSIs, which is available for all 89 hospitals and 712 observations, is the weighted average of PSI-5, PSI-6, PSI-7, and PSI-15. Composite score of six PSIs, which is available for 594 observations among 82 hospitals, is the weighted average of PSI-2, PSI-3, PSI-5, PSI-6, PSI-7, and PSI-15. The weights are based on the frequency of the numerator of each PSI in our sample (8-year inpatient discharges among 89 Iowa rural hospitals). A numerator weight puts more weight on indicators with higher frequencies of adverse events. We also developed two composite PSIs weighted by the denominator of each indicator. We examine the impacts of CAH conversion on these two denominator-weighted composite PSIs in our sensitivity analyses. We also created a binary variable for each PSI. If the value of a PSI is higher than the median of the PSI in our sample (8-year 89 Iowa rural hospitals), the binary variable for the PSI (poor performance) is equal to 1; otherwise, it is equal to 0.

CAH Conversion Measurement

Hospital CAH status was extracted from IHA records. Two variables were used to measure hospital CAH status in the analyses. First, we examined the effects of CAH conversion using a dummy variable, where CAH was equal to one if the hospital was CAH status that year. Second we followed the approach recommended by Antel, Ohsfeldt, and Becker (1995) of using a moving average of CAH conversion. The moving average CAH variable was defined as the average of CAH status of the year t, t−1, and t−2 lags. The moving average CAH variable was used to examine the long-run effects of CAH conversion by putting less weight on hospitals during the first 2 years of conversion.2

Descriptive Analyses

In the descriptive analyses, we computed cross-sectional comparisons between rural PPS and CAH hospitals for PSIs by year. Given that the distributions for PSIs are skewed, a Wilcoxon rank-sum test (Pagano and Gauvreau 2000) was used to conduct cross-sectional comparisons of PSIs between PPS and CAH hospitals by year. We also conducted pre- and postconversion comparisons on PSIs for the 66 hospitals in CAH status in 2004. We calculated pre- and postconversion means and used Wilcoxon signed-rank test to test whether there were statistically significant improvements in PSIs for hospitals that converted to CAHs.

Generalized Estimating Equations (GEE) Logit Model

The value of PSIs is zero inflated and nonnegative, which suggests that ordinary least squares estimations may be problematic (Jones 2000). Thus, we employed a GEE logit model (Fitzmaurice, Laird, and Ware 2004) to estimate the effects of CAH conversion on the binary PSI variables (i.e., the association of CAH conversion with odds of poor performance for a specific PSI). The unit of analysis is hospital-year. The GEE logit model has the following three-part specifications (Fitzmaurice, Laird, and Ware 2004):

1. The conditional expectation of the response is assumed to depend on the independent variables through a logit link function.

   (1)

   where p_it is the expected probability of hospital i having poor performance for the PSI in year t. CAH_it is the value of the CAH variable for the ith hospital at year t. X_it is a vector of other explanatory variables that presumably are associated with patient safety for the ith hospital at year t. X_it includes the percentage of Medicare patient days, the percentage of Medicaid days, the hospital mean of the Charlson comorbidity score (Charlson et al. 1987; Deyo et al. 1992), the percentage of surgical discharges, market concentration (Herfindahl–Hirschman Index [HHI]), county-level per capita income, and county-level population density. Z_t is a vector of the year dummy variables, which will adjust the effects of unmeasured, time-specific factors. b₀, b₁, b₂, and b₃ are parameters associated with these variables.

2. The variation function is p_it(1–p_it).

3. We assume the within-subject association among repeated measures is a first-order autoregressive correlation pattern.

By using “sandwich” estimation, GEE yields consistent and valid estimates of coefficients and their standard errors even if within-subject associations are not correctly specified (Fitzmaurice, Laird, and Ware 2004). The dependent variables consist of eight binary quality functions with six individual PSIs and two composite PSIs. CAH conversion has been shown to be associated with significant changes in hospital characteristics, such as bed size and total number of discharges (e.g., Hagopian and Hart 2001; Rural Hospital Flexibility Program Tracking Team 2002, 2003). Thus, including hospital characteristics in multiple regression models may yield a multicollinearity issue with the CAH variables. Consequently, we only adjusted for patient characteristics and market variables in the multiple regression models.

Sensitivity Analyses

We undertook several sensitivity analyses to examine the robustness of our findings. First, we examined the impact of CAH conversion on the two denominator-weighted composite PSI scores. Second, one limitation of this study is the potential endogeneity of CAH variables given that CAH conversion is a decision made by rural hospitals themselves, rather than random assignment. In addition, some rural PPS hospitals may never choose to convert to CAH status. Using these hospitals as the comparison group may introduce some selection bias to the extent that these hospitals may be different from hospitals that convert to CAH. It is possible that the difference in PSI performance between rural PPS and CAH hospitals is attributable to heterogeneity in hospital characteristics rather than CAH conversion. In order to deal with this issue, we repeated the original GEE analyses based on a sample with 81 hospitals, which eventually converted to CAH. All of these hospitals were in rural PPS status in 1997 and in CAH status by the end of 2005. In addition, to conduct sensitivity analyses of hospital characteristics, we estimated GEE models, which included hospital bed size and number of discharges.

Sensitivity analyses also included panel data random-effects Tobit models to estimate the effects of CAH conversion on continuous PSIs (using Stata 7.0; see StataCorp. 2001b). The distributions of PSIs are skewed. The value of PSIs is nonnegative with a mass at 0. Tobit models have a conceptual advantage in analyzing data where the distribution of the dependent variable is normal above a limiting value (e.g., Woolridge 2002; Salkever, Slade, and Karakus 2006). The Tobit model assumes a continuous latent variable, denoted as PSI^*, with normal distribution (equations [2] and [3])

(2)

(3)

In this expression, we observe PSI_it for the ith hospital in year t. Equation (3) indicates that we observed PSI_it^* if it is larger or equal to 0. If PSI_it^* is smaller than 0, we observe 0. CAH_it, X_it, and Z_t are the same as we defined in the GEE logit model (equation [1]). The terms β₀, β₁, β₂, and β₃ are parameters associated with these variables. The error term is expressed as ɛ_it.

The random-effects Tobit models were estimated using Gauss–Hermite quadrature to compute the log likelihood and its derivatives (StataCorp. 2001a). We used the quadchk command in Stata 7.0 to check the numerical soundness of the quadrature approximation. The quadchk command enables examination of the extent to which changing the number of quadrature points affects the results. If the relative difference in coefficients is smaller than 0.01 percent, the results may be confidently interpreted. However, if the relative difference is >1 percent, the result is uncertain (StataCorp. 2001a). We ran random-effects Tobit models for both risk-adjusted rates of PSIs and observed rates of PSIs. One assumption for random-effects Tobit models is that there is no correlation between independent variables and omitted variables. This assumption may not be valid. By adding hospital IDs as dummy variables in cross-sectional Tobit models, the unconditional fixed-effects Tobit models reduce the omitted variable bias by ruling out the impact of hospital-specific omitted variables.3 All data analysis programs are available upon request from the corresponding author.

RESULTS

The mean values of PSIs for CAHs and rural PPS hospitals are shown in Table 2. Wilcoxon rank sum tests indicated that CAHs had statistically significantly (p≤.05) higher performance (i.e., lower PSI value) than rural PPS hospitals on PSI-6 in 2001, on PSI-2, PSI-7, PSI-15, and composite score of four PSIs in 2002, on PSI-3, PSI-6, PSI-7, PSI-15 and composite score of four PSIs in 2003, and on PSI-6, PSI-7, PSI-15 and composite score of four PSIs in 2004. Cross-sectional comparisons showed that CAHs generally had better performance in patient safety than rural PPS hospitals.

Table 2.

Cross-Sectional Comparison of Means of PSIs^‡ between Iowa Rural PPS and CAHs, 1997–2004

Year	Hospital Categories	Number of Hospitals	PSI-2†	PSI-3†	PSI-5	PSI-6	PSI-7	PSI-15	Composite Score of Four PSIs	Composite Score of Six PSIs†
1997	Rural PPS	89	0.46	6.9	0.04	0.47	0.83	2.44	1.93	4.74
1998	Rural PPS	89	1.53	9.18	0.05	0.27	0.36	2.72	2.04	5.76
1999	Rural PPS	88	0	10.93	0.02	0.3	0.69	2.99	2.29	6.9
	CAH	1	NA	NA	0	0	0	0	0	NA
2000	Rural PPS	78	0	9.31	0.06	0.36	0.6	3.09	2.35*	5.72
	CAH	11	0	22.94	0	0	0.41	2.12	1.58*	7.85
2001	Rural PPS	57	1.11	11.28	0.07	0.21**	0.69	2.65	2.03	6.9
	CAH	32	1.62	9.17	0	0.07**	1.17	2.24	1.8	5.61
2002	Rural PPS	45	0.93**	9.62	0.04	0.23	0.64**	2.00**	1.56**	5.95
	CAH	44	0.40**	8.31	0.21	0.34	0.14**	1.89**	1.41**	5.02
2003	Rural PPS	34	0.42	10.49**	0.01	0.46**	0.88**	2.38**	1.89**	6.38*
	CAH	55	0.85	7.38**	0.06	0.26**	0.41**	1.89**	1.45**	4.47*
2004	Rural PPS	23	1.04	10.98	0.12*	0.29**	0.54**	2.68**	2.04**	7.01
	CAH	66	0.92	11.19	0.06*	0.14**	0.17**	1.46**	1.09**	5.41

^*Significant difference in PSIs between rural PPS and CAH at p≤.10 (Wilcoxon rank-sum test).

^**Significant difference in PSIs between rural PPS and CAH at p≤.05 (Wilcoxon rank-sum test).

^†Means are based on those hospitals with nonmissing values.

^‡Lower PSI value is better performance.

PSI, patient safety indicators; CAH, critical access hospitals; PPS, prospective payment system.

In the pre- and postconversion comparisons, as shown on Table 3, after conversion to CAH status, hospitals experienced statistically significant improvement in performance as measured by PSI-7, PSI-15, and composite score of the four PSIs. There were no significant changes in hospital performance in PSI-2, PSI-3, PSI-5, and PSI-6. In general, following conversion, there are more hospitals exhibiting better performance than those exhibiting worse performance in all PSIs.

Table 3.

Changes in PSIs^* after Converting to CAHs for Iowa Rural Hospitals, 1997–2004

	PSI-2	PSI-3	PSI-5	PSI-6	PSI-7	PSI-15	Composite Score of Four PSIs	Composite Score of Six PSIs
N	59	57	66	66	66	66	66	54
Mean value of change in PSI (after–before)	0.187	−0.005	0.030	−0.090	−0.250	−0.800	−0.623	−0.310
Number of hospitals having better performance in PSIs after conversion (reducing PSI value)	15	32	8	18	29	37	41	30
Number of hospitals having worse performance in PSIs after conversion (increasing PSI value)	11	22	5	8	8	16	16	24
p-value for Wilcoxon signed-rank test	.68	.60	.64	.34	.01	<.01	<.01	.41

^*Lower PSI value is better performance.

PSI, patient safety indicators.

The descriptive comparisons in Tables 2 and 3 do not control for the effects of independent variables, selection bias, or history bias. To address these issues, GEE logit models were examined. The GEE logit models showed that CAH conversion yielded significant improvement in hospital performance (or lower odds of poor performance) in PSI-6, PSI-7, PSI-15, and composite score of the four PSIs (Table 4). The odds ratios of poor performance in CAH hospitals compared with rural PPS hospitals are 0.30 (confidence interval [CI]: 0.14–0.64) for PSI-6, 0.29 (CI: 0.15–0.56) for PSI-7, 0.40 (CI: 0.24–0.67) for PSI-15, and 0.49 (CI: 0.31–0.80) for composite score of four PSIs. We also observed higher scale improvement for CAHmv in PSI-6, PSI-7, PSI-15, and composite score of four PSIs.2 CAH conversion did not yield significant changes in the performance in PSI-2, PSI-3, PSI-5, and composite score of six PSIs. GEE logit models also showed that a higher percentage of surgical discharges in a hospital was associated with a higher odds of poor performance in observed rates of PSI-5, and risk-adjusted rates of PSI-6, PSI-15, and the two composite PSI measures. Also, the hospital mean for the Charlson comorbidity score was positively associated with poor performance in risk-adjusted PSI-3, PSI-5, PSI-15, and two composite PSIs. These results indicate that hospitals with more severe cases might have worse performance in some risk-adjusted PSIs.

Table 4.

GEE Logit Models of the Performance of PSIs (1=Poor Performance for the PSI) in Iowa Rural Hospitals, 1997–2004

	PSI-2‡	PSI-3	PSI-5	PSI-6	PSI-7	PSI-15	Composite Score of Four PSIs	Composite Score of Six PSIs
CAH	−0.54	−0.24	−0.80	−1.19**	−1.26**	−0.92**	−0.70**	−0.30
% Medicare days	0.09**	0.00	−0.01	−0.01	−0.01*	−0.01	−0.01	0.00
% Medicaid days	0.09**	0.03	−0.01	0.02	0.03	0.02	0.03	0.04
% of surgical discharges	0.01	0.01	0.09**	0.05**	0.01	0.18**	0.16**	0.04**
Charlson Index	−2.60**	1.36*	2.01**	0.69	1.05	1.51**	1.37**	2.14**
Market concentration (HHI)	0.11	−0.42	−2.11	0.37	0.24	−0.21	−0.15	−0.44
Per capita income ($1,000)	0.09	0.16**	0.00	−0.03	0.09	0.06	0.07**	0.15
Population density	0.02**	0.00	−0.01	0.02	0.01	0.00	0.00	0.00
y1998		0.04	−0.16	−0.43	−0.52	−0.16	−0.49	0.05
y1999		0.30	−0.63	−0.29	−0.24	0.11	−0.23	0.53
y2000		0.34	−0.53	0.18	−0.41	0.23	−0.32	0.16
y2001		0.68**	0.07	0.19	0.33	0.66*	0.21	0.44
y2002		0.34	0.41	−0.34	−0.24	−0.08	−0.54	0.24
y2003		0.21	−0.21	0.53	0.21	0.27	−0.13	−0.02
y2004		0.16	0.84	−0.20	−0.38	0.10	−0.50	0.27
Year	0.09
Intercept	−185.30	−4.81**	−3.23	−2.67*	−3.73**	−3.87**	−3.54**	−5.76**
Observations	630	624	712	712	712	712	712	594

^†Unit of analysis is hospital-year. y1998–y2004 were dummy variables for year 1998–2004. The year of 1997 is the reference category.

^*Statistically significant at .10 level.

^**Statistically significant at .05 level.

^‡GEE logit models for PSI-2 using categorical year variables were not converged. Continuous year variable was used.

PSI, patient safety indicators; CAH, critical access hospitals; HHI, Herfindahl–Hirschman Index.

In our sensitivity analyses, we found that CAH conversion was associated with lower odds of poor performance in denominator-weighted composite score of four PSIs. There is no significant impact on denominator-weighted composite score of six PSIs. The GEE logit models using a sample excluding eight hospitals, which were in rural PPS status in 2005, showed a similar result to those shown in Table 4. Sensitivity analyses using random-effects Tobit models showed that CAH conversion led to significantly lower values in PSI-6, PSI-7, PSI-15, and composite score of four PSIs. Fixed-effects Tobit models showed that CAH conversion was associated with significantly lower values of PSI-7, PSI-15, and composite score of four PSIs. In general, estimates for the moving average measure of CAH status had a larger scale than CAH estimates. In addition, we estimated a simulation using a random-effects Tobit model to examine the magnitude of CAH conversion impact on both composite scores of PSIs. CAH conversion is associated with 60 and 16 percent reduction in composite score of four PSIs and composite score of six PSIs, respectively. As we discussed above, due to the issue of multicollinearity, hospital characteristic variables were not included in the GEE logit models in Table 4.

DISCUSSION

We show a strong and consistent effect of CAH status on PSIs. In the sample of Iowa rural hospitals, the cross-sectional comparisons showed that CAHs had better performance than rural PPS hospitals. The pre- and postconversion comparisons showed that postconversion hospitals had better performance in PSIs than preconversion hospitals. The results are robust to a number of appropriate estimation strategies, and hold up to rigorous sensitivity analyses. To address the concern that difference in PSIs might reflect primarily differences in patient mix, time trends, and differences in markets and environment, we added two additional analyses. First, we used risk-adjusted PSIs4 as the measures for the performance of hospital patient safety. Risk-adjusted rates reflect the performance of providers on the PSIs if those providers had an “average case mix” (AHRQ 2006a, AHRQ 2006c). Thus, it greatly reduces the effects of patient case mix on patient safety measures. Second, we used GEE logit models, random-effects Tobit models and other sensitivity analyses to control for the impact of patient case mix, market variables, and time trend. Owing to multicollinearity, hospital characteristics were not included in the original models.

A possible explanation for these findings could be that hospitals might change their coding behaviors in response to the change in Medicare reimbursement scheme; that is, observed lower rates of adverse events may be due to a change in hospital coding behavior when they converted from PPS to cost-based reimbursement. In general, under PPS, hospitals have more incentive to code severe cases, which are associated with higher reimbursement rates. If there is no change in patient case mix after conversion, a significant change in coding behavior after converting from PPS to cost-based would result in a less severe observed case mix in rural hospitals. To examine this possibility, we conducted separate analyses to examine the relationship between CAH conversion and hospital-level average number of patient diagnoses, average number of Elixhauser comorbidities (Elixhauser et al. 1998), and mean score of the Charlson comorbidity index. We found that there was no significant association between CAH conversion and these measures.

Another alternative explanation for the positive effects of CAH conversion on patient safety could be that it reflects a trend toward improvement in patient safety for all hospitals. However, according to AHRQ, PSI-2, PSI-3, PSI-5, PSI-6, PSI-7, and PSI-15 in the Nationwide Inpatient Sample (NIS) did not exhibit an overall trend toward improvement over the 1997–2003 time period (AHRQ 2006b). Likewise, our computations of the Iowa SIDs from 1997 to 2004 using AHRQ PSI software indicated that there was no significant overall improvement trend for PSI-2, PSI-5, PSI-6, and PSI-15 in Iowa's urban and rural referral hospitals. In addition, we controlled for time effects by including year dummy variables in the GEE and tobit models. The 8-year panel data design helps adjust for selection-maturation threats (Shadish, Cook, and Campbell 2002), regression to the mean biases5 (Antel, Ohsfeldt, and Becker 1995), and delay causation (Shadish, Cook, and Campbell 2002). These analyses consistently showed that CAH conversion led to improved performance of PSI-6, PSI-7, and PSI-15. In the sensitivity analyses, GEE logit models and Tobit models also showed that moving-average estimates of the CAH effect had a larger scale than the dichotomous CAH measure, which indicates that CAH conversion may lead to larger improvement in patient safety in the long run.

CAH conversion has been shown to significantly improve rural hospitals' financial condition. Previous studies have shown that preconversion rural hospitals faced serious financial pressure (Cameron, Zelman, and Stewart 2001; Stensland, Davidson, and Moscovice 2004) and that over half of the hospitals that converted to CAH in FY1999 or FY2000 were losing money before conversion (Stensland, Davidson, and Moscovice 2004). Stensland, Davidson, and Moscovice (2004) reported that hospitals that converted to CAH in FY1999 experienced an average increase in Medicare payment of 36 percent, around $500,000 in FY2000 inflation adjusted dollars after conversion, and that CAH conversion increased hospital profit margins by 2–4 percentage points.

Improved financial conditions and lower risk sharing associated with CAH conversion are likely to contribute to improvements in patient safety. After conversion, under cost-based reimbursement, risk sharing decreased substantially. Under PPS, the marginal costs associated with QI are not reimbursed, and the hospital has to bear all the cost incurred by the increased intensity and quality (Cutler 1995). Under cost-plus reimbursement, marginal costs associated with increased quality are fully reimbursed. Hospitals tend to have higher intensity and produce a higher level of quality under lower risk sharing (Hodgkin and McGuire 1994; Cutler 1995; Chalkley and Malcomson 2000).

To the extent that more resources are needed to bring about meaningful improvements in quality of care, patient safety is inextricably linked to the financial condition of hospitals (Encinosa and Bernard 2005). It is harder for hospitals with financial problems to make investments in patient safety improvements (e.g. error-reducing information technology system), or to attract or retain high-cost specialists. Likewise, hospitals with financial problems might cut nurse staffing which may adversely affect patient safety (Encinosa and Bernard 2005). Cutler (1995) found that the fiscal pressures from the PPS in Medicare in the 1980s were associated with higher mortality. Shen (2003) also has shown that financial pressure was associated with increased mortality rates after treatment of acute myocardial infarction. Encinosa and Bernard (2005) found that a within-hospital erosion of hospital operating margins was associated with an increased rate of adverse patient safety events.

The Rural Hospital Flexibility Program Tracking Team found that QI- or QA-related activities were widely undertaken in CAHs and have been reinforced over time since CAH conversion (Moscovice and Gregg 2001; Moscovice, Gregg, and Klingner 2002). Staffing improvement was one of the most significant factors contributing to progress in quality of care and increased reimbursement was cited as the reason for improved staffing (Moscovice, Gregg, and Klingner 2002). Consistent with these findings, and expanding them to outcomes, we found that CAHs strengthened their scores on PSIs after conversion, at the time that they would have been experiencing higher reimbursement.

Apart from the change in reimbursement method and the resulting financial relief, other factors may also contribute to better quality of care in CAHs. These factors may include the establishment of a network relationship with affiliated hospitals, improvement in case management and discharge planning, expansion in qualified QA and QI staff, and enhancement in equipment (Moscovice and Gregg 2001; Moscovice, Gregg, and Klingner 2002).

There are several limitations in our study. First, there are limitations inherent to administrative databases including the possibility of missing codes, coding errors, and variation in coding practices across hospitals (Simborg 1981; Hsia et al. 1988; Iezzoni 1997; Zhan and Miller 2003). Although we did not find a significant association between CAH conversion and various measures of patient severity, it is possible the lower rates of adverse events in CAH hospitals are simply due to a reduced incentive to code certain diagnoses. In this study, we are not able to completely rule out this possibility.

A second issue of concern is endogeneity and omitted variable bias. The CAH variable is likely to be endogenous given that hospitals choose to convert to CAH status. However, endogeneity is to a large extent mitigated through the fixed-effects panel models with year dummy variables, which capture the effects of unmeasured hospital-specific and time-specific factors (Woolridge 2002). Furthermore, the results of the sensitivity analysis using a sample of 81 Iowa rural hospitals (all of which were in rural PPS status in 1997 and in CAH status at the end of 2005) suggests that endogeneity is not an exceptionally large concern. The consistency between the sensitivity analyses and our main results indicate that our findings are robust.

A third limitation involves using AHRQ software to calculate PSIs and evaluate hospital patient safety performance. The AHRQ software is not able to fully identify all the preventable adverse events due to the limited clinical information available in administrative data (AHRQ 2006a). In addition, only six of the 29 PSIs had adequate cases to measure patient safety in our sample of rural hospitals. The six PSIs are less comprehensive than the complete package of PSIs created by AHRQ PSI software. Furthermore, the assumption for the Tobit models is that the distribution is a censored normal distribution. However, we were not able to test this assumption given that the censored data are unobservable (Duan 1983). We only include the results of Tobit models in the sensitivity analyses. The findings of Tobit models were consistent with GEE logit models. Finally, our sample only includes Iowa rural hospitals, thus, our results may not generalize to other states.

CONCLUSION

CAH conversion was associated with significantly increased performance of risk-adjusted rates of iatrogenic pneumothorax, selected infections due to medical care, accidental puncture or laceration, and a composite score of four PSIs. No significant effect was found for observed rates of death in low-mortality DRGs, foreign body left during procedure, risk-adjusted rate of decubitus ulcer, or composite score of six PSIs. GEE logit and Tobit models also showed that estimates using a moving average CAH indicator variable had a larger scale effect than a CAH dummy variable, indicating that CAH conversion may lead to larger improvement in patient safety in the long run.

We speculate that the most likely mechanism linking CAH conversion and improved quality performance is the change in payment mechanism from prospective to cost based. Enhancement of financial resources may have contributed directly to an expansion in qualified QA and QI staff and enhancement in patient safety infrastructure. Other mechanisms that may have contributed likely include the establishment of a network relationship with affiliated hospitals, improvement in case management and discharge planning, and enhancements in patient safety processes. Although the mechanisms involved are not yet clear, the Medicare Rural Hospital Flexibility Program of 1997 appears to have attained its objective to improve quality of care for previously financially vulnerable rural hospitals.

Supplementary material

The following supplementary material for this article is available:

Table S1:

Weights for Composite PSI Score.

Table S2:

Median for Continuous PSIs, Frequency and Percentage for Binary PSIs.

Table S3:

GEE Logit Models of the Performance of PSIs (1=Poor Performance, 0=Good Performance) in Iowa Rural Hospitals (Composite Scores Weighted by Denominators), 1997–2004.

Table S4:

Sensitivity Analyses of GEE Logit Models of PSIs.

Table S5:

Sensitivity Analyses of Tobit Models of PSIs.

Table S6:

Random-Effects Tobit Model Simulation of the Impact of CAH conversion on the Risk-Adjusted Composite PSIs.

Table S7:

Fixed-Effects Panel Data Models of CAH Conversion on Hospital Bed Size and Discharges.

Table S8:

Fixed-Effects Panel Data Models of CAH Conversion on Hospital Case Mix.

Table S9:

National Trends for PSI-2, PSI-3, PSI-5, PSI-6, PSI-7, and PSI-15.

Table S10:

Trend for PSI-2, PSI-3, PSI-5, PSI-6, PSI-7, and PSI-15 among other Iowa Hospitals (Urban Hospitals and Rural Referral Hospitals).

Table S11 to Table S17:

Quadchk Results for PSIs.

This material is available as part of the online article from: http://www.blackwell-synergy.com/doi/abs/10.1111/j.1475-6773.2007.00731.x (this link will take you to the article abstract).

Please note: Blackwell Publishing is not responsible for the content or functionality of any supplementary materials supplied by the authors. Any queries (other than missing material) should be directed to the corresponding author for the article.