Changes in economic outcomes before and after rural hospital closures in the United States: A difference‐in‐differences study

Abstract

Objective

The objective of this study is to assess changes in local economic outcomes before and after rural hospital closures.

Data Sources

Rural hospital closures from January 1, 2005, to December 31, 2018, were obtained from the Sheps Center for Health Services Research. Economic outcomes from this same period were obtained from the Bureau of Labor Statistics, Bureau of Economic Analysis, Quarterly Workforce Indicators, U.S. Federal Reserve Economic Data, RAND Corporation state statistics database, U.S. Social Security Administration, and U.S. Census Bureau.

Design

Difference‐in‐differences study of 2094 rural counties.

Data Collection/Extraction

The primary exposure was county‐level rural hospital closures. The primary outcomes were county‐level unemployment rates; employment‐population ratios; labor force participation‐population ratios; per capita income; total jobs; health care sector jobs; disability program participation‐population ratios; percent of the population with subprime credit scores; total filings for bankruptcies per 1000 population; and population size.

Principal Findings

A total of 104 rural counties experienced a hospital closure, compared to 1990 rural counties that did not. Rural hospital closures were associated with significant reductions in health care sector employment (−13.8%; 95% CI: −22%, −5.6%; p < 0.001), but not with changes in any other economic measure. For unemployment rates, employment‐population ratios, per capita income, disability program participation‐population ratios, and total jobs, we found evidence of adverse trends preceding hospital closures. Findings were robust to adjusting for county‐specific time trends, specifying exposure at the commuting zone‐level, and using alternate definitions of rurality to define sample counties.

Conclusion

With the exception of a decline in jobs within the health care sector, there was no association between rural hospital closures and county‐level economic outcomes. Instead, economic conditions were already declining in counties experiencing closures compared to those that did not.

What is known on this topic

• Rural hospital closures have accelerated since 2010 and are expected to continue in the pandemic era.

• Health care comprises over 10% of employment in rural areas.

• The extent to which recent hospital closures may impact local economies is not well understood.

What this study adds

• We found differential declines in health sector jobs, but no differences in overall employment, income, disability program participation, or household finances after closures in counties with versus without closures.

• Several economic outcomes were already worsening prior to hospital closures.

• These findings suggest the potential importance of adverse economic trends preceding rural hospital closures, and the need for more granular local economic data to track their determinants and consequences.

1. INTRODUCTION

Nearly 2000 hospitals in the United States deliver health care to rural communities and serve as important contributors to local economies. ¹ , ² Health care comprises over 10% of employment in rural areas and rural hospitals are often the sole employers for surrounding areas. ³ , ⁴ However, over 150 rural hospitals have closed since 2005 with concern that more than 300 rural hospitals are at risk of closure in the coming years due to declining occupancy rates, high fixed costs, and mounting market pressures. ¹ , ⁵ , ⁶ , ⁷ These trends have occurred in the context of decades of economic contraction and declining economic opportunities in rural areas. ⁸ The economic pressures introduced by the COVID‐19 pandemic have already begun to exacerbate the rural hospital closure crisis and its potential consequences for rural communities. ⁹ , ¹⁰

The extent to which recent hospital closures may impact local economies ‐ and the extent to which adverse economic trends may presage closures ‐ is not well understood. On one hand, rural hospital closures may worsen general unemployment and poverty by creating joblessness and discouraging other financial investments in rural communities. Consistent with this, prior work on rural hospital closures through 1999 showed short‐term, area‐level declines in per capita income and increases in unemployment. ¹¹ Preserving job security may be critical for population health in rural areas where adults are more likely to experience poverty and growing economic precarity. ¹² On the other hand, local reports from communities experiencing rural hospital closures suggest that existing economic decline and joblessness were important factors that drove hospitals to close in the first place. ¹³ In this way, rural hospital closures may be a symptom of economic decline, as well as a potential driver. Two recent working papers, each focusing on distinct sets of county‐level economic outcomes, find contrasting evidence around these questions. ¹⁴ , ¹⁵

In this national study, we deploy robust new difference‐in‐differences analytic methods to examine the evolution of county‐level economic outcomes before and after rural hospital closures from 2005 to 2018. Specifically, we estimated changes in a broad set of economic indicators ‐ area‐level per capita income, employment opportunities, measures of household financial security, and population ‐ in counties that experienced rural hospital closures compared to those that did not over the same period.

2. METHODS

2.1. Data

We obtained data on rural hospital closures between 2005 and 2018 from the Cecil G. Sheps Center for Health Services Research at the University of North Carolina‐Chapel Hill. ⁶ These data include acute care hospital closures that fit at least one of the following criteria: they were (1) located outside of metropolitan counties, (2) assigned to rural‐urban commuting area codes of 4 or greater, or (3) critical access hospitals. These data were linked to the county in which the hospital was located.

For consistency with the Sheps Center, we identified our sample of rural counties using core‐based statistical areas (CBSA) in which non‐metropolitan counties were designated as rural (CBSA codes 5 and 6). In sensitivity analyses described below, we also considered alternate definitions of rurality. We restricted the sample to all rural counties containing at least one operational rural hospital as of the baseline year (2005).

We evaluated 10 economic outcomes, all collected at the county‐year level: (1) unemployment rates; (2) labor force participation‐population ratios; (3) employment‐population ratios (which collectively reflect both employment and labor force participation); (4) per capita income, logged; (5) total number of jobs across all industries, logged; (6) total number of jobs in the health care industry, transformed using the inverse hyperbolic sine function, which allows for the inclusion of zeroes ¹⁶ ; (7) disability program participation‐population ratios; (8) percent of the overall population with subprime credit scores; (9) total filings for bankruptcies per 1000 population; and (10) total population, motivated by the fact that declining population has been posited to be a key driver of rural hospital closures. ¹⁷ , ¹⁸

The first four measures are available from the Bureau of Labor Statistics and the Bureau of Economic Analysis and represent actual data with no imputation performed. Data on the total number of jobs by industry, averaged across four quarters to obtain yearly measures, were obtained from the Quarterly Workforce Indicators. ¹⁹ These data are derived from the Longitudinal Employer‐Household Dynamics Database which covers over 95% of private sector jobs. While there is potential for error, it is likely small given the extent of population coverage. Data on disability program participation are based on the census of administrative records from the U.S. Social Security Administration, with the exception of 104 counties for which participation was imputed (for privacy purposes) using state‐specific means. ²⁰ , ²¹ These counties were more likely to be rural, but still represent a small proportion of the total number of counties in our sample. Subprime credit score data, which reflect the fourth quarter of each calendar year, were obtained from the U.S. Federal Reserve Economic Data. These data are based on a 5% longitudinal sample of the U.S. population. Bankruptcy filings data were obtained from the RAND Corporation state statistics database and correspond to the census of all observations with no imputation. ²² County population data were obtained from the U.S. Census Bureau. For employment, labor force, and disability participation measures, we used baseline population (Census year 2000) to calculate ratios, given potential changes in population size as a result of hospital closures. We could not do this for the subprime credit score or bankruptcy measures, as these data are only available as rates. Our final dataset was comprised of county‐year observations.

2.2. Main analyses

First, we summarized the number of rural hospital closures annually. Next, we mapped the location of rural counties that experienced a closure relative to control rural counties that did not experience a closure during the study period. Following prior work, ¹¹ we defined the level of exposure at the county‐level. Thus, control counties were comprised of all other non‐metropolitan counties that did not experience closure over the study period and had a hospital in the county in the baseline year (2005). We then summarized county‐level characteristics in the baseline year for both exposed and control counties.

We estimated the association between a rural hospital closure and changes in county‐level economic outcomes using a difference‐in‐differences approach. Specifically, we implemented an “event study” specification in which differences in outcomes between counties experiencing a closure versus those that did not were allowed to vary over time. We estimated both lead and lag terms in the evolution of outcomes across counties with closures and control counties. The coefficients on the lead terms allowed us to assess whether outcomes were differentially changing between the exposed and non‐exposed counties before the timing of a closure, which would represent a failure of the assumption of parallel trends (see Appendix S1, Statistical Model).

Because hospital closures occurred at different times during the study period, and because the consequences of hospital closures may vary over both time and place, we used the newly developed difference‐in‐differences estimator of de Chaisemartin and D'Haultfoelle to estimate regression models. ²³ This estimator is robust to well‐known biases in difference‐in‐differences estimates that can occur when exposures take place at different times and have heterogeneous effects. ²⁴ , ²⁵ In addition, this estimator is robust to biases that may occur in evaluating pre‐existing trends when using standard regression approaches (see Appendix S1, Statistical Model). ²⁶ For all models, we also report single coefficients that summarize the average difference‐in‐difference estimate across all post‐closure time periods. For rural counties with more than one closure, the first closure was used as the primary event of interest.

To address potential spillover effects of hospital closures across contiguous counties, we additionally estimated models in which the exposure was specified at the commuting zone level. Commuting zones are collections of counties that define local labor markets based on cluster analyses of commuting patterns performed in 2000 by the Economic Research Service at the U.S. Department of Agriculture. ²⁷ In these models, all counties within a given commuting zone were considered “exposed” if a rural hospital closure occurred in any county within that commuting zone.

We also estimated specifications that accounted for any pre‐closure differential trends by adjusting for county‐level linear time trends. Standard errors were clustered at the commuting zone‐level, to account for spatial and temporal correlation in the main outcomes within labor market areas. Statistical significance was determined at the level of p < 0.05. All analyses were conducted using Stata 16 (StataCorp) and performed from August 1, 2020, to August 15, 2021. Due to the Common Rule and the exclusive use of publicly available data, this study was exempt from review by an institutional review board.

2.3. Additional sensitivity analyses

We conducted several additional sensitivity analyses. First, we present results using standard (two‐way fixed effects) ordinary least‐square regressions. Second, given the heterogeneity of rural communities and the limitations of the CBSA definition, ²⁸ we defined rural counties using a secondary measure known as the Index of Relative Rurality (IRR). ²⁹ This continuous index ranging from 0 (least rural) to 1 (most rural) represents a composite of county population size, population density, geographic remoteness estimated by network distances, and the fraction of urban area out of the total land area. We reanalyzed our main models using all counties above the sample median of IRR in the baseline year. Third, because rural hospital closures in the Sheps Center database comprise both “converted closures” (where hospital services were scaled down or repurposed to other services) as well as “completed closures” (where all services cease), we estimated models including only the latter type of closure. Fourth, we estimated our main models including base year population weights, as differences between unweighted and weighted models can serve as a check for model misspecification. ³⁰ Fifth, to address potential concerns regarding limited power to detect meaningful effects for any individual economic outcome, we estimated our main models using a single composite index of the 10 economic indicators as the outcome. ³¹

3. RESULTS

Among the total 141 hospital closures that occurred between 2005 and 2018, we excluded 36 that occurred in metropolitan counties. Among the 105 closures that occurred in rural counties, closures occurred every year of the study period and were distributed primarily in the Southeastern, Southwestern, and Midwestern states (Figure 1).

FIGURE 1.

Trends in rural hospital closures and their distribution. Panel A plots the cumulative number of rural hospitals that closed in the study sample during the period 2005–2018. Panel B displays the spatial distribution of rural hospital closures among rural counties (defined using core‐based statistical areas [CBSA] with non‐metropolitan counties designated as rural, i.e., CBSA codes 5 and 6) [Color figure can be viewed at wileyonlinelibrary.com]

The study sample included the 104 exposed rural counties (since one county experienced two closures during the study period) and 1990 unexposed rural counties, yielding a total of 2094 counties situated across 665 commuting zones. At the beginning of the study period (2005), counties experiencing a rural hospital closure had larger average population sizes (33,827 vs. 23,705, Table 1) and greater numbers of total and health care specific jobs (19,118 vs. 16,599 and 2656 vs. 2249, respectively) compared to counties that did not. Unemployment rates, employment‐population ratios, labor force participation‐population ratios, disability program participation‐population ratios, and consumer finance outcomes (subprime credit scores and bankruptcy filings per capita) were similar across both groups of counties.

TABLE 1.

Baseline characteristics of study counties

	Closure N = 104 counties	No closure N = 1990 counties
Unemployment rate (%)	6.0 (1.8)	5.8 (1.8)
Employment‐population ratio	0.45 (0.06)	0.46 (0.07)
Labor force participation‐population ratio	0.47 (0.57)	0.49 (0.07)
Per capita income (USD)	26,269 (3362)	27,527 (5741)
Total jobs (all industries)	19,118 (13,773)	16,599 (13,472)
Total health care jobs	2656 (1809)	2249 (2027)
Disability program participation‐population ratio	0.36 (0.13)	0.31 (0.12)
Subprime credit scores (%)	37.8 (8.6)	34.7 (9.1)
Total bankruptcy filings (per 1000)	6.1 (2.6)	5.8 (2.5)
Population	33,827 (30,290)	23,705 (23,491)

Note: Mean (standard deviation) of sample county characteristics for the baseline year of 2005. All means are weighted by working‐age (18–65‐year‐old) population, with the exception of total (i.e., all‐age) population.

For the outcomes of unemployment rate, employment‐population ratio, labor force participation‐population ratio, total per capita income, total number of jobs, and disability program participation‐population ratio, the difference‐in‐differences event study models revealed differential pre‐closure trends between counties experiencing hospital closures versus those that did not (Figure 2). The trends reflect a differential worsening of economic outcomes in counties experiencing a hospital closure, prior to the time a closure took place. For the majority of the primary outcomes, the point estimates revealed no statistically significant differences in changes in economic outcomes after hospital closures across counties experiencing rural hospital closures versus those that did not (Figure 2 and Table 2). In some cases, the 95% confidence intervals (CI) did include substantively meaningful declines in economic outcomes. For example, the lower bound of the confidence interval for per capita income included a 2.3% decline post‐closures and the upper bound for bankruptcy filing rates included a 13.8% increase.

FIGURE 2.

Event study plots of the association between rural hospital closures and economic outcomes. Each panel presents estimates from a separate difference‐in‐differences event study specification. Each dot plots the coefficient estimate and the vertical bars denote the 95% CI, which adjusts for clustering at the commuting zone‐level. The x‐axis refers to time (in years) relative to hospital closure and the y‐axis is the value of the difference‐in‐difference coefficient. The reference year for the event study specification is time = −1, the year before hospital closure. The dependent variable is specified in the panel headers. All models were estimated using the approach of de Chaisemartin and D'Haultfoeuille (2020) and account for time invariant county‐specific factors and secular time trends. For the health care jobs variable, IHS refers to the inverse hyperbolic sine transform of the variable. Sample sizes for each model can be found in Table 1. Coefficients displayed on the y‐axes can be interpreted as follows: for unemployment rates and subprime credit scores (variables that can range from 0 to 100), the estimates reflect the percentage point change. For the employment‐population ratio (which can range from 0 to 1), the estimates reflect the change in the ratio. For per capita income, total jobs and health care jobs, total bankruptcy filings, and population multiplying the coefficient estimates by 100 yields the percent change in the outcome [Color figure can be viewed at wileyonlinelibrary.com]

TABLE 2.

Difference‐in‐differences estimates of the association between rural hospital closures and economic outcomes

	County‐level exposure	CZ‐level exposure	N, county‐years
Unemployment rate (%)	0.038 [−0.36, 0.44], p = 0.85	−0.0093 [−0.410, 0.391], p = 0.96	27,945
Employment‐pop. ratio	−0.0036 [−0.011, 0.004], p = 0.33	−0.0004 [−0.104, 0.010], p = 0.94	27,945
Labor force participation‐pop. ratio	−0.0039 [−0.012, 0.005], p = 0.38	−0.0007 [−0.007, 0.006], p = 0.83	27,945
Per capita income (logged)	−0.0064 [−0.023, 0.011], p = 0.47	−0.0056 [−0.18, 0.007], p = 0.37	27,941
Total jobs (logged)	−0.027 [−0.056, 0.003], p = 0.075	−0.011 [−0.043, 0.021], p = 0.49	27,874
Total health care jobs (IHS)	−0.138 [−0.22, −0.056], p < 0.001	−0.056 [−0.101, −0.012], p = 0.012	27,632
Disability participation‐pop. ratio	−0.0003 [−0.0009, 0.0003], p = 0.32	0.00005 [−0.0007, 0.0008], p = 0.89	27,539
Subprime credit scores (%)	0.123 [−0.307, 0.557], p = 0.58	0.018 [−0.306, 0.342], p = 0.91	27,317
Total bankruptcy filings (per 1000, logged)	0.051 [−0.036, 0.138], p = 0.25	0.047 [−0.013, 0.107], p = 0.12	27,468
Population (logged)	0.0006 [−0.014, 0.016], p = 0.94	−0.0008 [−0.012, 0.010], p = 0.89	27,921

Note: Each cell contains estimates from a separate difference‐in‐differences model. The dependent variable is specified in the row headers. All models were estimated using the approach of de Chaisemartin and D'Haultfoeuille (2020) and account for time invariant county‐specific factors and secular time trends. Estimates from four different specifications (column header) are presented here. “County‐level exposure” defines exposed counties as those that had a hospital closure occur within them; “CZ‐level exposure” defines expose counties as those residing in a commuting zone (CZ) in which a hospital closure occurred. 95% CI, which accounts for clustering at the CZ‐level, is provided in square brackets. Event study specification plots for the CZ‐level exposure estimates are presented in Figure S1 of the Supplement. For the health care jobs variable, IHS refers to the inverse hyperbolic sine transform of the variable. Coefficients can be interpreted as follows. For unemployment rates and subprime credit scores (variables that can range from 0 to 100), the estimates reflect the percentage point change (e.g., in the first column, hospital closure was associated with a 0.038 percentage point change in unemployment). For the employment‐population ratio (which can range from 0 to 1), the estimates reflect the change in the ratio (e.g., hospital closures were associated with a −0.0036 unit change in the employment‐population ratio). For per capita income, total jobs and health care jobs, total bankruptcy filings, and population, the estimates should be multiplied by 100 to recover the percent change in the outcome (e.g., hospital closures were associated with a 13.8% decline in health care jobs, b = −0.138*100).

We did find statistically significant point estimates for total health care sector jobs, with the difference‐in‐difference estimates suggesting a 13.8% decrease (95% CI −22.0%, −5.6%; p < 0.001) in health care jobs after hospital closures in counties experiencing a closure versus those that did not over the follow‐up period (Table 2, first column). There were no differential changes in population between counties experiencing rural hospital closure compared to those that did not (0.06% relative increase, 95% CI −1.4%, 1.6%; p = 0.094).

In sensitivity analyses, we found substantively similar results when specifying the exposure of rural hospital closure at the commuting zone‐level (Table 2, second column; Figure S1), though the magnitude of the estimates on the outcome of health care sector jobs was roughly half the size as the estimate obtained from models using a county‐level exposure (5.6% relative decrease vs. 13.8% relative decrease, respectively). Estimates from models that additionally included county‐specific linear time trends (Table S1) were not substantively different from those from the main models. Models using standard two‐way fixed effects regressions found larger and statistically significant estimates on several economic measures, though these became substantively and statistically insignificant once differential pre‐existing trends were accounted for (Figure S2 and Table S2). Estimates were largely unchanged when using an alternate approach to define rural counties (Figure S3). A similar pattern was also found in models excluding converted closures (Figure S4), though statistically significant relative increases were found for the outcome of total bankruptcies per capita (16.3% relative increase, 95% CI: 6.2%, 26.4%, p < 0.001) and the estimates for post‐closure relative changes in the number of health care jobs was larger in magnitude than in the main specification (−17.8% relative decrease, 95% CI: −29.9%, −5.8%, p < 0.001). Substantive estimates were also unchanged when using population weights (Figure S5). Finally, findings were unchanged when using a single index of the 10 economic indicators as the outcome of interest with estimates showing adverse pre‐closure trends (Figure S6).

4. DISCUSSION

In this national study of changes in economic outcomes after rural hospital closures, we found significant differential declines in health care sector jobs, but no relative differences in overall employment outcomes, per capita income, disability program participation, or household financial circumstances after rural hospital closures in U.S. counties experiencing closures compared to those that did not. We also found evidence that several economic outcomes were already worsening prior to hospital closures. These findings were robust to different definitions of exposure, rurality, and changes in regression specifications.

Our findings, which are consistent with a recent working paper that examined an overlapping, yet distinct, set of consumer finance and labor market outcomes, inform debates around rural hospital closures in three ways. ¹⁴ First, the lack of association between rural hospital closures and post‐closure economic outcomes beyond health care sector employment suggests either (1) that rural workers may have been able to find employment in other sectors without compromising household income or financial health, or (2) that the adverse economic impacts of closure occur more locally than at the county level. The second possibility is supported by the larger estimates found on the reduction in health care sector jobs when using the more local county‐level exposure compared to the broader commuting zone‐level exposure. Differentiating between these distinct possibilities will require higher‐resolution (i.e., sub‐county‐level) data on local economies moving forward.

Second, the finding that economic decline precedes rural hospital closures suggests that previously hypothesized determinants of closures – for example, declining occupancy and worsening hospital finances – may themselves result from broader “upstream” economic drivers. ⁷ , ³² , ³³ These factors may include declining economic opportunity, loss of employment in other, larger, sectors of the economy, or the flight of investors and loss of other sources of community capital. If so, efforts to reduce the risk of hospital closures may require a broader focus on local communities and economies in order to be successful. ³⁴ For example, existing rural economic development efforts, which include state tax credits to encourage industries to enter rural markets or place‐based federal investments (e.g., “Empowerment Zones”), may play an important and complementary role in reducing the risk of rural hospital closures. ³⁵ , ³⁶

Third, our findings have implications for future studies examining the effects of hospital closures on health outcomes. Pre‐existing economic declines in areas with hospital closures may confound the relationship between closures and health, as economic decline itself is a social determinant of health. ³⁷ , ³⁸ , ³⁹ Attributing health or other consequences causally to rural hospital closures will be difficult in the absence of a more comprehensive understanding of the upstream drivers of closure.

This study has limitations. First, our definition of rural communities was limited to that of the county and commuting zone surrounding a rural hospital. Though we tested alternate definitions of “rural” at the county‐level, the use of this geographic unit may reduce the available statistical power to detect adverse economic consequences that may occur more locally, a concern which is potentially exacerbated by the relatively small number of closure events during the study period. Consistent with this concern, we note that some of the confidence intervals for the difference‐in‐difference estimates do include economically meaningful negative effects. For example, the lower bound of the confidence interval for per capita income included a 2.3% decline post‐closures and the upper bound for bankruptcy filing rates included a 13.8% increase. Second, beyond economic consequences, rural hospital closures may result in broader population health risks related to limited access to inpatient and outpatient care. ³² , ⁴⁰ , ⁴¹ , ⁴² Assessment of these outcomes was outside the scope of this study, and our finding of pre‐existing differential trends in economic outcomes challenges the use of difference‐in‐differences and other research designs to obtain unbiased estimates of these health effects. Third, despite our quasi‐experimental study design, there is still potential for omitted variable bias or confounding. Fourth, while we focused on a broader set of economic outcomes than in prior work, there are other economic outcomes, such as household debt or food and housing insecurity that were not captured in this study.

Finally, while our results recover estimates on average for rural hospital closures nationally, prior evidence suggests that the causes and consequences of rural hospital closures are likely to be local and heterogeneous. While some rural closures are thought to be due to differential declines in population and growing outmigration, ¹³ our findings suggest that this is not the case on average, though perhaps may apply to specific localities. This is further supported by recent trends suggesting a potential reversal in the pattern of rural population decline as of 2016. ⁴³ There is also variation in how susceptible some rural hospitals might be to financial stressors related to government reimbursement or physician shortages, which are thought to be important drivers of closures. ⁴⁴ , ⁴⁵ , ⁴⁶ , ⁴⁷ Finally, rural communities are demographically and socioeconomically heterogeneous ⁴⁸ such that economic circumstances may have driven hospital closures in some areas more strongly than others may. All of these factors bolster the call for more local, granular data collection to better characterize the drivers and downstream impact of rural hospital closures, and how they may vary across different areas.

Nonetheless, this study provides new evidence on the economic experiences of rural communities before and after hospital closures. We believe that the findings are relevant for researchers and policy makers seeking to stem the tide of rural hospital closures and support rural communities more broadly.

5. CONCLUSION

With the exception of a decline in health care sector jobs, there was no association between rural hospital closures and county‐level economic outcomes. Instead, economic conditions were already declining in rural counties experiencing hospital closures compared to those that did not. These findings suggest the potential importance of adverse economic trends in portending the risk of hospital closure and the need for more granular local economic data to track their determinants and consequences.

Supporting information

Data S1.Supporting information.

Chatterjee P, Lin Y, Venkataramani AS. Changes in economic outcomes before and after rural hospital closures in the United States: A difference‐in‐differences study. Health Serv Res. 2022;57(5):1020‐1028. doi: 10.1111/1475-6773.13988