Defining Rurality in Hospice Research: Evaluation of Common Measures

Abstract

Inconsistency in identifying rural hospices has biased research findings and policy analysis. The purpose of this study was to conduct a comprehensive evaluation of eight rural-urban classifications against the gold standard of the Office of Budget and Management (OMB) to determine the utility of alternative measures in hospice research. These classifications included: Urban-Rural Classification Scheme for Counties (URCSC), Urban Influence Codes (UIC), Rural-Urban Continuum Code (RUCC), Federal Office of Rural Health Policy (FORHP), Index of Relative Rurality (IRR), the U.S. Census Bureau, Rural-Urban Commuting Area codes (RUCA), and Frontier and Remote (FAR). The last and the U.S. Census Bureau classified the smallest number of hospices; URCSC, UIC, and RUCC were indistinguishable from the OMB; and RUCA, IRR, and FORHP classified as rural the largest number of hospices. The latter three classifications also had good agreement with the OMB and therefore can be recommended for use instead of the OMB.

The growing crisis in rural health care leaves many rural patients to die in pain because they do not have access to end-of-life hospice care.^1,2 Hospices provide services that are crucial at the end of life, ranging from pain management to bereavement support of family members. A growing need for hospice care in rural areas is reflected in a two-fold increase of patients over the last 15 years; today hospices in the U.S. serve more than 1.55 million people. Despite the increase in demand for hospice services, the number of rural hospices in recent years has been declining by 3% per year, leaving rural patients severely underserved.^3,4 The problem is complicated by the fact that most of the hospices provide only home-based hospice care which in rural areas comes with high travel costs.^1,5 In addition, the demographic decline of the rural population, the labor intensity of hospice care, and staffing challenges, lead to high operational costs that are inadequately covered by health insurance plans.⁶

The development of policy interventions that would improve access to rural hospices can be complicated because many studies and policy reports have been based on definitions of rurality that do not match with one another and were untested in a hospice environment. The classification of rurality developed by the Office of Management and Budget (OMB) has been the gold standard which for more than 70 years provided the most consistent classification of rural areas, covering all U.S. counties.⁷ However, many governmental organizations that aim to improve the quality of health care in rural areas, such as the U.S. Department of Agriculture (USDA) and the Federal Office of Rural Health Policy (FORHP), have been using their own classifications.^8–10 They dramatically differ from each other, and have never been compared against the OMB’s classifications. This inconsistency in classification of rural areas has been biasing research findings and policy analysis, further complicating our understanding of challenges that rural hospices are facing.^8,11

Evaluating the performance of rural measures is timely and relevant in end-of-life research. The U.S. hospice response to the COVID-19 pandemic (e.g., telehospice) has highlighted issues of providing hospice care in rural areas. Although that conversation has been focused on telecommunication infrastructure and patient/family acceptability,^12,13 there has been another pressing problem encountered by end-of-life researchers: defining rurality. Researchers need an accurate measure of rurality for measuring access to health care, use of telehospice and telemedicine services, and so forth. ^12,13 Therefore, the purpose of this study was to conduct a comprehensive evaluation of measures of rurality against the OMB measure.

Background.

There are many measures of rurality that have largely gone untested in hospice and palliative care research. The OMB’s measure of rurality provided the federal statistical standard for identifying rural areas based on their proximity to core-based statistical areas (CBSAs).¹⁴ It designated counties as metropolitan, micropolitan, or non-core: a metropolitan area contained an urban core of 50,000 or more population; a micropolitan area contained an urban core of at least 10,000 (but less than 50,000) population; all other counties were considered as non-core or rural.¹⁴ This classification has been the gold standard for defining rural areas in health care research.^7,8 It has also had a number of advantages over other classifications: it is the oldest classification of rurality in use since the 1950s, it has covered all U.S. counties, and it has been widely used by many governmental and non-governmental U.S. agencies.^7,8,10 However, recent studies have identified several concerns with using the OMB measure. The OMB’s classification was not designed to distinguish rural and urban areas but to measure economic activity in a specific area.¹¹ Although indeed rural areas may have lower economic activity than urban areas, rurality is a multidimensional concept that must be based on multiple dimensions and not just one.¹⁵ OMB’s classification also lacked specificity for differentiating finer subcategories within metropolitan, micropolitan, and non-core areas, often pigeonholing dissimilar counties together.¹⁵ This results in undercounting of rural areas located in metropolitan counties.⁸ OMB’s classification also had rigid thresholds that were based exclusively on population density that have not been updated since they were created in the 1950s.^15,16 This has been especially problematic considering the significant changes in composition of rural and urban populations since then.¹⁶

Other measures of rurality have emerged for end-of-life researchers to use (see Table 1). Some of these measures have been based on CBSA’s distinctions provided by OMB and further split metropolitan, micropolitan, and non-core areas using their own criteria. For example, the U.S. Department of Agriculture has had a strong rural focus and has provided several classifications with smaller subcategories of non-core areas.¹⁶ The Rural-Urban Continuum Code (RUCC) distinguished rural areas based on their adjacency to metropolitan and micropolitan areas, whereas Urban Influence Codes (UIC) and Rural-Urban Commuting Area codes (RUCA) reflected differences in commuting patterns.^17,18 Urban-Rural Classification Scheme for Counties (URCSC) developed by the National Center of Health Statistics (NCHS)¹⁹ is different from any classification developed by USDA because it is primarily focused on metropolitan areas, delineating them depending on population density, and not distinguishing subclasses of non-core areas.¹⁰

Table 1.

Description of Nine Rural-Urban Classifications

Name		Developer	Based on	Unit	Purpose	Classification	Database
1.	OMB (gold standard)	OMB	CBSA	County	Statistical purposes	3 Classes: MSA; MiSA; non-core:	N = 1,882 FIPS with CBSAs
2.	Urban-Rural Classification Scheme for Counties (URCSC)	(NCHS)	CBSA	County	Study the associations between urbanization level of residence and health and to monitor the health of urban and rural residents.	6 classes: 4-MSAs, 1-MiSA, 1-non-core	N= 3,149 FIPS
3.	Urban influence codes (UIC)	USDA	CBSA	County	Differences in economic opportunities among rural areas.	12 classes: 2 MSAs; 3 MiSA; 7 noncore	N= 3,221 FIPS
4.	Rural-urban continuum codes (RUCC)	USDA	CBSA	County	Degree of urbanization and adjacency to a metro area	9 classes: 3 MSA, 6 urban, 2 rural	N= 3,221 FIPS
5.	FORHP rural areas	FORHP	Multiple measures	County	Establish eligibility for funding opportunities	2 classes: rural, and non-rural.	2,590 rural FIPS; 18,776 rural ZIPs
6.	Index of relative rurality (IRR)	Purdue University	Multiple measures	County	Measure county’s degree of rurality	A scale from 0 to 1	N= 3,141 FIPS
7.	Percent of rural and urban	U.S. Census Bureau	Census measures	County	Statistical purposes	3 classes: mostly urban; mostly rural, and rural	N = 3,146 FIPS
8.	The Rural-Urban Commuting Area codes (RUCA)	USDA	CBSA	ZIP	Commuting patterns	10 classes: 3 urban, 3 large rural, 3 small rural; 1 isolated rural.	N = 74,002 FIPS (11 digit) N = 41,901 ZIPs
9.	Frontier and remote (FAR)	USDA	Multiple measures	ZIP	Access to goods and services	4 levels	30,337 remote ZIPs

Note. OMB = Office of Management and Budget; USDA = United States Department of Agriculture; NCHS = National Center for Health Statistics; FORHP = Federal Office of Rural Health Policy

There are also four measures that were designed to directly measure rurality, not using OMB’s categories. Classification developed by the U.S. Census Bureau²⁰ has been based on the proportion of rural population in counties; a county is counted as rural if it has 50% or more of rural population and is urban otherwise. The Federal Office of Rural Health Policy (FORHP)²¹ classifies counties as rural if they have medical institutions eligible for rural development grants. The Index of Relative Rurality (IRR),¹⁵ developed by Purdue University, has a scale that ranges from 0- “completely urban” to 1- “completely rural,” with a score assigned to each county based on its geographic and population characteristics. For example, the IRR of New York County is 0.04 and the IRR of the Northwest Arctic Borough in Alaska is 0.89. The USDA’s Frontier and Remote (FAR)²² classification is based on measures of geographic isolation and assigns each county a level from 0- not isolated to 4- most isolated.

Despite the availability of multiple measures of rurality, there has been a lack of rigorous comparison between the performance of the OMB measure and alternative measures. The published reviews of rural measures in health care research have been predominantly focused on conceptual differences between measures of rurality, stressing the importance of context, available data, and the goal of the research. In this study we approach this problem from a slightly different perspective by providing an empirical comparison of rural classifications against the gold standard. We address three important questions: (a) What are the differences in proportion of hospices that classified as rural by the gold standard and alternative classifications? (b) How do alternative measures of rurality agree with the gold standard? and c) How well do alternative measures of rurality predict the rurality of hospices based on their characteristics and socio-demographic characteristics of communities where they are located in comparison with the gold standard?

Methods

Study design and data sources.

A retrospective non-experimental research design was used to analyze data from the nationally representative sample of hospices in the 2014 Medicare Provider Utilization and Payment Dataset (PUF). The dataset contained information about health services provided in 3,911 of U.S. Hospices and was collected by the Centers for Medicare and Medicaid Services (CMS)²³. Because the PUF dataset did not include information about the organizational characteristics of hospices, such as age and type of ownership, it was merged with the CMS’s 2014 Providers of Services Dataset (POS)²⁴ using the provider’s identification numbers.

Sample.

The rurality of each hospice in the sample was assessed using nine publicly available measures, including: OMB, URSCS, UIC, RUCC, FORHP, IRR, the U.S. Census Bureau, RUCA, and FAR. The first four measures were merged with the sample of hospices using county-level FIPS codes; the rest used postal ZIP codes. Because RUCA classification did not include ZIP codes from Puerto Rico, hospices from this territory (n = 37 or 0.92% of the sample) were eliminated; as a result, the final sample was comprised of 3,858 hospices. The total number of hospices eliminated in this study because of missing data was small, n = 53 or 1.35%, which is significantly less than 5%—a benchmark for missing data.²⁵

Measures.

A set of measures was created for this analysis that included rural-urban classifications, hospice characteristics, hospice services, and community characteristics. Each rural classification, including OMB, URSCS, UIC, RUCC, FORHP, the U.S. Census Bureau, and RUCA, was recoded into a binary class of rural or nonmetropolitan (1) versus urban or metropolitan (0). For FAR’s classification remote Levels 1 thru 4 were recoded as rural; Level 0 was coded as urban. For IRR classification we followed Inagami et al.²⁶ and considered a value of 0.4 as the threshold for rural areas; therefore, values larger than 0.4 were coded as rural, and urban otherwise. Organizational characteristics included: a) age, in years; b) type of ownership: for profit, non-profit, and governmental; and c) daily census. Hospice services were estimated by measuring the duration of hospice visits of three types of hospice workers providing service: home health aide, skilled nurse, and social worker. Community characteristics for each county where hospices were located were estimated using data from the American Community Survey²⁷ including: a) proportion of female residents; b) proportion of non-Hispanic Whites; c) mean age; d) median annual household income, and e) location: Northeast, Midwest, Southern, and Western United States.

Data analysis.

Descriptive statistics were used to describe the sample of hospices with frequencies, means, and standard deviations. The agreement of rural classifications with the gold standard was estimated using three measures: sensitivity index, positive predictive value (PPV), and kappa statistic.²⁸ Agreement was measured using Cohen’s kappa statistics,²⁹ which was converted in percent scale to be consistent with the scales used in specificity and PPV measures. Values of kappa statistic below 20% indicated slight agreement; 21% and 40%—fair; 41% to 60%—moderate; 61 to 80%—substantial; 81% to 100%—almost perfect.²⁹

The performance of rural-urban classifications was assessed using Pearson’s chi-square goodness of fit statistic for logistic regression models.³⁰ Hospice characteristics were used to predict rural (1) versus urban (0), location in OMB (Model 1), RUCA (Model 2), FORHP (Model 3), IRR (Model 4), FAR (Model 5), and U.S. Census (Model 6) classifications.

Results

Descriptive statistics.

The descriptive statistics presented in Table 2 show that both FORHP and IRR classifications classified the largest number of hospices as rural (55.11% and 44.04% respectively); RUCA classified as such only a third of hospices (28.95%); four classifications, including OMB, URCSC, UIC and RUCC identified identical numbers of rural hospices—23.46%. The smallest number of rural hospices was identified by the U.S. Census (16.59%) and FAR (9.15%). The mean age of hospices in the sample was around 14 years (SD = 9.02). A third of the hospices (36.60%) were for-profit, half of all hospices were non-profit (50.73%), and the rest were owned by the government (12.67%). The average daily census in hospices was 64.40, the mean duration of hospice services provided by home health aides or skilled nurses was on average slightly less than 20 minutes per day (19.12 and 17.35 minutes respectively), and services provided by social workers were much shorter: 2.6 minutes per day. The mean age of residents of counties where hospices were located was 37 years; half of them were women (50.77%), two thirds (75.60%) were non-Hispanic Whites, and mean annual household income was around $52,000. Hospices were predominantly located in the South (40.57%), the West (24.13%), and the Midwest (23.64%), while the number of hospices from the Northeast was small (11.66%).

Table 2.

Sample Characteristics (n = 3,858)

	Statistics		Min	Max
Measures of ruralitya
OMB, n (%)	905	23.46	0	1
URCSC, n (%)	905	23.46	0	1
UIC, n (%)	905	23.46	0	1
RUCC, n (%)	905	23.46	0	1
FORHP, n (%)	2,126	55.11	0	1
IRR, n (%)	1,699	44.04	0	1
U.S. Census, n (%)	640	16.59	0	1
RUCA, n (%)	1,111	28.95	0	1
FAR, n (%)	353	9.15	0	1
Hospice characteristics
Age in years, M (SD)	13.66	9.02	0.17	31.17
Ownership, n (%)
For-profit	1,412	36.6	0	1
Non-profit	1,957	50.73	0	1
Government	489	12.67	0	1
Daily censusb, M (SD)	64.40	128.50	0.22	5,534.17
Duration of hospice visitsc, M (SD)
Home health aide	19.12	13.93	0.00	159.24
Skilled nurse	17.35	10.25	0.00	235.16
Social worker	2.62	1.53	0.00	19.61
Community characteristics, M (SD)
Percent of women	50.77	1.26	36.68	59.15
Percent of non-Hispanic Whites	75.60	16.08	13.73	98.93
Age	37.30	4.37	21.60	55.40
Income, in thousands of US dollars	51.53	12.80	22.60	110.93
Regions of the U.S., n (%)
Northeast	450	11.66	0	1
Midwest	912	23.64	0	1
South	1,565	40.57	0	1
West	931	24.13	0	1

Note.

^aproportion of rural population

^bmeasured in days of service per year

^cin minutes per day

Agreement.

The results of the agreement analysis are presented in Figure 1. Four measures of rurality, namely URSCS, UIC, and RUCC had sensitivity, PPV, and kappa statistic values of 100%. Agreement of the other measures varied. According to kappa statistic, RUCA had almost perfect agreement with the OMB (83.7%), whereas FORHP, IRR, and U.S. Census had moderate agreement (41.6%, 55.4%, and 59.1%, respectively), and FAR had only fair agreement (39.6%). The RUCA, FORHP, and IRR also demonstrated almost perfect sensitivity with the OMB with the values above 99%; sensitivity of the U.S. Census was much lower—57.8%, while the FAR was only 33.1%. Positive predictive value was the highest for U.S. Census, FAR, and RUCA classifications (81.3%, 87.2%, and 80.3% respectively), and moderate for FORHP and IRR (44.8% and 53.9%, respectively).

Figure 1. Comparison of alternative methods of rural-urban classification of hospices against the gold standard (n= 3,858).

This figure shows the agreement of eight alternative measures of rurality with the gold standard of the Office of Budget and Management’s (OMB). The inner hexagon corresponds agreement of 10% and the outer–100%. The agreement was measured using kappa statistic, sensitivity, and PPV measures. Eight alternative classifications of rurality included: Urban-Rural Classification Scheme for Counties (URCSC), Urban Influence Codes (UIC), Rural-Urban Continuum Code (RUCC), Federal Office of Rural Health Policy (FORHP), Index of Relative Rurality (IRR), the U.S. Census Bureau, Rural-Urban Commuting Area codes (RUCA), and Frontier and Remote (FAR). Agreements of URCSC and UIC classifications with OMB were identical to ones of RUCC and therefore were not presented. Example: RUCC had a 100% agreement with OMB for all three measures of agreement, including kappa statistic, sensitivity, and PPV. In contrast, FAR classification had high agreement with OMB, measured by PPV (87.2%), but low agreement measured by sensitivity and kappa statistics (33.1% and 39.6% respectively).

Regression analysis.

The results of regression analysis are presented in Table 3. Overall, in predicting the rurality of hospices, Model 4 demonstrated the best model fit with χ²(14) = 2,540.47, p < .001, whereas Model 5 had the smallest model fit with χ²(14) = 477.34, p < .001. Overall, most predictors across models had similar directions of effects, but varied in size of coefficients and were not consistent in statistical significance. Specifically, in comparison with hospices in urban areas, rural hospices were more likely to be smaller, older, and be owned by governmental than non-profit organizations. The duration of services provided by home health aides were shorter in rural hospices than in urban. In comparison with urban communities, rural communities were more likely to have a larger proportion of male residents and non-Hispanic Whites, who also were older, and had less income. These communities were also more likely to be located in the Midwest, South, or West rather than in the Northeastern United States.

Table 3.

Logistic Regression Results: Predicting Rural Location (N= 3,858)

	Gold Standard (Model 1)			RUCA (Model 2)			FORHP (Model 3)			IRR (Model 4)			FAR (Model 5)			U.S. Census (Model 6)
	OR	SE	P	OR	SE	p	OR	SE	p	OR	SE	p	OR	SE	p	OR	SE	p
Hospice
Age	1.049	0.007	<.001	1.047	0.007	<.001	1.010	0.006	0.096	1.051	0.007	<.001	1.045	0.009	<.001	1.012	0.008	.117
Ownership
Non-profit	0.797	0.105	.083	0.642	0.080	<.001	0.893	0.097	0.298	0.636	0.077	<.001	0.668	0.104	.009	0.880	0.122	.357
Government	1.069	0.181	.694	1.038	0.172	.824	1.042	0.154	0.779	1.131	0.191	.465	1.064	0.192	.733	1.022	0.174	.897
Daily censusa	0.984	0.001	<.001	0.985	0.001	<.001	0.997	0.001	<.001	0.992	0.001	<.001	0.988	0.002	<.001	0.988	0.001	<.001
Visitsb
Health aide	0.989	0.005	.029	0.997	0.005	.498	0.992	0.004	0.040	0.999	0.005	.786	0.990	0.007	.121	0.995	0.005	.356
Skilled nurse	0.993	0.005	.175	0.988	0.005	.016	1.004	0.005	0.429	0.985	0.005	.003	0.984	0.008	.065	1.000	0.006	.991
Social worker	1.012	0.033	.705	1.022	0.033	.501	1.019	0.031	0.538	1.043	0.037	.230	1.013	0.039	.732	0.993	0.034	.838
Community
Women	0.648	0.032	<.001	0.538	0.029	<.001	0.560	0.031	<.001	0.472	0.029	<.001	0.842	0.032	<.001	0.768	0.030	<.001
Non-Hispanic Whites	1.026	0.004	<.001	1.029	0.004	<.001	1.018	0.003	<.001	1.064	0.004	<.001	1.008	0.005	.089	1.025	0.005	<.001
Age	1.179	0.016	<.001	1.231	0.017	<.001	1.230	0.016	<.001	1.229	0.018	<.001	1.062	0.015	<.001	1.284	0.020	<.001
Income	0.852	0.007	<.001	0.889	0.006	<.001	0.903	0.004	<.001	0.873	0.005	<.001	0.928	0.008	<.001	0.912	0.007	<.001
Region
Midwest	2.751	0.580	<.001	2.774	0.552	<.001	0.916	0.152	0.597	3.488	0.686	<.001	1.293	0.320	.299	1.368	0.273	.117
South	2.191	0.484	<.001	4.304	0.894	<.001	1.093	0.185	0.601	8.430	1.729	<.001	1.030	0.270	.911	2.985	0.629	<.001
West	1.371	0.336	.197	1.892	0.433	.005	17.123	3.411	<.001	4.741	1.045	<.001	2.194	0.570	.002	0.218	0.060	<.001
χ2(14)	1,928.47, p < .001			2,107.85, p < .001			1,835.55, p < .001			2,540.47, p < .001			477.34, p < .001			1,375.33, p < .001

Note. IRR is OLS regression model; IRR scale was transformed to a scale from 1 to 10 to ease of interpretation. OMB, RUCA, and FORHP are binomial logistic regression models; dependent variable: 1= rural and 0 = urban. Comparison group for hospice ownership is for profit; for regions – Northeast.

^ameasured in years

^bmeasured in minutes per day

Discussion

The overall goal of this study was to evaluate the performance of eight common classifications of rurality against the gold standard of the OMB’s classification. This study has shown that three classifications of rurality—URCSC, UIC, and RUCC—identified identical numbers of rural hospices as did the gold standard. This was not surprising considering that they were based on OMB’s distinctions of metropolitan versus non-metropolitan areas.^8,31 This is an important finding because it shows that differences attributed to the use of different OMB-based classifications in existing publications can be considered minimal.

This study has also demonstrated large variability in classifications of rural hospices: from as few as 9% in FAR classification to as many as 55% in FORHP, with the gold standard classifying around 24% of hospices as rural. This finding was expected considering conceptual differences in how rurality is operationalized in each classification.^8,9,31 However, this study adds to the literature by showing that classifications using single-dimensional constructs that were based on one characteristic of rurality, whether it was geographic isolation (FAR) or population density (U.S. Census Bureau) were less inclusive, underestimating proportions of rural hospices. The main issue with these single-dimensional measures is that they do not capture nuances specific to rural health.³² Thus, FAR uses geographic isolation as a proxy indicator of rurality, while previous studies showed that some rural areas that are not geographically isolated may also have limited access to hospices.¹ The main issue with the U.S. Census bureau classification is that, to our knowledge, there have been no publications that would adjust benchmarks of rurality to reflect access to health care. In this study we used the default benchmarks and they seemed to underestimate proportions of rural hospices.

Conversely, complex, multidimensional classifications such as IRR, RUCA, and FORHP, were more inclusive and identified more rural hospices than the OMB did. All three classifications used different sets of complex measures. Thus, RUCA classification was more inclusive than OMB because it was based on census tracts instead of counties and therefore was able to catch nuances in rurality. Federal Office of Rural Health Policy’s (FORHP) classification was designed to identify counties that had health care organizations eligible for rural development grants and some geographically large metropolitan counties could include areas with such organizations. Index of Relative Rurality’s (IRR) classification was based on the most complex definition of rurality that included population density, geographic isolation, and commuting patterns; however, because IRR is a continuous scale, there has been very little research discussing which benchmarks should be used in public health research.²⁶ Overall, considering the existing criticism of OMB’s classification that it underestimates the total number of rural areas,¹⁶ it can be reasonably assumed that these three classifications can be seen as good alternatives to OMB for identifying rural hospices.

The latter finding was also supported by the results of agreement analysis. It showed that three measures of rurality—RUCA, FORHP, and IRR—had considerable levels of agreement with the OMB’s classification. They had moderate to substantial agreement with the gold standard based on kappa statistic, meaning that there is a significant overlap between OMB and each alternative measure in classifying rural hospices. Additionally, all three measures had values of sensitivity ratio close to 100%, completely matching with the gold standard in classifying rural hospices. They also had medium to large values of PPV, adding a significant proportion of new rural hospices to the gold standard. However, the main advantage of RUCA and FORHP classifications over IRR is that they are well-established measures that have been in use by the U.S. Department of Veteran Affairs (VA)³³ and Health Resources and Services Administration (HRSA),²¹ respectively. In contrast, the IRR measure is very promising but is still in experimental stages, and there is little literature of its utility in public health research.¹⁵

The results of the regression analysis regarding specific effects of each predictor in the models paralleled findings in the existing literature which had shown that, in comparison with urban hospices, rural hospices were more likely to be smaller, older, and be owned by the governmental than for-profit organizations.³ The new finding of this study was that models predicting rurality using RUCA, FORHP, and IRR had a relatively high model fit, either similar to the gold standard, as in the case of FORHP and RUCA, or significantly higher than it, as in the case of IRR. To our knowledge, there have been no publications that would address the question for why the IRR model predicted rurality better than OMB. Regression analysis also revealed an interesting problem about the inconsistency of estimates across models that used different measures of rurality. Some of them were consistent across models in the direction of effects (i.e., positive or negative), but were inconsistent regarding their statistical significance. This finding is important for researchers and policymakers because it shows that the choice of the rural measure determines the statistical significance of findings.

Limitations.

This study had a number of limitations. One limitation is that the database included only Medicare hospices and thus did not have hospices that may provide services to patients who have private insurance plans. However, the number of these hospices in rural areas is very small, and their effect on overall findings is negligible. Second, in this study the authors used databases from 2013 and 2014, which at the time of preparing this paper were six years old. Considering that the latest census was conducted in 2010 and took several years to process its results, and also that it takes CMS several years to process national data on the status of hospices, the database selected in this study was the best available at the time of working on the manuscript. Finally, the use of the most recent databases would provide biased results because they would not reflect the demographic situation as it was in 2010.

Conclusion.

To our knowledge there have been no studies that compared the performance of various classifications of rurality against the gold standard of the OMB’s classification. In this evaluation, the findings revealed that all OMB-based classifications demonstrated similar results, classifying a quarter of hospices as rural, whereas other classifications had a wide range of estimates from as little as 9% to as high as 55%. Among the latter, three classifications—RUCA, FORHP, and IRR—can be considered as the alternatives to the gold standard because they had overall good agreement with it, performed well in regression models that predicted rurality of hospices, and also had larger rural classes than did the OMB’s classification. The latter finding is especially important considering the criticism that has been growing around the OMB’s classification, in particular, that it underestimates the proportion of rural areas in the U.S. More research is needed in this area of study. The findings of this study can help researchers and policymakers choose the rural measures that fit their needs.