Rural Veterans’ Experiences With Outpatient Care in the Veterans Health Administration Versus Community Care

Supplemental Digital Content is available in the text.

Abstract

Background:

The 2014 Veterans Access, Choice and Accountability Act was intended to improve Veterans’ access to timely health care by expanding their options to receive community care (CC) paid for by the Veterans Health Administration (VA). Although CC could particularly benefit rural Veterans, we know little about rural Veterans’ experiences with CC.

Objective:

The objective of this study was to compare rural Veterans’ experiences with CC and VA outpatient health care services to those of urban Veterans and examine changes over time.

Research Design:

Retrospective, cross-sectional study using data from the Survey of Healthcare Experiences of Patients (SHEP) and VA Corporate Data Warehouse. Subjects: All Veterans who responded to the SHEP survey in Fiscal Year (FY) 16 or FY19.

Measures:

Outcomes were 4 measures of care experience (Access, Communication, Coordination, and Provider Rating). Independent variables included care setting (CC/VA), rural/urban status, and demographic and clinical characteristics.

Results:

Compared with urban Veterans, rural Veterans rated CC the same (for specialty care) or better (for primary care). Rural Veterans reported worse experiences in CC versus VA, except for specialty care Access. Rural Veterans’ care experiences improved between FY16 and FY19 in both CC and VA, with greater improvements in CC.

Conclusions:

Rural Veterans’ reported comparable or better experiences in CC compared with urban Veterans, but rural Veterans’ CC experiences still lagged behind their experiences in VA for primary care. As growing numbers of Veterans use CC, VA should ensure that rural and urban Veterans’ experiences with CC are at least comparable to their experiences with VA care.

The 2014 Veterans Access, Choice and Accountability Act (Choice) was passed in response to revelations of excessive delays in care at the Veterans Health Administration (VA). Choice was intended to improve timely access to outpatient health care for Veterans by expanding their options to receive care from community providers paid for by the VA.1 Under Choice, eligible Veterans could receive community care (CC) if they had to drive >40 miles to the nearest VA facility or wait >30 days for needed care.2 By 2018, approximately one-third of Veterans receiving care through the VA used CC.3 Recent implementation of the VA Maintaining Internal Systems and Strengthening Integrated Outside Networks Act of 2018 (MISSION) expanded the eligibility criteria under which Veterans can use CC.4 Thus, growing numbers of Veterans are expected to use CC services in the coming years.3

Increased access to CC could particularly benefit rural Veterans who have historically faced more barriers in accessing VA care than their urban counterparts due to greater geographic obstacles, provider shortages, and hospital closures.5,6 At the same time, health care delivery system deficits in rural regions, such as workforce shortages and poor infrastructure, could limit the ability of CC to deliver timely, high-quality care, which in turn could adversely affect how rural Veterans experience CC. However, we know very little about rural Veterans’ experiences in CC. To our knowledge, only 1 recent study compared Veterans’ experiences with care in the VA and CC. Vanneman et al7 found that Veterans reported better experiences with outpatient care received in the VA versus CC, except for access to specialty care, where CC was rated more highly. These results are generally comparable to those from studies that have compared the experiences of Veterans receiving care in VA and non-VA settings through Medicare, in which Veterans report being equally or more satisfied with their VA care.8–11

We build on Vanneman and colleagues seminal work by focusing specifically on rural Veterans’ experiences in CC, a critical issue given that rural Veterans represent about 30% of all VA enrollees, yet comprised 39% of Veterans who used CC in Fiscal Year (FY) 15 and FY16.12 We used data from the Survey of Healthcare Experiences of Patients (SHEP) to: (1) compare rural and urban Veterans’ experiences with outpatient care in CC and VA; (2) compare rural Veterans’ experiences with outpatient care in CC and VA in FY16 (early in the post-Choice period) and FY19 (the most recent year SHEP data were available); and (3) assess whether rural Veterans’ experiences in CC changed over time. We hypothesized that: (1) rural Veterans would report worse experiences with care in CC than their urban counterparts; (2) rural Veterans would report worse experiences with care in CC than in VA; and (3) rural Veterans’ experiences in CC would improve over time.

METHODS

Data Sources and Sample

We obtained 2 years of administrative data [FY16 (10/1/15-9/30/16) and FY19 (10/1/18-9/30/19)] from the VA Corporate Data Warehouse (CDW)13 and merged these with SHEP survey data obtained from VA’s Office of Reporting, Analytics, Performance, Improvement, and Deployment using Veterans’ scrambled social security numbers. SHEP uses items from the Consumer Assessment of Healthcare Providers and Systems survey, a well-established survey developed by the Agency for Healthcare Research and Quality to assess patients’ experience with care.14 The VA administers separate surveys for VA outpatient primary care, VA outpatient specialty care, and CC (all services).

The sample included all Veterans who responded to the SHEP survey in FY16 or FY19. Although very few Veterans responded to both CC and VA surveys (ie, 0.03% of the sample in FY16 and 0 Veterans in FY19), we retained their survey responses in the analysis. The sampling strategies for the VA and CC surveys differ slightly. The VA sample is designed to enable facility-level comparisons and random samples by type of care are drawn for each VA facility. For the CC survey, random samples are drawn on a rolling basis within major categories of care (ie, type of service) based on use of CC within 3-month periods. We used the “category of care” variable to identify primary and specialty outpatient users.

We obtained Institutional Review Board approval from the [VA Boston Healthcare System] for this study.

Measures

Outcome Measures

Our dependent variables included 3 composite measures: Access (4 items), Communication (4 items), and Coordination (3 items)—and a measure of Overall Provider Rating (1 item) (hereafter “Provider Rating”) from the SHEP data. The Access composite, for example, focused on appointment wait times, time spent in the waiting room, and providers’ responsiveness to Veterans’ medical questions (such as when a Veteran calls the clinic after-hours). (See Supplementary Materials [A] for the individual items included in each composite, Supplemental Digital Content 1, http://links.lww.com/MLR/C232). As in Vanneman et al’s study,7 we used composite measures versus individual items to allow greater comparability since there were slight differences across surveys in the wording of individual questions. We averaged individual items to create the composite measures. For the composites, scores ranged from 1 to 4 (1=never to 4=always), with higher scores indicating greater care satisfaction. For the Provider Rating scale, scores ranged from 0 to 10 (0=worst to 10=best).

Primary Independent Variables

We classified Veterans into 1 of 4 subgroups based on their site of care from the SHEP data (VA vs. CC) and rural/urban status from the CDW: (1) Rural VA; (2) Rural CC; (3) Urban VA; and (4) Urban CC. Veterans classified as “rural” or “highly rural” were combined because of the small number of highly rural Veterans in the sample.

Other Covariates

We included covariates found to be important from previous work,15 including: age (coded continuously), sex (male/female), race (White, Black, and Other), education level (ranging from ≤8th grade to ≥4-year college degree), marital status (married, divorced/separated, widowed, single), VA enrollment priority (coded into 3 priority groups: 1–2, 3, and 4–6; lower scores indicate higher priority for VA services based on eligibility criteria, including severity of service-connected disabilities and income level),16,17 VA Nosos risk scores (mean score=1; scores >1 indicate predicted higher cost and clinical complexity),17,18 and self-rated physical and mental health (both 5-point scales, with 1=excellent and 5=poor).16,17 All covariates were obtained from the CDW except Veterans’ self-rated physical and mental health, which were obtained from SHEP data.

Analysis

Our unit of analysis was the survey response. We first compared VA and CC SHEP respondents on sociodemographic characteristics, stratified by rurality status. We then ran multiple regression models, one for each outcome measure for both primary and specialty care, including the above covariates. The models included the 4 user subgroups, covariates, and the interaction of these variables with an indicator variable for FY19 since we were interested in changes in SHEP scores over time. For each user group in each time period, we calculated adjusted means (specifically, population marginal means).

Because our sample sizes were large, and small nonmeaningful differences in the means can be statistically significant, we report effect sizes (ESs), which are not dependent on sample sizes. ESs of 0.10 are often interpreted as indicating “negligible” differences between groups; ESs of 0.20, 0.50, and 0.80 are considered “small,” “medium,” and “large,” respectively.19 We used bootstrapping (with 100 bootstrap samples) to estimate P values when comparing the magnitude of decline from FY16 to FY19 across groups. For these analyses, we report P values rather than ESs because we did not have individual-level observations on changes in scores between FY16 and FY19. We used SAS version 9.4 for all analyses.

RESULTS

Demographic Characteristics and Self-rated Physical and Mental Health

Our sample included a total of 1,083,370 respondents: 465,413 rural respondents [435,726 VA (93.6%); 29,687 CC (6.4%)] and 617,957 urban respondents [584,006 VA (94.5%); 33,951 CC (5.5%)]. Most differences in demographic characteristics between VA and CC users were small to negligible (Table 1).

TABLE 1.

Veterans’ Sociodemographic Characteristics

Characteristic	Overall	Rural VA	Rural CC	Effect Size*	Urban VA	Urban CC	Effect Size†
N	1,083,370	435,726	29,687		584,006	33,951
Age, mean (SD)	68.92 (11.44)	69.49 (10.60)	67.39 (11.03)	0.19	68.81 (11.92)	64.96 (12.65)	0.31
Male, mean (SD)	0.95 (0.23)	0.96 (0.20)	0.93 (0.25)	0.11	0.94 (0.24)	0.90 (0.30)	0.16
Nosos risk score, mean (SD)‡	1.09 (1.44)	0.99 (1.30)	1.14 (1.40)	0.11	1.15 (1.52)	1.27 (1.54)	0.08
VA priority group, N (%)§
Unknown	69 (0.0)	28 (0.0)	4 (0.0)	0.01	35 (0.0)	2 (0.0)	<0.001
1–2	410,224 (37.9)	156,470 (35.9)	13,010 (43.8)	0.16	224,139 (38.4)	16,605 (48.9)	0.21
3	234,891 (21.7)	97,050 (22.3)	6301 (21.2)	0.03	124,502 (21.3)	7038 (20.7)	0.01
4–6	438,186 (40.4)	182,178 (41.8)	10,372 (34.9)	0.14	235,330 (40.3)	10,306 (30.4)	0.21
Race, N (%)
White	870,438 (80.3)	378,297 (86.8)	24,972 (84.1)	0.08	442,005 (75.7)	25,164 (74.1)	0.04
Black	121,098 (11.2)	23,517 (5.4)	1753 (5.9)	0.02	90,896 (15.6)	4932 (14.5)	0.03
Other	91,834 (8.5)	33,912 (7.8)	2962 (10.0)	0.08	51,105 (8.8)	3855 (11.4)	0.09
Marital status, N (%)
Married	680,813 (62.8)	292,249 (67.1)	19,513 (65.7)	0.03	348,708 (59.7)	20,343 (59.9)	0.00
Divorced/separated	240,612 (22.2)	87,948 (20.2)	6648 (22.4)	0.05	137,426 (23.5)	8590 (25.3)	0.04
Widowed	68,334 (6.3)	27,712 (6.4)	1527 (5.1)	0.05	37,546 (6.4)	1549 (4.6)	0.08
Single	84,307 (7.8)	24,230 (5.6)	1710 (5.8)	0.01	55,243 (9.5)	3124 (9.2)	0.01
Unknown	9304 (0.9)	3587 (0.8)	289 (1.0)	0.02	5083 (0.9)	345 (1.0)	0.02
Education, mean (SD)	3.85 (1.11)	3.68 (1.08)	3.78 (1.08)	0.09	3.98 (1.12)	4.07 (1.08)	0.09
Self-rated physical health, mean (SD)	3.10 (0.99)	3.13 (0.98)	3.32 (0.97)	0.20	3.06 (1.00)	3.25 (0.99)	0.19
Self-rated mental health, mean (SD)	2.63 (1.15)	2.61 (1.13)	2.81 (1.14)	0.17	2.63 (1.17)	2.83 (1.18)	0.18

^*Effect size for difference between rural VA and rural CC.

^†Effect size for difference between urban VA and urban CC.

^‡Nosos risk score=risk adjustment score; scores are scaled such that the mean Nosos score =1; scores >1 indicate greater-than-average cost and clinical complexity.

^§Priority/eligibility group=system of assigning priority to Veterans for VA services based on military service, disability rating, income level, and other factors; lower scores indicate higher priority level.

CC indicates community care; SD, standard deviation; VA, Veterans Health Administration.

Multivariate Results

Because the unadjusted and adjusted results were comparable, we focus on the adjusted results. (see Supplementary Tables 1 and 2 for unadjusted results, Supplemental Digital Content 1, http://links.lww.com/MLR/C232). Compared with urban Veterans, rural Veterans rated CC the same (for specialty care) or better (for primary care), although the variation in primary care ratings decreased over time (see Figs. 1, 2, and Supplemental Tables 3 and 4, Supplemental Digital Content 1, http://links.lww.com/MLR/C232). We used reduced scales on the y-axes of the figures to highlight the differences between groups, which causes the figures to provide a misleading view of overall trends. Therefore, we also include a version of the figures using the full scales in Supplementary Materials (Supplemental Digital Content 1, http://links.lww.com/MLR/C232). In FY16, the ESs in ratings between rural and urban Veterans using CC for primary care were medium for all outcomes, with ESs ranging from 0.24 for Access to 0.33 for Overall Provider Rating (Fig. 1, column E). By FY19, ESs for rural versus urban primary care ratings decreased to small (for Coordination and Provider Rating) to negligible (for Access and Communication). For specialty care, rural and urban Veterans’ experiences in CC were similar across all measures in both FY16 and FY19 (Fig. 2, column E). Supplemental Table 5 presents the full regression results (Supplemental Digital Content 1, http://links.lww.com/MLR/C232).

FIGURE 1.

Adjusted Mean Ratings for 4 Survey of Healthcare Experiences of Patients Outcome Measures—Primary Care, Rural and Urban Veterans, Fiscal Year (FY)16 and FY19.

FIGURE 2.

Adjusted Mean Ratings for 4 Survey of Healthcare Experiences of Patients Outcome Measures—Specialty Care, Rural and Urban Veterans, Fiscal Year (FY)16 and FY19.

For rural Veterans, those who used CC generally reported lower scores (ie, worse experiences) compared with those who used VA. (Figs. 1, 2, and Supplemental Tables 6 and 7, Supplemental Digital Content 1, http://links.lww.com/MLR/C232). For rural Veterans using primary care, all outcome measures were consistently worse in CC versus VA. For example, ESs ranged from 0.17 for care Access to 0.49 for Coordination in FY16 and remained small to medium for all outcomes in FY19 (Fig. 1, column F). CC versus VA differences were less dramatic but still present for rural Veterans using specialty care, with small to negligible differences in both FY16 and FY19 (Fig. 2, column F).

Rural Veterans’ care experiences improved between FY16 and FY19 in both CC and VA, with greater improvements in CC. For example, for primary care, the CC Provider Rating improved from 8.14 to 8.56 (0.42 points) (Fig. 1, column B) compared with the VA Provider Rating, which improved only 0.09 points (from 8.80 to 8.89) (Fig. 1, column A). Similarly, for specialty care, the CC Provider Rating increased from 8.43 to 8.72 (0.29 points) (Fig. 2, column B) compared with the VA Provider Rating (from 8.73 to 8.92, 0.19 points) (Fig. 2, column A). Despite improvements over time, rural Veterans’ experience ratings remained consistently lower in CC versus VA in FY19, although the differences were smaller than in FY16.

DISCUSSION

Contrary to our expectation that rural Veterans would have worse CC experiences than their urban counterparts, rural Veterans rated their CC experiences the same (in the case of specialty care) or better (in the case of primary care) than urban Veterans did. This finding contradicts recent research which found that care satisfaction decreased with increasing rurality among Medicare beneficiaries.20 It is consistent, although, with another recent study in which rural Veterans rated communication between their VA and non-VA providers higher than their urban counterparts.11 Consistent with our second and third hypotheses, we found that rural Veterans reported worse experiences in CC than VA, except for access to Specialty Care. Although rural Veterans’ CC experiences improved over time, their experiences continued to lag behind those in VA. These findings also align with prior studies in which Veterans rate their VA care equally or more highly than care they receive in non-VA settings, including CC.7,10,11 Our study both underscores and extends these findings by focusing specifically on the care experiences of rural Veterans.

Our finding that Veterans’ CC experiences were worse than those in VA for primary care is not surprising, given VA’s extensive implementation of Patient Aligned Care Teams (PACTs).21 PACT, the VA’s version of the patient-centered medical home, has been effective at improving primary care access and outcomes.22 This finding does, however, raise concerns about Veterans’ use of CC for primary care. As the VA builds its CC provider networks, it will be important to engage community providers that have adopted some type of medical home model. It will also be essential to inform Veterans about possible trade-offs in receiving primary care in CC versus VA.

Given the VA’s substantial investments since 2015 to address the challenges with CC in the early implementation period,23–25 we expected rural Veterans’ experiences in CC to improve over time. Some of these investments, such as community provider training on military culture and posttraumatic stress disorder, more expansive care coordinator functions, and infrastructure to support health information exchange, might have contributed to improvements in Veterans’ ratings of CC providers and their experiences with CC access and coordination. However, given that Veterans’ CC experiences continued to be poorer than their experiences in VA in FY19, this suggests that there remains room for further improvement. Under MISSION, the VA continues to invest in strengthening the CC program.26 Future research should examine whether investments, such as those noted above, continue to improve Veterans’ experiences with care and ensure that their experiences in CC are at least comparable to their VA experiences.

Our study has several limitations. There are minor differences between the VA and CC versions of the SHEP survey and sampling strategies, as previously noted, which could contribute to measurement bias in comparing results of the 2 surveys. We were unable to examine whether Veterans’ reasons for CC use (ie, due to waiting times or driving distance) were associated with differences in their CC experiences as this is not available in the SHEP data. For similar reasons, we were unable to examine the extent to which CC addressed Veterans’ geographic access to care. Of interest but outside the scope of this study, we did not examine possible experience differences related to the type of specialty care services Veterans utilized. Future research should explore these differences.

Despite these limitations, this study represents an important contribution to understanding rural Veterans’ experiences in CC and VA. Although rural regions may pose challenges for both VA and non-VA providers, our study suggests that rural Veterans’ experiences in CC are comparable to or better than the experiences of urban Veterans, although CC experience still lags behind VA in primary care for both rural and urban Veterans. These findings suggest that CC fills an important need for more timely and convenient health care services for rural Veterans.