A comparison of patient‐reported mental health outcomes for the Department of Veterans Affairs' regional telehealth and Community Care Programs

Abstract

Objectives

To compare patient‐reported outcomes for veterans with limited access to Department of Veterans Affairs (VA) mental health services referred to the Veterans Community Care Program (VCCP) or regional telehealth Clinical Resource Hubs‐Mental Health (CRH‐MH).

Data Sources

This national evaluation used secondary data from the VA Corporate Data Warehouse, chart review, and primary data collected by baseline survey between October 8, 2019 and May 27, 2020 and a 4‐month follow‐up survey.

Study Design

A quasi‐experimental longitudinal study design was used to sample 545 veterans with VCCP or CRH‐MH referrals for new treatment episodes. Patient‐reported outcomes included symptom severity, perceived access, utilization, and patient‐centeredness.

Data Collection

During the baseline and follow‐up surveys, all veterans were administered the Patient Health Questionnaire‐8 (PHQ‐8) to assess depression severity, and veterans with a provisional diagnosis of posttraumatic stress disorder (PTSD) were also administered the PTSD Checklist for DSM‐5 (PCL‐5) to assess PTSD symptom severity. The 4‐month follow‐up survey also asked about perceived access using the Perceived Access Inventory, the number of encounters, and patient‐centeredness of care using the Patient‐Centered Care portion of the Veterans Satisfaction Survey.

Principal Findings

Results indicated that compared to VCCP consults, veterans with CRH‐MH consults reported 0.65 (CI₉₅ = 0.51–0.83, p < 0.01) times the number of barriers to care, but a non‐significant lower number of encounters (−0.792, CI₉₅ −2.221, 0.636, p = 0.28). There was no significant (p = 0.24) difference in satisfaction with patient‐centeredness, with both groups “agreeing” on average to positively worded questions. Veterans in both groups experienced little improvement in depression or PTSD symptom severity, and there were no clinically meaningful differences between groups.

Conclusions

Overall findings indicate that the CRH‐MH and VCCP generate similar patient‐reported outcomes. Future research should compare the quality and cost of care delivered by the VCCP and CRH‐MH programs.

What is known on this topic

• Prior comparisons of in‐person VA mental health care to the private sector have found that VA scores significantly higher for provider ratings, communication, and coordination.

• No prior research has compared VA telemental health care to the private sector.

• To inform VA's “make or buy” decisions, research is needed comparing the VA's telemental health program to private sector services paid for by VA.

What this study adds

• The VA's regional telehealth Clinical Resource Hubs‐Mental Health (CRH‐MH) program was significantly and substantially more likely to serve rural veterans than the Veterans Community Care Program (VCCP).

• Veterans with referrals to a CRH‐MH reported a significantly and substantially lower number of barriers compared to veterans with VCCP referrals, though utilization, patient‐centeredness, and clinical outcomes were similar.

• Results indicate that VA could expand both the CRH‐MH program and the VCCP, and let veterans and their providers choose which option best fits their needs.

1. INTRODUCTION

Veterans facing long travel times and/or long appointment wait times for mental health services at the Department of Veterans Affairs (VA) Medical Centers have two alternative options for receiving care. The first is the Veterans Community Care Program (VCCP) in which veterans are authorized to receive services from non‐VA providers paid for by the VA. The second is a regional Clinical Resource Hubs‐Mental Health (CRH‐MH) program that offers clinic‐to‐clinic interactive video encounters to veterans at their local VA facility or home‐based interactive video encounters via VA Video Connect (using a personal computer or VA‐issued tablet). ¹ , ²

Title I (Caring for our Veterans), section 101, of the 2018 VA Maintaining Systems and Strengthening Integrated Outside Networks (MISSION) Act enables eligible VA enrollees to receive health services in their community paid for by the VA. ³ VA staff place consults for VCCP at the request of veterans. Consults are authorized if any of the following conditions are met: (1) the veteran needs a service that is not available at a VA facility (e.g., obstetrical care), (2) VA cannot provide care within certain designated access standards (<30 min average drive time for mental health or <20 days wait time for a mental health appointment), (3) VA service line does not meet certain quality standards, or 4) it is in the veteran's best medical interest. ⁴ If a consult for VCCP is authorized, a third‐party administrator (i.e., TriWest Healthcare Alliance, Optum Public Sector Solutions, Inc.) is responsible for finding a contracting provider accepting new patients and facilitating appointment scheduling. Approximately 25% of veterans enrolled in VA care have used VCCP. ⁵

Title IV (Health Care in Underserved Areas), section 402, of the 2018 MISSION Act also required the VA to conduct a three‐year pilot program to address the problem of underserved facilities (i.e., those with long appointment wait times). In response, the VA established Clinical Resource Hubs‐Mental Health (CRH‐MH) in Fiscal Year 2019 by combining and expanding regional Tele‐Primary Care Hubs with regional Tele‐Mental Health Hubs. The CRH‐MH program provides mental health services to veterans receiving care at underserved VA facilities, primarily Community Based Outpatient Clinics (CBOCs). There are CRH‐MH programs in each of the VA's 18 Veterans Integrated Service Networks (VISNs), which are regional multi‐state networks of facilities responsible for providing comprehensive services to veterans. The MISSION Act (Title 1, Subtitle B, Sec. 151) facilitated the CRH‐MH programs by allowing tele‐providers to practice across state lines with a single state license. Similar to VCCP, VA staff place CRH‐MH consults on behalf of veterans.

The mandates in the 2018 Mission Act have prompted VA to revisit many of its past “make or buy” decisions. When discussing such make‐or‐buy decisions, many have argued that mental health care should be a core service provided by VA due to high volume, VA provider expertise, and non‐VA provider shortages. ⁶ , ⁷ , ⁸ In fact, a prior comparison of in‐person VA mental health care to the private sector found that VA scored significantly higher for provider ratings, communication, coordination, and equally as well for timely access. ⁹ However, community care has not been compared to CRH‐MH. Because of the critical importance of providing access to effective mental healthcare for veterans, the objective was to compare the experience of veterans with approved consults to VCCP and CRH‐MH. Because of the high volume of mental health services provided by the VA, we hypothesized that compared to veterans with VCCP consults, those with CRH‐MH consults would be less likely to report barriers to care, report more encounters and greater satisfaction with patient‐centeredness of care, and report greater reductions in symptom severity.

2. METHODS

The VA Quality Enhancement Research Initiative (QUERI) supports rigorous evaluations of VA quality improvement initiatives, and this evaluation was conducted by the Virtual Care QUERI program. Per VHA Handbook 1200.21 (Veterans Health Administration 2019), all VA authors of this manuscript attest that the activities that resulted in producing this manuscript were not conducted as part of a research project, but as part of the non‐research evaluation conducted under the authority of the Office of Mental Health and Suicide Prevention. This quasi‐experimental evaluation used secondary data from the VA Corporate Data Warehouse, chart review, and primary data collected by a survey to compare the outcomes of care between those veterans with a VCCP consult and those with a CRH‐MH consult. Using the VA Corporate Data Warehouse, we identified all VCCP and CRH‐MH consults for a new episode of mental health care in four VISNS, further assessed eligibility via chart review, and administered baseline and follow‐up surveys to a sample of eligible veterans. We focused on VISNs 10, 19, 20, and 22 because the CRHs in these VISNs were well established and serving patients at the beginning of the evaluation. VISN 10 includes the states of Michigan, Ohio, and Indiana, while VISNs 19, 20, and 22 include the rocky mountain and west coast states (except for northern California and western Nevada).

Inclusion criteria for consults were: (1) consult type was VCCP or CRH‐MH, (2) originated from a CBOC in VISNs 10, 19, 20, and 22, (3) placed between September 16, 2019 and May 3, 2020, and (4) had a provisional diagnosis of either depression or posttraumatic stress disorder (PTSD). To assess the inclusion criteria, we examined the Con.Consult table in the Corporate Data Warehouse, which includes information about VA consults. For the first inclusion criteria, we identified records where the field ToRequestServiceName was one of the names of the four CRH‐MH programs (e.g., “Western Telehealth Network” or “WTN”) or included the words “community,” “choice,” “non‐VA,” or “purchased.” For the fourth inclusion criteria, we identified records where the field ProvisionalDiagnosis indicated a diagnosis of depression (e.g., “unipolar depression,” “f33”) or PTSD (e.g., “posttraumatic stress,” “f43.1”). Consults were excluded if they had a CPRS status of discontinued or canceled at the time of data extraction. Consults were excluded if they had a consult with the same or similar ToRequestServiceName name with a Request. Date in the past 13 months.

Because consults can be forwarded from one facility to another, we identified the source (VISN and CBOC) of the referral based on each veteran's pattern of primary care utilization. For the second inclusion criteria, the veteran's most frequent location of primary care in the past 12 months was used as a proxy for the CBOC requesting the consult. If the most frequent location was a VAMC, the consult was excluded. We initially intended to sample equal numbers of VCCP and CRH‐MH consults from each CBOC in order to control for community and referring facility characteristics. However, the consult patterns at the CBOCs were highly polarized with nearly all the consults from a given CBOC being for either VCCP or CRH‐MH. Instead, we stratified the consults by the four VISNs and two provisional diagnoses (depression or PTSD) and attempted to sample equal numbers of VCCP and CRH‐MH consults for each of the eight strata. Consults were identified on a weekly basis and sampling was done each week, with strategic oversampling to balance numbers in each cell. From the 2877 consults, we sampled 1019 unique consults from 76 CBOCs. Because VISN consult patterns were also somewhat polarized, we were not able to perfectly balance the sample across all eight strata in the final sample (see Table 1). Covariates for the sample were collected from the Corporate Data Warehouse including, age, sex, race, ethnicity, marital status, rurality, suicidality, diagnostic categories, visual/hearing impairment, Charlson score (a diagnosis‐based measure of overall comorbidity), psychotropic medications, and prior VA outpatient mental health use in the past four months. Mental health diagnoses, suicide/self‐injury ideation visual/hearing impairment were identified using the Clinical Classifications Software, Revised categories.

TABLE 1.

Completed baseline interviews by sampling strata

	Veterans Community Care Program N = 242		Clinical Resource Hubs‐Mental health N = 303
VISN	PTSD	Depression	PTSD	Depression
10	8	4	15	6
19	17	18	52	55
20	65	68	63	33
22	32	30	45	34
Total	122	120	175	128

We subsequently excluded consults if the veteran did not have a phone or was in inpatient/residential care. We also conducted a chart review of all sampled VCCP consults to further assess eligibility. VCCP consults were subsequently excluded if they were: (1) declined (or not yet approved by the end of enrollment), (2) a Secondary Authorization Request (SAR) for the continuation of care with the same VCCP provider, (3) similar to a completed consult in the past 13 months to the same provider (SAR‐like), or (4) put on hold for the COVID‐19 pandemic because VCCP providers were not accepting new patients. Veterans with VCCP consults, who were not yet authorized, did not enter the sample until they became authorized. Veterans with approved consults were not removed from the sample if the consult was later canceled or discontinued. As can be seen in the Consort Diagram (Figure 1), a greater number of VCCP consults were determined to be ineligible based on chart review because of these additional exclusion criteria. Even though we oversampled VCCP consults to account for the greater rate of ineligibility, the final sample of eligible consults included more CRH‐MH consults (n = 437) than VCCP consults (n = 360).

FIGURE 1.

CONSORT flow diagram. COVID‐19, Coronavirus Disease 2019; SAR, Secondary Authorization Request; VCCP, Veterans Community Care Program.

Baseline surveys were administered between October 8, 2019 and May 27, 2020. The home address and phone numbers of veterans with eligible consults were collected in order to contact them for the survey. Sampled veterans were mailed invitation letters (including an opt‐in/opt‐out postcard) for baseline survey participation. Those opting in, or not opting out, were contacted by phone. During the baseline survey, all veterans were administered the Patient Health Questionnaire‐8 (PHQ‐8) to assess depression severity along with a question about their satisfaction trying to schedule the initial appointment. Veterans with a provisional diagnosis of PTSD were also administered the PTSD Checklist for DSM‐5 (PCL‐5) to assess PTSD symptom severity. The 4‐month follow‐up telephone survey re‐administered the PHQ‐8 and PCL‐5 (for those with a provisional diagnosis of PTSD). The follow‐up survey also administered the Perceived Access Inventory (PAI) for VA care ¹⁰ , ¹¹ and VCCP. ¹² The PAI is a psychometrically validated patient‐centered measure of perceived access to care that was developed based on qualitative interviews with rural and urban veterans and a Delphi panel of VA policy makers. We used a subset of eight items from the longer PAI that focused on perceived geographic, temporal, and cultural access, ¹³ and an item asking about “red tape.” Two of the PAI items were reverse coded to consistently frame PAI item results with negative connotations. The PAI for VCCP had four additional items asking about: (1) delays in getting approval for community care, (2) delays in VA paying community providers, (3) lack of coordination between VA and community providers, and (4) problems with VA and community providers sharing medical records. The follow‐up survey also asked questions about a number of consult encounters, and for those reporting any; we also asked veterans questions about the patient‐centeredness of care using the Patient‐Centered Care portion of the Office of Mental Health Veterans Satisfaction Survey. The Patient‐Centered Care questions asked about being treated with respect and kindness, the ability to choose treatments, and taking preferences and goals into account. To account for the impact of the COVID‐19 pandemic, we also created a dichotomous variable to determine whether the baseline survey was administered before or after March 17, 2020, (the date Medicare started paying for interactive video encounters regardless of rurality). To account for potential differences in outcomes between the VCCP and CRH‐MH outcomes pre‐ and post‐COVID‐19, we conducted a sensitivity analysis by adding a dichotomous indicator as a main effect and interaction effect with group status.

Chi‐square tests and t‐tests were used to compare the baseline characteristic of veterans with a VCCP or CRH‐MH consult to determine the balance between the two groups. In addition, multivariate statistical analysis, that controlled for non‐balanced casemix factors (e.g., provisional diagnosis, suicidality, rurality, and prior VA mental health use), was used to compare the change in PHQ‐8 and PCL‐5 scores between baseline and follow‐up, as well as the difference in satisfaction with scheduling appointments at baseline, and perceived access barriers, self‐reported utilization and patient‐centeredness of care at follow‐up. For access barriers, we originally hypothesized that compared to veterans with VCCP consults; those with CRH‐MH consults would be less likely to endorse each barrier domain (e.g., long travel distance). We also conducted a post‐hoc analysis comparing the total count of barriers endorsed using a negative binomial regression. Based on the preliminary findings of each barrier domain, we expected that veterans with CRH‐MH consults would report a lower total count of barriers endorsed.

3. RESULTS

Of the 797 veterans invited to participate in the telephone survey, 1.8% returned an opt‐out card and 13.2% declined participation after contact. The overall baseline survey completion rate was 68.4%. Veterans completing the survey were not significantly different than those not completing the survey with regard to almost all of the characteristics available in the Corporate Data Warehouse, with the exception that younger age (p = 0.04) and Asian or Pacific Islander race (p = 0.003) were negatively correlated with completing the baseline survey. The 545 veterans completing the baseline survey were referred from 70 CBOCs. Of the 545 completed baseline surveys due for the 4‐month follow‐up, 466 (85.5%) were completed.

Descriptive statistics are presented in Table 2. Just over half of the sample (55.6%) had a CRH‐MH consult and 44.4% had a VCCP consult. Despite this being a quasi‐experimental design, there were few baseline differences between those who were referred to CRH‐MH versus VCCP. A notable exception was that significantly (p = 0.001) and substantially more veterans with a CRH‐MH Hub consult (51.2%) were from rural or highly rural areas compared to veterans with a VCCP consult (36.4%). In addition, significantly (p = 0.004) more veterans with a CRH‐MH Hub consult (6.6%) were prescribed a mood stabilizer than veterans with a VCCP consult (1.2%). The biggest observed difference related to prior use of VA mental health care. Significantly (<0.001) and substantially more veterans with a CRH‐MH Hub consult (76.6%) had used other outpatient VA mental health care services in the previous four months compared to veterans with a VCCP consult (58.3%). Baseline survey data indicated no meaningful differences in PHQ‐8 scores or PCL‐5 scores, with both groups reporting moderate depression and PTSD severity.

TABLE 2.

Sociodemographic and clinical characteristics of baseline sample (n = 545)

Descriptive statistic	Overall (N = 545) mean (SD) or No. (%) among non‐missing	Veterans Community Care Program (N = 242) mean (SD) or No. (%) among non‐missing	Clinical Resource Hubs‐Mental Health (N = 303) mean (SD) or no. (%) among non‐missing	p‐value a
Electronic health record data
Age	49.5 (15.6)	49.0 (16.0)	50.0 (15.2)	0.462
Male	421 (77.2%)	190 (78.5%)	231 (76.2%)	0.598
Hispanic ethnicity	66 (13.2%)	29 (13.7%)	37 (12.8%)	0.851
Race				0.728
White	420 (86.2%)	181 (86.6%)	239 (86.0%)
Black	30 (6.2%)	15 (7.2%)	15 (5.4%)
American Indian	17 (3.5%)	7 (3.3%)	10 (3.6%)
Asian or Pacific Islander	14 (2.9%)	4 (1.9%)	10 (3.6%)
Multiracial	6 (1.2%)	2 (1.0%)	4 (1.4%)
Marital status				0.485
Married	270 (50.5%)	116 (48.9%)	154 (51.7%)
Divorced/separated/widowed	147 (27.5%)	66 (26.6%)	84 (28.2%)
Single/never married	118 (22.1%)	58 (24.5%)	60 (20.1%)
Rural/highly rural	243 (44.6%)	88 (36.4%)	155 (51.2%)	0.001
Charlson score	1.6 (2.1)	1.6 (2.1)	1.6 (2.2)	0.876
Hearing impairment	46 (8.4%)	23 (9.5%)	23 (7.6%)	0.520
Visual impairment	5 (0.9%)	0 (0.0%)	5 (1.7%)	0.120
Alcohol use disorder	59 (10.8%)	25 (10.3%)	34 (11.2%)	0.846
Anxiety/phobia disorder	128 (23.5%)	52 (21.5%)	76 (25.1%)	0.378
Depressive disorder	280 (51.4%)	116 (47.9%)	164 (54.1%)	0.177
Post‐traumatic stress disorder	272 (49.9%)	109 (45.0%)	163 (53.8%)	0.052
Antidepressant	264 (48.4%)	118 (48.8%)	146 (48.2%)	0.962
Antipsychotic	43 (7.9%)	18 (7.4%)	25 (8.3%)	0.849
Anxiolytic	28 (5.1%)	12 (5.0%)	16 (5.3%)	1.000
Benzodiazepine	32 (5.9%)	14 (5.8%)	18 (5.9%)	1.000
Mood stabilizer	23 (4.2%)	3 (1.2%)	20 (6.6%)	0.004
Suicidality flag	30 (5.5%)	18 (7.4%)	12 (4.0%)	0.114
Past VA outpatient mental health encounter	373 (68.4%)	141 (58.3%)	232 (76.6%)	<0.001
Data from consult
PTSD provisional on consult diagnosis	297 (54.5%)	122 (50.4%)	175 (57.8%)	0.104
Depression provisional on consult diagnosis	248 (45.5%)	120 (49.6%)	128 (42.2%)	0.104
Survey data
PTSD severity
PCL‐5 baseline score	44.8 (15.6)	45.1 (15.9)	44.6 (15.4)	0.800
PCL‐5 follow‐up score	40.1 (17.8)	37.8 (17.4)	41.7 (18.0)	0.101
Depression severity
PHQ‐8 baseline score	13.3 (5.8)	13.2 (5.8)	13.3 (5.7)	0.849
PHQ‐8 follow‐up score	11.4 (6.1)	10.7 (6.0)	11.9 (6.2)	0.029
Satisfaction with appointment scheduling				<0.001
Very satisfied	226 (48.0%)	83 (39.3%)	143 (55.0%)
Somewhat satisfied	67 (14.2%)	30 (14.2%)	37 (14.2%)
Neither satisfied nor dissatisfied	42 (8.9%)	21 (10.0%)	21 (8.1%)
Somewhat dissatisfied	53 (11.3%)	31 (14.7%)	22 (8.5%)
Very dissatisfied	50 (10.6%)	34 (16.1%)	16 (6.2%)
Not tried to schedule an appointment	33 (7.0%)	12 (5.7%)	21 (8.1%)

Abbreviations: PCL‐5, 20‐item Posttraumatic Stress Disorder (PTSD) Checklist for Diagnostic Statistical Manual of Mental Disorders, Fifth Edition (DSM‐5); PHQ‐8, 8‐item Patient Health Questionnaire (excludes questions about suicide ideation); PTSD, posttraumatic stress disorder; SD, standard deviation; VA, Department of Veterans Affairs.

^a p‐value reflects the difference between the Clinical Resource Hub ‐ Mental Health group and the Veterans Community Care Program group for continuous (t‐test) and categorical (chi‐square test) variables.

At baseline, veterans with VCCP consults were significantly and substantially more dissatisfied with scheduling an appointment compared to veterans with CRH‐MH consults (p < 0.001), with 30.8% of veterans with VCCP consults being “dissatisfied” or “very dissatisfied” compared to 14.7% of veterans with CRH‐MH consults (Table 2). Likewise, consistently fewer veterans with CRH‐MH consults reported barriers to care on the PAI compared to veterans with VCCP consults (Table 3). The post hoc negative binomial regression analysis indicated that veterans with CRH‐MH consult reported a significantly lower number of barriers compared to veterans with VCCP consults (IRR_{CRH‐MH/VCCP} = 0.65, CI₉₅ = 0.51 to 0.83, p < 0.001). However, only one barrier domain (providers lack knowledge of military culture) was significantly lower independently (OR_{CRH‐MH/VCCP} = 0.30, CI₉₅ = 0.18 to 0.52, p < 0.001). For veterans with VCCP consults, results from the VCCP version of the PAI indicated that over a third (37.3%) reported at follow‐up that there were delays with the VA authorizing the consult, and a third (34.8%) were concerned about the lack of coordination between VA and VCCP providers. However, relatively few veterans with VCCP consults reported problems with the VA paying VCCP providers (12.7%) or with the VA and VCCP providers sharing mental health medical records (10.5%).

TABLE 3.

Comparison of Community Care and Telemental Health Hub consult outcomes

Outcomes	Veterans Community Care Program consults (N = 207) a Mean (SD) or No. (%) among non‐missing	Clinical Resource Hubs‐Mental Health consults (n = 259) a Mean (SD) or No. (%) among non‐missing	Adjusted difference effect size 95% confidence interval, p‐value
Individual barriers
Travel a long distance	25 (12.7%)	32 (12.5%)	ORCRH‐MH/VCCP = 0.707, CI95 0.381, 1.312, p = 0.270
Inconvenient appointment times b	28 (14.6%)	18 (7.3%)	ORCRH‐MH/VCCP = 0.535, CI95 0.286, 1.001, p = 0.050
Appointments too short	30 (16.6%)	26 (10.6%)	ORCRH‐MH/VCCP = 0.550, CI95 0.292, 1.035, p = 0.064
Providers not available as soon as needed	44 (23.0%)	39 (15.8%)	ORCRH‐MH/VCCP = 0.615, CI95 0.377, 1.004, p = 0.052
Providers lack knowledge of military culture	45 (27.1%)	30 (13.4%)	ORCRH‐MH/VCCP = 0.304, CI95 0.177, 0.524, p < 0.001
Lack trust in providers	37 (19.9%)	40 (16.1%)	ORCRH‐MH/VCCP = 0.657, CI95 0.388, 1.111, p = 0.117
Stuck in “red tape” or paperwork	34 (17.3%)	29 (11.6%)	ORCRH‐MH/VCCP = 0.643, CI95 0.372, 1.112, p = 0.114
Lack of awareness of available services b	61 (31.1%)	52 (21.8%)	ORCRH‐MH/VCCP = 0.644, CI95 0.397, 1.044, p = 0.074
Number of barriers endorsed	1.3 (1.6)	0.9 (1.3)	IRRCRH‐MH/VCCP = 0.65, CI95 = 0.51, 0.83, p < 0.001
Service utilization
Any encounters	155 (75.2%)	211 (82.1%)	ORCRH‐MH/VCCP = 1.241, CI95 0.747, 2.063, p = 0.404
Number of encounters	6.2 (6.8)	5.9 (7.3)	βCRH‐MH‐VCCP = ‐0.792, CI95‐2.221, 0.636, p = 0.276
Patient centeredness	N = 155	N = 211
Patient centered satisfaction survey score	4.2 (0.7)	4.3 (0.6)	βCRH‐MH‐VCCP = 0.155, CI95−0.107, 0.417, p = 0.243
Clinical outcomes
ΔPHQ‐8	−2.2 (5.3)	−1.2 (4.9)	βCRH‐MH‐VCCP = 1.413, CI95 0.321, 2.505, P = 0.011
ΔPCL‐5	−6.0 (12.6)	−3.4 (12.5)	βCRH‐MH‐VCCP = 2.980, CI95−1.071, 7.031, p = 0.148

Abbreviations: CI₉₅, 95% confidence interval; IRR_{CRH‐MH‐VCCP}, incidence rate ratio, the ratio of the number of events for Clinical Resource Hubs‐Mental Health and Veterans Community Care Program; OR_{CRH‐MH‐VCCP}, odds ratio, odds of observing outcome in Clinical Resource Hubs‐Mental Health compared to Veterans Community Care Program; β_{CRH‐MH‐CC}, difference in scale's units between Clinical Resource Hubs‐Mental Health and Veterans Community Care Program; Δ, Change in; ΔPCL‐5, change in 20‐item Posttraumatic Stress Disorder (PTSD) Checklist for Diagnostic Statistical Manual of Mental Disorders, Fifth Edition (DSM‐5); ΔPHQ‐8, change in 8‐item Patient Health Questionnaire (excludes question about suicide ideation).

^a Sub‐sample with completed 4‐month follow‐up surveys.

^b Reverse scored.

A large proportion of veterans with CRH‐MH consults (82.1%) and VCCP consults (75.2%) attended at least one appointment with no significant between‐group difference (OR_{CRH‐MH/VCCP} = 1.24, CI₉₅ = 0.75–2.06, p = 0.404) controlling for casemix. The average number of encounters was similar in both groups, 5.9 (SD = 7.3) for CRH‐MH and 6.2 (SD = 6.8) for VCCP (β_{CRH‐MH‐VCCP} = −0.79, CI₉₅ = −2.22 to 0.64, p = 0.28). In the CRH‐MH group, among veterans with any encounters, 14.2% had all clinic‐to‐clinic interactive video encounters while 85.8% had at least one interactive video encounter in their home with the average number being 4.2 (SD = 5.9). In the VCCP group, among veterans with any encounters, 38.1% had at least one interactive video encounter in their home and the average was 2.3 (4.3).

For those veterans with encounters, both groups “agreed” on average with questions about satisfaction with patient‐centeredness (β_{CRH‐MH‐VCCP} = 0.16, CI₉₅ = −0.11 to 0.42, p = 0.24). For veterans attending CRH‐MH encounters (n = 211), the vast majority agreed or strongly agreed at follow‐up that the “meeting by interactive video goes smoothly” (80.7%), that the “meeting by interactive video as helpful as in‐person” (76.4%), and that they were “satisfied with the quality of care” (90.9%).

Among veterans completing the 4‐month follow‐up, there was an average −1.6 unit decrease (SD = 5.1) in PHQ‐8 depression symptom severity, which is not clinically meaningful. Controlling for casemix, veterans with VCCP consult had a statistically significant, but not clinically meaningful, greater decrease in severity than those with CRH‐MH consults (β_{CRH‐MH‐VCCP} = 1.4, CI₉₅ = 0.32–2.51, p = 0.01). Among those with a provisional diagnosis of PTSD at baseline (n = 297) and completing the 4‐month follow‐up, there was an average −4.5 unit decrease (SD = 12.6) in PCL‐5 PTSD symptom severity, which is not clinically meaningful, and the group difference was not statistically significant after adjusting for casemix (β_{CRH‐MH‐VCCP} = 2.98, CI₉₅ = −1.07 to 7.03, p = 0.15). Few of the findings changed meaningfully in sensitivity analyses accounting for the main effect and group‐interaction effect of conducting the baseline survey after March 17, 2020. Neither the main nor the interaction effects were significant. Veterans with CRH‐MH consults still reported a fewer number of barriers than those with VCCP do consult, but the difference was no longer statistically significant. In addition, differences in PHQ‐8 scores were no longer statistically significant.

4. DISCUSSION

The CRH‐MH program and VCCP appeared to serve similar populations, with a few important exceptions. The CRH‐MH program was significantly and substantially more likely to receive consults from rural veterans than VCCP. This is important because the VCCP was designed in part to serve veterans living >30 min from a VA facility. However, it is likely that in some rural communities, there was insufficient access to VCCP mental health providers. This highlights the important role that the CRH‐MH program is playing for veterans living in rural areas. However, the fact that a third of veterans with a VCCP consult also had at least one interactive video visit suggests that VCCP providers could potentially serve more rural Veterans. Most veterans were satisfied with scheduling an initial appointment, but those with a CRH‐MH consult were significantly and substantially more satisfied than those with a VCCP consult.

Importantly, Veterans with CRH‐MH consults reported a significantly and substantially lower number of barriers compared to veterans with VCCP consults, especially with regard to inconvenient appointment times, appointments being too short, providers not available as soon as needed, lack of awareness of available services, and providers lacking knowledge of military culture. However, the differences between veterans with CRH‐MH and VCCP consults were substantially mitigated after the COVID‐19 pandemic began, perhaps because community providers pivoted to delivering mental health care to veterans in their homes via interactive video. This finding suggests that community providers can use interactive video to reduce barriers to care for veterans with VCCP consults.

Importantly, a relatively high proportion (~80%) of veterans in both groups had at least one encounter. In contrast, 45%–62% of veterans (diagnosed with PTSD) within the VA healthcare system are successfully referred from primary care to specialty mental health. ¹⁴ , ¹⁵ , ¹⁶ Overall, results indicate that when given a choice (as opposed to being randomized) of CRH‐MH or VCCP, service utilization, and satisfaction with patient‐centeredness are similarly good. It is likely that veterans or their providers choose the option that works best given the context. A major concern is the lack of symptom improvement in both groups. Prior research indicates that veterans with depression are more treatment‐resistant than civilians ¹⁷ and veterans likely need more intensive care (e.g., telepsychiatry collaborative care ¹⁸ , ¹⁹ ) than was received by the veterans completing the surveys.

Overall the findings provide policy‐relevant information to VA leadership facing make or buy decisions. Specifically, should VA increase access for veterans facing long travel times and/or long appointment wait times by expanding VA telemental health services or contracting with community providers? Though an additional examination of cost and quality is needed, results indicate that VA could expand both the CRH‐MH program and the VCCP, and let veterans and their providers choose which option best fits the need. Importantly, a third of veterans with VCCP consults were concerned about lack of coordination between their VA and VCCP providers and it is possible that referring veterans to CRH‐MH is preferable to risking care fragmentation. ⁹ Lack of care coordination across VA and VCCP providers could negatively impact quality, ⁵ and is a known concern among VA primary care providers. ²⁰

A limitation of this evaluation is that we were not able to assess the quality and cost of care. The most recent data available indicate that the VA provides higher quality outpatient care for depression than the private sector. ²¹ Another limitation is that data are limited to four of VA's 21 regional networks, and the result could vary across regions, as well as within regions. Additional evaluation is needed to examine geographic variation. A major strength of this evaluation was that we completed surveys with patients with consults who were unable to schedule/attend visits. This contrasts with other evaluations (e.g., the Survey of Healthcare Experiences of Patients [SHEP]) which only samples veterans attending visits. Other strengths included a high survey response rate (30%–50% higher than SHEPS ⁹ ), and the inclusion of a follow‐up survey (also with high completion rates) to assess symptom change over time. There were few meaningful differences between baseline survey completers and non‐completers, suggesting that selection bias was minimal. Likewise, there were few differences in clinical characteristics between those with CRH‐MH and VCCP consults, making for a balanced comparison (even without randomization).

FUNDING INFORMATION

This work was supported by a grant (QUE 15‐282) from the Department of Veterans Affairs, Health Services Research & Development (HSR&D), Quality Enhancement Research Initiative (QUERI), and the VA's Primary Care Analytics Team (PCAT), funded by the VA Office of Primary Care. The views expressed are those of the authors and do not necessarily reflect the position or policy of the University of Washington, the Department of Veterans Affairs, or the U.S. Government.

CONFLICT OF INTEREST

The authors report no known conflicts of interest associated with this publication and no significant financial support for this work that could have influenced its outcome.

Fortney JC, Carey EP, Rajan S, Rise PJ, Gunzburger EC, Felker BL. A comparison of patient‐reported mental health outcomes for the Department of Veterans Affairs' regional telehealth and Community Care Programs. Health Serv Res. 2022;57(4):755‐763. doi: 10.1111/1475-6773.13993