Abstract
The current study examined patient and provider differences in use of phone, video, and in-person mental health (MH) services. Participants included patients who completed ≥1 MH appointment within the Department of Veterans Affairs (VA) from 10/1/17-7/10/20 and providers who completed ≥100 VA MH appointments from 10/1/17-7/10/20. Adjusted odds ratios (aORs) are reported of patients and providers: 1) completing ≥1 video MH appointment in the pre-COVID (10/1/17-3/10/20) and COVID (3/11/20-7/10/20) periods; and 2) completing the majority of MH visits via phone, video, or in-person during COVID. The sample included 2,480,119 patients/31,971 providers in the pre-COVID period, and 1,054,670 patients/23,712 providers in the COVID period. During the pre-COVID and COVID periods, older patients had lower odds of completing ≥1 video visit (aORs <0.65). During the COVID period, older age and low socioeconomic status predicted lower odds of having ≥50% of visits via video versus in-person or phone (aORs <0.68); schizophrenia and MH hospitalization history predicted lower odds of having ≥50% of visits via video or phone versus in-person (aORs <.64). During the pre-COVID and COVID periods, non-psychologists (e.g., psychiatrists) had lower odds of completing video visits (aORs <0.44). Older providers had lower odds of completing ≥50% of visits via video during COVID (aORs <.69). Findings demonstrate a digital divide, such that older and lower income patients, and older providers, engaged in less video care. Non-psychologists also had lower video use. Barriers to use must be identified and strategies must be implemented to ensure equitable access to video MH services.
Keywords: Telehealth, telemental health, COVID-19, Veterans, digital divide
Introduction
Telemental health (TMH), in which care is delivered in real-time via videoconferencing or phone, can substantially increase access to mental health (MH) services (Rosen et al., 2020). MH care is particularly well-suited to be delivered from a distance as it typically does not require physical examination (Grady et al., 2011). TMH can eliminate barriers to care including: 1) travel distance, time, and cost; 2) stigma and anxiety around receiving MH care in-person; and 3) physical limitations and caregiving responsibilities that can make leaving home difficult (Fletcher et al., 2018; Rosen et al., 2020). The Department of Veterans Affairs (VA), the nation’s largest integrated healthcare system, had encouraged use of video telehealth to increase access to MH care well before the COVID-19 pandemic. VA set national goals to increase the percentage of outpatient MH providers completing at least one video appointment to a Veteran’s home or non-VA location by the end of September 20191. While many providers successfully completed at least one video visit in response to this initiative, overall use rates remained relatively low, in part due to provider and patient hesitance as well as logistical barriers (e.g., difficulties integrating video appointments into workflows; Rosen et al., 2020).
In-person appointment decreases to prevent coronavirus disease 2019 (COVID-19) infection led to a dramatic rise in TMH. Although providers were encouraged to conduct video appointments whenever possible, this largescale shift to remote care resulted in a substantial amount of care being delivered via phone, a modality that previously had not been prioritized within VA; historically, phone appointments had received less workload credit than video or in-person care prior to pandemic-related reimbursement increases (Connolly et al., 2020a; Rosen et al., 2020). Phone care, while eliminating the visual component of video or in-person appointments, often has fewer barriers to use (e.g., patients and providers do not need to own a video-enabled device, know how to complete a video session, and/or have sufficient internet connectivity; Jaklevic, 2020). Indeed, there is a well-established digital divide in who is able to receive care via video versus by phone or in-person; older, lower income, and rural individuals are significantly less likely to own the necessary equipment, have adequate connectivity, and/or feel comfortable enough with technology to effectively utilize video telehealth services (Chunara et al., 2020; Eberly et al., 2020; Gray et al., 2020; Rodriguez et al., 2020). Black and Hispanic individuals may also be less likely to receive care via video; these results held when controlling for socioeconomic status (Chunara et al., 2020; Eberly et al., 2020; Lam et al., 2020). Collectively, barriers to accessing video care may result in older, lower income, rural, and minority populations being more likely to receive care via phone during COVID-19, given considerable restrictions on in-person treatment (Jaklevic, 2020; Eberly et al., 2020).
These barriers to access are particularly concerning as there may be differences in the quality of MH care provided via phone, video, or in-person. While a large body of randomized controlled trials (RCTs) has demonstrated that video care is non-inferior to in-person MH services (Acierno et al., 2016; Egede et al., 2015; Morland et al., 2015; Morland et al, 2020), there is considerably less literature examining phone versus in-person care using robust methodology (Castro et al, 2020; Varker et al., 2019). Of the few existing RCTs, two have shown equivalent effects of phone and in-person care in treating obsessive compulsive disorder (Lovell et al., 2006; Turner et al., 2014), while another demonstrated poorer outcomes for those receiving depression treatment by phone versus in-person at six-month follow-up (Mohr et al., 2012). Few studies have directly compared the quality of video and phone care; however, a smoking cessation study found video to be superior to phone with regards to patient medication compliance, satisfaction, and perceived support from their provider (Liebmann et al., 2019; Richter et al., 2015). Another smoking cessation RCT found higher rates of treatment completion and continued abstinence at six-month follow-up in the video condition versus phone (Kim et al., 2018).Furthermore, a meta-analysis revealed that video care was more effective than phone in treating veterans with depression and trauma, with considerable differences in outcome effect sizes between the two modalities (McClellan et al., 2021). Together, these findings suggest that the quality of phone care may be inferior to that of video or in-person services in certain circumstances. However, further rigorous research is needed, particularly to understand whether these differences in quality may vary based on mental health disorder or the type of treatment being received (e.g., psychotherapy versus medication management).
Providers may also question the effectiveness of treating higher risk patients (e.g., patients with psychotic disorders or a history of MH hospitalization) from a distance, whether by video or phone (Connolly et al., 2020b; Feijt et al., 2020; Gilmore et al., 2019). They therefore may be more likely to prioritize seeing higher risk patients in-person, even during a time of pandemic restrictions, if they feel the benefits of in-person MH treatment outweigh the risks (Uscher-Pines et al., 2020). Collectively, it is critical to understand whether there are demographic and/or clinical differences between patients who receive their care via phone, video, or in-person, as this could have substantial implications regarding care quality and access.
Provider-level differences in use of phone, video, and in-person care are also important to examine. Providers can be considered the gatekeepers to TMH; if they do not view TMH as having significant advantages over the alternative of in-person care, or do not feel comfortable navigating TMH technology, they may be less likely to use it with their patients (Cowan et al., 2019; Whitten et al., 2005). Medical MH providers (e.g., psychiatrists, nurse practitioners) and certain non-medical providers (e.g., social workers engaged in case management) may have larger caseloads of patients whom they see less frequently and for shorter appointment times as compared to psychologists (Pingitore et al., 2002). These differences may make the increased complexity of video appointments more burdensome for non-psychologists given that they have less time to troubleshoot technology with patients and have more patients to manage. Older providers may also feel less comfortable navigating video telehealth, which could influence rates of use compared to in-person or phone care (Czaja et al., 2009).
The current research leverages national VA data from millions of patients and thousands of providers to understand demographic and clinical predictors of phone, video, and in-person MH care. Specifically, we examined: 1) patient and provider predictors of having any video telehealth experience both before and during the COVID-19 pandemic; and 2) patient and provider predictors of having the majority of care delivered via phone, video, or in-person during COVID-19.
Methods
Study Population
Patients
Patients in the pre-COVID group had to have at least one VA MH outpatient appointment (see Supplemental Materials for included stop codes) between 10/1/17 (the initial roll-out of VA’s current videoconferencing platform, VA Video Connect; US Department of Veterans Affairs, 2020) and 3/10/20 (n= 2,480,119). Patients in the COVID group had to have at least one VA MH outpatient appointment between 3/11/20 (date of the COVID-19 pandemic declaration) and 7/10/20 (n= 1,054,670). Patients could be included in both groups; 90.5% of patients in the COVID group was also included in the pre-COVID group (n = 954,704).
Providers
Included providers had to have completed at least 100 MH outpatient encounters (see Supplemental Materials for included stop codes) between 10/1/17 and 7/10/202; if at least one visit occurred between 10/1/17 and 3/10/20 they were included in the pre-COVID group (n= 31,971), and if at least one visit occurred between 3/11/20 and 7/10/20 they were included in the COVID group (n=23,712). Providers could be included in both groups; 98.7% of providers in the COVID group were also in the pre-COVID group (n=23,407). The provider sample consisted of 23.4% psychologists, 25.2% psychiatrists, 30.8% other non-medical providers (social workers, counselors, and marriage and family therapists), and 20.6% other medical providers (nurse practitioners, physician assistants, registered nurses, licensed practical nurses, clinical nurse specialists, and pharmacists). See Table 1 for additional patient and provider characteristics.
Table 1.
Characteristics of the Population
| No. (%) | ||
|---|---|---|
| Factor | Pre-COVID-19 sample | COVID-19 sample |
| Patient Sample | ||
| Total population | 2,480,119 | 1,054,670 |
| Age | ||
| <30 | 139,049 (5.8) | 59,407 (5.7) |
| 30-49 | 748,851 (31.0) | 340,221 (32.9) |
| 50-64 | 651,164(26.9) | 303,564(29.3) |
| 65-79 | 742,063 (30.7) | 301,032 (29.1) |
| 80+ | 135,655 (5.6) | 30,851 (3.0) |
| Sex | ||
| Male | 2,091,637 (86.5) | 871,919 (84.2) |
| Female | 325,225 (13.5) | 163,186 (15.8) |
| Socioeconomic status (ADI) | ||
| Least disadvantaged tercile | 575,557 (26.5) | 243,018 (26.2) |
| Middle tercile | 881,419 (40.5) | 378,256 (40.8) |
| Most disadvantaged tercile | 717,759 (33.0) | 306,713 (33.1) |
| Race and ethnicity | ||
| White, non-Hispanic | 1,502,868 (64.2) | 637,306 (63.6) |
| Black, non-Hispanic | 543,607 (23.2) | 233,437 (23.3) |
| Other race, non-Hispanic | 84,415 (3.6) | 35,972 (3.6) |
| Hispanic | 209,132 (8.9) | 96,088 (9.6) |
| Rurality | ||
| Urban | 1,694,096 (70.3) | 731,381 (70.8) |
| Rural | 695,582 (28.8) | 293,313 (28.4) |
| Highly rural | 21,784 (0.9) | 8,885 (0.9) |
| Marital status | ||
| Married | 1,172,568 (49.2) | 500,603 (48.9) |
| Never married | 438,839 (18.4) | 194,995 (19.1) |
| Divorced, separated, widowed | 774,425 (32.5) | 328,221 (32.1) |
| ≥50% VA disability rating | 1,385,364 (77.1) | 659,873 (80.9) |
| Anxiety disorder | 705,858 (28.5) | 339,915 (32.2) |
| Bipolar disorder | 174,255 (7.0) | 105,802 (10.0) |
| Depressive disorder | 1,018,872 (41.1) | 491,144 (46.6) |
| PTSD | 932,619 (37.6) | 471,799 (44.7) |
| Substance use disorder | 532,280 (21.5) | 249,956 (23.7) |
| Schizophrenia | 93,953 (3.8) | 57,164 (5.4) |
| History of MH hospitalization | 130,847 (5.3) | 74,673 (7.1) |
| Provider Sample | ||
| Total population | 31,971 | 23,712 |
| Provider type | ||
| Psychologist | 6770 (22.7) | 5404 (24.3) |
| Psychiatrist | 7834 (26.3) | 5277 (23.8) |
| Other non-medical provider | 9097 (30.5) | 6928 (31.2) |
| Other medical provider | 6101 (20.5) | 4591 (20.7) |
| Age | ||
| 20-49 | 15,577 (59.5) | 11,634 (59.7) |
| 50-64 | 8143 (31.1) | 6400 (32.8) |
| 65+ | 2462 (9.4) | 1470 (7.5) |
| Sex | ||
| Male | 8567 (35.8) | 6263 (35.2) |
| Female | 15,350 (64.2) | 11,544 (64.8) |
Pre COVID-19 period: 10/1/17- 3/10/20. COVID-19 period: 3/11/20- 7/10/20.
ADI: Area Deprivation Index; least disadvantaged tercile= 1-33, middle tercile= 34-66, most disadvantaged tercile= 67-100.
Statistical Analysis
Binomial logistic regression was used to predict the odds of patients and providers having any video experience (delivered to the patient’s home or another non-VA location) during both the pre-COVID (10/1/17-3/10/20) and COVID (3/11/20- 7/10/20) periods based on demographic and clinical characteristics. Multinomial logistic regression was used to predict the odds of patients and providers having had ≥50% of their visits via phone or video versus in-person during the COVID period only. These analyses were not completed for the pre-COVID period, as the vast majority of patients (91.2%) and providers (94.5%) had ≥50% of their visits in-person at that time.
The following variables were included in all patient regressions: age, sex, socioeconomic status (via the Area Deprivation Index; University of Wisconsin Department of Medicine, 2020), self-reported race and ethnicity, rurality (via Rural-Urban Commuting Area codes; Rural Health Research Center, 2020), marital status, ≥50% VA disability rating, diagnosis of anxiety disorder, bipolar disorder, depression, posttraumatic stress disorder (PTSD), substance use disorder, or schizophrenia, and history of MH hospitalization. The following variables were included in all provider regressions: age, sex, and provider type (psychologists, psychiatrists, other non-medical providers, and other medical providers). All variables were obtained from the VA Corporate Data Warehouse, which integrates data from multiple national sources including the electronic health record (Price, Shea, and Gephart, 2015). Variables were entered into each model simultaneously (i.e., not stepwise).
Given the inclusion of multiple variables in all regressions, findings are reported as adjusted odds ratios (aORs). When sample sizes are large, p values may be significant (<.05) despite negligible between-group differences in variables of interest. Therefore, for the current analyses an aOR threshold of >1.43 or <.70 was used as an indicator of significance; these values reflect a Cohen’s d ≥ .2 (at least a small effect size; Chinn, 2000).
Results
Patient Findings
Patients had an average of 16.9 (SD 38.2) MH visits during the 29-month pre-COVID period and 4.5 (SD 8.7) visits during the four-month COVID period. The percentage of patients who had completed at least one video visit was 3.5% in the pre-COVID sample and 24.8% in the COVID sample. 85.3% of those with video experience in the COVID period were first-time users. In the pre-COVID sample, most patients had ≥50% of their visits in-person (91.2%); 7.9% had most by phone, 0.8% had most by video. In the COVID sample, most had ≥50% of their visits by phone (63.3%); 18.3% had most in-person, 16.6% had most by video.
In the pre-COVID period, patients who were 65+ had lower odds of completing at least one video visit (aORs < 0.34); female sex, highly rural location, and diagnosis of depression or PTSD was associated with higher odds of completing at least one video visit (aORs > 1.55; Table 2). During the COVID period, age (50+), low socioeconomic status, and schizophrenia diagnosis was associated with lower odds of completing at least one video visit (aORs < 0.69); female sex predicted higher odds of completing a video visit (aOR = 1.46). A higher percentage of first-time video users in the COVID period were from urban locations as opposed to those with pre-COVID video experience (76.0% versus 63.8%, χ2 = 4823.6; see Supplementary Table 1 for additional analyses examining first-time video users during COVID).
Table 2.
Adjusted odds of patients having any video experience prior to and during COVID-19
| Factor | % with ≥ 1 video visit prior to COVID-19 |
Adjusted OR (95% CI) |
% with ≥ 1 video visit during COVID-19 |
Adjusted OR (95% CI) |
|---|---|---|---|---|
| Age | ||||
| <30 | 4.5 | 1 [Ref] | 36.5 | 1 [Ref] |
| 30-49 | 5.7 | 1.04 (1.00-1.07) | 33.7 | 0.92 (0.89-0.94) |
| 50-64 | 3.5 | 0.73 (0.70-0.75) | 23.8 | 0.65* (0.63-0.66) |
| 65-79 | 1.7 | 0.34* (0.32-0.35) | 14.5 | 0.35* (0.34-0.36) |
| 80+ | 0.9 | 0.24* (0.22-0.27) | 11.6 | 0.27* (0.25-0.28) |
| Sex | ||||
| Male | 3.0 | 1 [Ref] | 22.8 | 1 [Ref] |
| Female | 6.8 | 1.73* (1.69-1.76) | 35.1 | 1.46* (1.44-1.48) |
| Socioeconomic status (ADI) | ||||
| Least disadvantaged tercile | 3.8 | 1 [Ref] | 28.9 | 1 [Ref] |
| Middle tercile | 3.7 | 0.93x (0.91-0.94) | 24.0 | 0.79 (0.78-0.80) |
| Most disadvantaged tercile | 3.2 | 0.86 (0.84-0.88) | 20.2 | 0.68* (0.67-0.69) |
| Race and ethnicity | ||||
| White, non-Hispanic | 3.5 | 1 [Ref] | 23.5 | 1 [Ref] |
| Black, non-Hispanic | 3.1 | 0.85 (0.83-0.87) | 24.6 | 0.97 (0.96-0.98) |
| Other race, non-Hispanic | 4.4 | 0.99x (0.95-1.03) | 32.0 | 1.21 (1.18-1.25) |
| Hispanic | 4.3 | 1.08 (1.05-1.11) | 30.0 | 1.16 (1.14-1.18) |
| Rurality | ||||
| Urban | 3.2 | 1 [Ref] | 26.0 | 1 [Ref] |
| Rural | 4.2 | 1.36 (1.34-1.39) | 21.7 | 0.88 (0.87-0.89) |
| Highly rural | 6.3 | 2.28* (2.13-2.44) | 21.0 | 0.92x (0.87-0.98) |
| Marital status | ||||
| Married | 3.7 | 1 [Ref] | 24.8 | 1 [Ref] |
| Never married | 3.7 | 0.87 (0.85-0.88) | 27.5 | 0.93 (0.92-0.95) |
| Divorced, separated, widowed | 3.2 | 0.87 (0.86-0.89) | 22.8 | 0.92 (0.91-0.94) |
| VA disability rating | ||||
| < 50% disability rating | 2.8 | 1 [Ref] | 24.3 | 1 [Ref] |
| ≥ 50% disability rating | 4.5 | 1.21 (1.19-1.24) | 26.7 | 1.05 (1.03-1.06) |
| Anxiety Disorder | ||||
| No anxiety disorder | 2.8 | 1 [Ref] | 24.2 | 1 [Ref] |
| Anxiety disorder | 5.2 | 1.38 (1.36-1.41) | 26.1 | 1.03 (1.02-1.04) |
| Bipolar Disorder | ||||
| No bipolar disorder | 3.4 | 1 [Ref] | 24.9 | 1 [Ref] |
| Bipolar disorder | 5.3 | 1.39 (1.35-1.43) | 24.2 | 1.00x (0.98-1.02) |
| Depressive Disorder | ||||
| No depressive disorder | 2.5 | 1 [Ref] | 24.5 | 1 [Ref] |
| Depressive disorder | 4.9 | 1.55* (1.52-1.58) | 25.2 | 1.06 (1.05-1.07) |
| PTSD | ||||
| No PTSD | 2.4 | 1 [Ref] | 24.3 | 1 [Ref] |
| PTSD | 5.3 | 1.70* (1.67-1.73) | 25.6 | 1.01x (0.99-1.02) |
| Schizophrenia | ||||
| No schizophrenia | 3.5 | 1 [Ref] | 25.3 | 1 [Ref] |
| Schizophrenia | 2.8 | 1.00x (0.96-1.06) | 16.2 | 0.69* (0.67-0.71) |
| Substance use disorder | ||||
| No substance use disorder | 3.4 | 1 [Ref] | 25.1 | 1 [Ref] |
| Substance use disorder | 4.0 | 0.98x (0.96-1.00) | 24.1 | 1.00x (0.98-1.01) |
| MH hospitalization | ||||
| No hospitalization history | 3.4 | 1 [Ref] | 24.8 | 1 [Ref] |
| Hospitalization history | 5.1 | 0.99x (0.95-1.02) | 25.7 | 1.09 (1.07-1.12) |
Adjusted OR >1.43 or <.70, indicating a Cohen’s d ≥ .2 (at least a small effect size; Chinn, 2000)
p > .05. All other adjusted ORs have p < .01.
Pre COVID-19 period: 10/1/17- 3/10/20. COVID-19 period: 3/11/20- 7/10/20.
ADI: Area Deprivation Index; least disadvantaged tercile= 1-33, middle tercile= 34-66, most disadvantaged tercile 67-100.
Older age (50+) and low socioeconomic status was associated with lower odds of having ≥50% of visits via video versus by phone or in-person during the COVID period (aORs < 0.68, Table 3). Female sex was associated with higher odds of having ≥50% of visits via video versus in-person (aOR= 1.64). Patients with a schizophrenia diagnosis and/or history of MH hospitalization were less likely to complete ≥50% of visits via phone or video versus in-person (aORs < 0.64).
Table 3.
Adjusted odds of patient visits occurring mostly via phone, video, or in-person during the COVID-19 period
| Patients with visits during COVID-19 period | ||||||
|---|---|---|---|---|---|---|
| Factor | % with ≥ 50% phone visits |
% with ≥ 50% video visits |
% with ≥ 50% in- person visits |
Adjusted OR of ≥50% video visits vs. phone (95% CI) |
Adjusted OR of ≥50% video visits vs. in- person (95% CI) |
Adjusted OR of ≥50% phone visits vs. in- person (95% CI) |
| Age | ||||||
| <30 | 57.6 | 25.1 | 17.3 | 1 [Ref] | 1 [Ref] | 1 [Ref] |
| 30-49 | 59.5 | 23.2 | 17.3 | 0.94 (0.91-0.96) | 0.91 (0.87-0.94) | 0.97 (0.94-1.00) |
| 50-64 | 65.4 | 15.5 | 19.1 | 0.65* (0.63-0.67) | 0.68* (0.65-0.71) | 1.04 (1.00-1.07) |
| 65-79 | 70.4 | 9.9 | 19.7 | 0.37* (0.36-0.38) | 0.37* (0.36-0.39) | 1.00 (0.96-1.03) |
| 80+ | 65.0 | 8.7 | 26.3 | 0.33* (0.31-0.35) | 0.23* (0.21-0.25) | 0.69* (0.65-0.72) |
| Sex | ||||||
| Male | 65.3 | 15.3 | 19.5 | 1 [Ref] | 1 [Ref] | 1 [Ref] |
| Female | 60.4 | 24.6 | 15.0 | 1.41 (1.38-1.43) | 1.64* (1.60-1.68) | 1.17 (1.15-1.19) |
| Socioeconomic status (ADI) | ||||||
| Least disadvantaged tercile | 61.5 | 20.6 | 17.9 | 1 [Ref] | 1 [Ref] | 1 [Ref] |
| Middle tercile | 65.9 | 16.2 | 17.9 | 0.75 (0.74-0.76) | 0.78x (0.77-0.80) | 1.04 (1.03-1.06) |
| Most disadvantaged tercile | 66.4 | 12.8 | 20.8 | 0.64* (0.63-0.65) | 0.62* (0.60-0.63) | 0.96 (0.94-0.97) |
| Race and ethnicity | ||||||
| White, non-Hispanic | 65.8 | 16.0 | 18.3 | 1 [Ref] | 1 [Ref] | 1 [Ref] |
| Black, non-Hispanic | 62.8 | 16.2 | 21.0 | 0.97 (0.96-0.99) | 0.86 (0.84-0.88) | 0.88 (0.87-0.89) |
| Other race, non-Hispanic | 59.1 | 23.3 | 17.6 | 1.27 (1.23-1.31) | 1.18 (1.13-1.23) | 0.93x (0.90-0.96) |
| Hispanic | 62.5 | 20.2 | 17.2 | 1.15 (1.13-1.18) | 1.09 (1.06-1.12) | 0.95x (0.93-0.97) |
| Rurality | ||||||
| Urban | 63.1 | 17.5 | 19.5 | 1 [Ref] | 1 [Ref] | 1 [Ref] |
| Rural | 67.9 | 14.8 | 17.3 | 0.88 (0.86-0.89) | 1.00 (0.98-1.02) | 1.14x (1.12-1.16) |
| Highly rural | 68.5 | 15.4 | 16.1 | 1.01 (0.94-1.09) | 1.24 (1.13-1.36) | 1.22 (1.14-1.31) |
| Marital status | ||||||
| Married | 65.4 | 17.6 | 17.0 | 1 [Ref] | 1 [Ref] | 1 [Ref] |
| Never married | 61.2 | 17.8 | 21.0 | 0.90 (0.88-0.91) | 0.82 (0.8-0.84) | 0.91 (0.90-0.93) |
| Divorced, separated, widowed | 65.2 | 14.5 | 20.3 | 0.89 (0.88-0.90) | 0.83 (0.81-0.85) | 0.93 (0.92-0.94) |
| VA disability rating | ||||||
| < 50% disability rating | 64.4 | 16.4 | 19.2 | 1 [Ref] | 1 [Ref] | 1 [Ref] |
| ≥ 50% disability rating | 64.2 | 18.2 | 17.5 | 1.05 (1.03-1.07) | 1.07 (1.05-1.10) | 1.02 (1.00-1.04) |
| Anxiety Disorder | ||||||
| No anxiety disorder | 63.9 | 16.7 | 19.3 | 1 [Ref] | 1 [Ref] | 1 [Ref] |
| Anxiety disorder | 65.0 | 17.0 | 17.9 | 1.00x (0.98-1.01) | 1.03 (1.02-1.05) | 1.04 (1.03-1.06) |
| Bipolar Disorder | ||||||
| No bipolar disorder | 64.3 | 17.2 | 18.5 | 1 [Ref] | 1 [Ref] | 1 [Ref] |
| Bipolar disorder | 64.4 | 13.5 | 22.1 | 0.89 (0.87-0.91) | 0.89 (0.86-0.91) | 1.00x (0.98-1.02) |
| Depressive Disorder | ||||||
| No depressive disorder | 63.1 | 17.1 | 19.8 | 1 [Ref] | 1 [Ref] | 1 [Ref] |
| Depressive disorder | 65.6 | 16.5 | 17.8 | 1.00x (0.99-1.02) | 1.10 (1.08-1.12) | 1.10 (1.09-1.12) |
| PTSD | ||||||
| No PTSD | 63.1 | 16.7 | 20.2 | 1 [Ref] | 1 [Ref] | 1 [Ref] |
| PTSD | 65.8 | 17.0 | 17.2 | 0.96 (0.94-0.97) | 1.11 (1.09-1.13) | 1.16 (1.15-1.18) |
| Schizophrenia | ||||||
| No schizophrenia | 64.5 | 17.4 | 18.1 | 1 [Ref] | 1 [Ref] | 1 [Ref] |
| Schizophrenia | 61.0 | 6.9 | 32.2 | 0.56* (0.54-0.59) | 0.36* (0.34-0.37) | 0.64* (0.62-0.65) |
| Substance use disorder | ||||||
| No substance use disorder | 64.4 | 17.9 | 17.7 | 1 [Ref] | 1 [Ref] | 1 [Ref] |
| Substance use disorder | 64.0 | 13.3 | 22.7 | 0.86 (0.84-0.87) | 0.75 (0.73-0.76) | 0.87 (0.86-0.89) |
| MH hospitalization | ||||||
| No hospitalization history | 64.7 | 17.3 | 18.1 | 1 [Ref] | 1 [Ref] | 1 [Ref] |
| Hospitalization history | 58.9 | 11.3 | 29.8 | 0.88 (0.86-0.91) | 0.56* (0.54-0.58) | 0.62* (0.61-0.64) |
OR >1.43 or <.70, indicating a Cohen’s d ≥ .2 (at least a small effect size; Chinn, 2000)
p > .05. All other adjusted ORs have p < .01.
COVID-19 period: 3/11/20- 7/10/20.
ADI: Area Deprivation Index; least disadvantaged tercile= 1-33, middle tercile= 34-66, most disadvantaged tercile 67-100.
Provider findings
Providers conducted an average of 1471.4 (SD 1629.6) visits during the 29-month pre-COVID period and 217.7 (SD 193.2) visits during the 4-month COVID period. The percentage of providers who had completed at least one video visit was 37.7% in the pre-COVID sample and 63.1% in the COVID sample. 40.0% of those with video experience in the COVID period were first-time users. In the pre-COVID sample, most providers conducted ≥50% of their visits in-person (94.5%; 5.0% conducted most by phone, 0.5% conducted most by video). In the COVID sample, it was most common for providers to conduct the majority of their visits via phone (46.3%; 32.7% conducted most in-person, 14.1% conducted most by video). See Supplementary Table 2 for analyses comparing pre-COVID video users to first-time video users during the COVID period.
During the pre-COVID and COVID periods, psychiatrists, other medical providers, and other non-medical providers had lower odds of completing at least one video visit compared to psychologists (aORs < 0.44, Table 4). During the COVID period, these groups also had lower odds of completing ≥50% of visits via video versus by phone or in-person as compared to psychologists (aORs < 0.33, Table 5). Providers who were 65+ had lower odds of completing the majority of their visits via video versus by phone or in-person as compared to younger providers during the COVID period (aORs < 0.69); the same pattern held for providers aged 50-64 but did not meet the current aOR cutoff of < .70 (aORs < .75). Psychiatrists and other medical providers had lower odds of completing ≥50% of visits via phone versus in-person as compared to psychologists (aORs < 0.53).
Table 4.
Adjusted odds of providers having any video experience prior to and during COVID-19
| Factor | % with ≥ 1 video visit prior to COVID-19 |
Adjusted OR (95% CI) |
% with ≥ 1 video visit during COVID-19 |
Adjusted OR (95% CI) |
|---|---|---|---|---|
| Provider type | ||||
| Psychologist | 55.7 | 1 [Ref] | 84.8 | 1 [Ref] |
| Psychiatrist | 29.2 | 0.33* (0.30-0.36) | 56.2 | 0.22* (0.20-0.25) |
| Other non-medical provider | 36.9 | 0.44* (0.41-0.48) | 63.1 | 0.30* (0.27-0.33) |
| Other medical provider | 29.7 | 0.32* (0.29-0.35) | 45.6 | 0.14* (0.13-0.16) |
| Age | ||||
| 20-49 | 36.2 | 1 [Ref] | 62.8 | 1 [Ref] |
| 50-64 | 38.4 | 1.26 (1.18-1.35) | 60.1 | 1.11 (1.03-1.20) |
| 65+ | 29.1 | 0.79 (0.71-0.87) | 55.8 | 0.84 (0.74-0.96) |
| Sex | ||||
| Male | 33.6 | 1 [Ref] | 58.0 | 1 [Ref] |
| Female | 37.9 | 1.19 (1.12-1.27) | 61.5 | 1.18 (1.10-1.27) |
OR >1.43 or <.70, indicating a Cohen’s d ≥ .2 (at least a small effect size; Chinn, 2000)
All adjusted ORs have p < .01.
Pre COVID-19 period: 10/1/17- 3/10/20. COVID-19 period: 3/11/20- 7/10/20.
Table 5.
Adjusted odds of providers conducting ≥50% of visits via phone, video, or in-person during COVID-19
| Visits occurring during COVID-19 | ||||||
|---|---|---|---|---|---|---|
| Factor | % with ≥ 50% phone visits |
% with ≥ 50% video visits |
% with ≥ 50% in- person visits |
Adjusted OR of ≥50% video visits vs. phone (95% CI) |
Adjusted OR of ≥50% video visits vs. in- person (95% CI) |
Adjusted OR of ≥50% phone visits vs. in-person (95% CI) |
| Provider type | ||||||
| Psychologist | 42.0 | 36.6 | 21.4 | 1 [Ref] | 1 [Ref] | 1 [Ref] |
| Psychiatrist | 47.9 | 6.2 | 45.9 | 0.16* (0.14-0.20) | 0.08* (0.07-0.10) | 0.53* (0.47-0.59) |
| Other non-medical provider | 58.3 | 15.1 | 26.7 | 0.32* (0.28-0.36) | 0.33* (0.29-0.38) | 1.04x (0.93-1.17) |
| Other medical provider | 51.8 | 3.5 | 44.7 | 0.09* (0.07-0.11) | 0.05* (0.04-0.06) | 0.51* (0.45-0.58) |
| Age | ||||||
| 20-49 | 47.7 | 17.4 | 34.9 | 1 [Ref] | 1 [Ref] | 1 [Ref] |
| 50-64 | 53.1 | 10.5 | 36.4 | 0.70 (0.62-0.79) | 0.75 (0.66-0.86) | 1.11 (1.02-1.21) |
| 65+ | 51.6 | 10.1 | 38.4 | 0.69* (0.55-0.87) | 0.61* (0.48-0.77) | 0.99x (0.86-1.14) |
| Sex | ||||||
| Male | 48.1 | 12.1 | 39.8 | 1 [Ref] | 1 [Ref] | 1 [Ref] |
| Female | 51.3 | 15.0 | 33.7 | 1.12x (0.99-1.26) | 1.36 (1.20-1.53) | 1.24 (1.15-1.35) |
OR >1.43 or <.70, indicating a Cohen’s d ≥ .2 (at least a small effect size; Chinn, 2000)
p > .05. All other adjusted ORs have p < .01.
COVID-19 period: 3/11/20- 7/10/20.
Discussion
The current analyses found substantial differences in use of phone, video, and in-person MH services both before and during COVID-19 within a large national sample of VA patients and providers. Rates of TMH use overall, and of video visits in particular, increased dramatically during the COVID-19 pandemic, with a more than seven-fold increase in the percentage of patients who had completed at least one appointment via video in the COVID sample as compared to the pre-COVID sample. These higher rates of video experience in the COVID sample versus the pre-COVID sample were less pronounced among providers (1.5x increase), in part because VA had encouraged MH providers’ use of video telehealth prior to the onset of the pandemic (Connolly et al., 2020a; Rosen et al., 2020). There was a massive shift in the modality through which patients and providers engaged in most of their MH care; over 90% of patients and providers in the pre-COVID sample participated in the majority of MH care in person, while within the COVID sample, the most common mode of care delivery was phone.
Results confirm the existence of a digital divide, such that older Veterans were significantly less likely to have video MH appointments both prior to and during the COVID-19 pandemic. Lower income Veterans were also less likely to receive MH care via video during COVID-19. While previous research within civilian populations has found Black and Hispanic patients to have less access to video appointments (Chunara et al., 2020; Lam et al., 2020), there were no significant differences in modality use based on race or ethnicity in the current VA sample; Black and Hispanic Veterans had comparable and at times slightly higher rates of video utilization as compared to White Veterans in our analyses. This finding is in keeping with a growing body of research demonstrating that minority Veterans tend to have better health outcomes compared to the civilian minority population; it is thought that VA’s equal-access healthcare system may eliminate some of the structural barriers to care experienced by minorities in the private sector (Freeman et al., 2017; Riviere et al., 2020). However, it remains the case that age and socioeconomic status were barriers to video use within the current sample. VA has made significant strides to equalize access to video care, including by distributing thousands of video and internet-enabled tablets to Veterans without devices and expanding availability of pre-appointment test calls (Heyworth et al., 2020; Zulman et al., 2019). However, additional work is needed to identify remaining barriers to video use (e.g., patient hesitance to try video appointments, provider unawareness of the tablet ordering program). Targeted training and information sharing to both patients and providers may help to overcome these hurdles to access.
Within the pre-COVID sample, Veterans in highly rural locations and those with diagnoses of PTSD or depression were more likely to have video telehealth experience. These variables were not predictive of video use within the COVID-19 sample. These findings indicate some degree of an equalizing effect of COVID-19 regarding which MH patients were offered and agreed to video telehealth. For instance, while living in a highly rural location was a predictor of video use in the pre-COVID sample likely due to difficulties traveling to the medical center, during COVID-19, patients were strongly encouraged to use video or phone versus in-person care regardless of where they lived; indeed, a greater percentage of first-time video users during COVID-19 were from urban locations. Female Veterans were more likely to receive video care both prior to and during the COVID period. While it is true that female Veterans are significantly younger than male Veterans on average, this finding was significant when controlling for age. Other research has reported a similar effect of women engaging in video telehealth at higher rates than men (Lewis et al., 2020); qualitative research is needed to understand what may be contributing to this finding of sex differences. Patients with more severe clinical presentations (those with schizophrenia and a past history of MH hospitalization) were more likely to receive most of their care in-person during COVID-19 versus by video or phone. This result aligns with reports of provider hesitancy around seeing higher risk patients from a distance; for instance, they may feel less comfortable coordinating a patient’s hospitalization when not with them in the same room (Connolly et al., 2020b; Feijt et al., 2020; Gilmore et al., 2019). To our knowledge, there is no research to date that compares quality outcomes when treating higher risk patients via TMH versus in-person; such information will be critical in helping providers determine whether these patients indeed need to be prioritized for in-person care or could be managed well from a distance.
Regarding providers, there was a stark difference in the extent of video use based on discipline, with psychologists having considerably higher video experience compared to psychiatrists, other medical providers (e.g., nurse practitioners), and other non-medical providers (e.g., social workers). This finding held when examining providers with any video experience both pre-COVID and during the pandemic, as well as when examining those who completed the majority of their visits via video during COVID-19. These latter percentages highlight the most extreme difference between disciplines; 37% of psychologists completed the majority of their visits via video during COVID-19, compared to 15% of other non-medical providers, 6% of psychiatrists, and 3% of other medical providers. It is likely that psychologists are unique in that they typically have the smallest caseloads of patients whom they see more frequently (e.g., 2-4 times per month) for the longest appointment times (e.g., for one hour as opposed to thirty minutes; Pingitore et al., 2002). These factors may increase the relative value of seeing their patients versus only hearing them and may allow for more time to assist with appointment set-up and troubleshoot technological difficulties. However, these are merely hypotheses and must be confirmed by more nuanced analyses conducted with providers across a variety of MH disciplines. If these hypotheses are supported, it may suggest a need for additional provider support surrounding video appointments (e.g., having support staff conduct test calls with patients in advance of sessions to troubleshoot and therefore preserve time for clinical care; Rosen et al., 2020). It is also important to note that medical MH providers saw a greater percentage of their patients in person during COVID-19, possibly due to unique components of the treatment they provide that may increase the relative value of in-person care (e.g., assessing extrapyramidal side effects of prescribed antipsychotic medications). Finally, older providers had lower odds of completing most of their visits via video versus by phone or in-person during COVID-19. These findings again speak to the existence of a digital divide, in that older providers may have less experience and comfort navigating video telehealth technologies and may benefit from increased support and training to gain confidence using this modality (Czaja et al, 2009).
Strengths of this work include its large national sample of millions of MH patients and thousands of providers, examining TMH use from 2017 through 2020 across a range of demographic and clinical predictors. Limitations include its restriction to a VA sample that is not subject to the same licensure and insurance regulations experienced in the private sector that could influence TMH use rates (Barnett et al., 2018; Choi et al., 2019; Shi et al., 2019). Further research, including qualitative work, is necessary to better contextualize the demographic and clinical differences reported above. This work will help to identify barriers and facilitators to TMH use in order to improve access to care. More nuanced analyses examining potential regional differences in TMH use, as well as disorder-specific differences (e.g., type of anxiety or substance use disorder, history of suicidality), are warranted. Employing a standardized measure of patient risk, such as a comorbidity score, will also be important to further examine whether more severe patients are less likely to be treated remotely. It should also be noted that while there was over 90% overlap between the patients and providers included within the pre-COVID and COVID samples, the fact remains that these samples were not identical over the two time periods, which introduces additional variability to analyses. All observed differences between the pre-COVID and COVID periods should be interpreted with the understanding that the included samples are not identical in composition.
Additional research is needed to compare the relative clinical effectiveness (i.e., effect on health outcomes) of phone, video, and in-person care to ensure that patients receive care via the modality that is most appropriate for their individual needs (Rosen et al., 2020). Indeed, given that phone care has significantly fewer barriers to use, it will be important to determine the effectiveness of this mode of care delivery as it may be a much more feasible option for certain patients. Research examining patient modality preference will also be critical. A largescale survey conducted during COVID-19 found that patients receiving MH care via video were more likely to want to continue remote treatment in the future as opposed to those receiving care via phone (Guinart et al., 2020). However, some prior work has shown older patients tend to prefer lower-tech treatment options (Gould et al., 2020), which may be an important factor influencing observed lower rates of video use by older patients in the current sample. Finally, it is important to remember that phone use increased in part due to temporary increases in workload credit granted during the pandemic. If credit were to decrease in the future, phone use rates may dramatically change. It is therefore critical for future research to examine the interplay between the relative advantages of phone care and reimbursement policies (Jaklevic, 2020).
The COVID-19 pandemic led to an unprecedented shift to virtual MH care to protect patients and providers from infection. While rates of in-person care are sure to rise as the pandemic subsides, TMH is likely to remain an attractive mode of care delivery for many patients and providers, given its ability to substantially increase access to care. It is therefore critical to identify and address barriers to TMH use, to ensure that all patients can receive high quality MH care when and where they need it.
Supplementary Material
Public Significance Statement:
This national study within the Department of Veterans Affairs found significant differences in the use of video, phone, and in-person mental health care both prior to and during the COVID-19 pandemic. Findings support the existence of a digital divide, such that older and lower income patients were less likely to receive video care. Older providers and non-psychologists were less likely to provide video care.
Acknowledgements:
Samantha L. Connolly was supported by a VISN 1 Career Development Award, Department of Veterans Affairs, Veterans Health Administration.
Content is solely the responsibility of the authors and does not necessarily represent the official views of the U.S. Department of Veterans Affairs or the U.S. Government.
Footnotes
Disclosures: The authors declare no conflicts of interest.
VA set a national operational goal of at least 45% of MH providers completing at least one video visit by the end of September 2019. MH providers were defined as those who had completed > 20 MH outpatient encounters within the 30 days prior to the reporting date; therefore, included providers fluctuated from month to month when calculating this metric, which is intended to be used primarily for operational and not research purposes. Of note, VA had surpassed this 45% goal by the end of September 2019, with 62.4% of included providers having completed at least one video visit.
We decided on the metric of >=100 VA appointments by examining the distribution of appointments completed by VA MH providers over our study time period. We found that 39.89% of MH providers had <= 10 visits logged over the period, and that 13.8% had between 11-99 visits. We therefore determined that the cutpoint of 100 visits was both clinically meaningful, in representing at least about 1 MH visit per week on average, while also retaining a significant portion of the overall provider sample (46.31%).
References
- Acierno R, Gros DF, Ruggiero KJ, et al. (2016). Behavioral activation and therapeutic exposure for posttraumatic stress disorder: a noninferiority trial of treatment delivered in person versus home-based telehealth. Depression and Anxiety. 33(5):415–423. [DOI] [PubMed] [Google Scholar]
- Barnett ML, Ray KN, Souza J, Mehrotra A. (2018). Trends in telemedicine use in a large commercially insured population, 2005-2017. JAMA.320(20):2147–2149. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Castro A, Gili M, Ricci-Cabello I, et al. (2020). Effectiveness and adherence of telephone-administered psychotherapy for depression: A systematic review and meta-analysis. Journal of Affective Disorders. 260:514–526. [DOI] [PubMed] [Google Scholar]
- Chinn S (2000). A simple method for converting an odds ratio to effect size for use in meta-analysis. Statistics in medicine. 19(22):3127–3131. [DOI] [PubMed] [Google Scholar]
- Choi S, Wilcock AD, Busch AB, et al. (2019). Association of Characteristics of Psychiatrists With Use of Telemental Health Visits in the Medicare Population. JAMA Psychiatry. 76(6), 654–657. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Chunara R, Zhao Y, Chen J, et al. (2020). Telemedicine and Healthcare Disparities: A cohort study in a large healthcare system in New York City during COVID-19. Journal of the American Medical Informatics Association. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Connolly SL, Stolzmann KL, Heyworth L, et al. (2020a). Rapid Increase in Telemental Health Within the Department of Veterans Affairs During the COVID-19 Pandemic. Telemedicine and e-Health. [DOI] [PubMed] [Google Scholar]
- Connolly SL, Miller CJ, Lindsay JA, Bauer MS. (2020b). A systematic review of providers’ attitudes toward telemental health via videoconferencing. Clinical Psychology: Science and Practice. e12311. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Cowan KE, McKean AJ, Gentry MT, Hilty DM. (2019). Barriers to Use of Telepsychiatry: Clinicians as Gatekeepers. Mayo Clinic Proceedings. 94(12): 2510–2523. [DOI] [PubMed] [Google Scholar]
- Czaja SJ, Lee CC. (2009). Information technology and older adults. Human-Computer Interaction: CRC Press. 35–50. [Google Scholar]
- Eberly LA, Kallan MJ, Julien HM, et al. (2020). Patient Characteristics Associated With Telemedicine Access for Primary and Specialty Ambulatory Care During the COVID-19 Pandemic. JAMA Network Open. 3(12):e2031640–e2031640. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Egede LE, Acierno R, Knapp RG, et al. (2015). Psychotherapy for depression in older veterans via telemedicine: a randomised, open-label, non-inferiority trial. Lancet Psychiatry. 2(8):693–701. [DOI] [PubMed] [Google Scholar]
- Feijt M, de Kort Y, Bongers I, Bierbooms J, Westerink J, W IJ. (2020). Mental Health Care Goes Online: Practitioners' Experiences of Providing Mental Health Care During the COVID-19 Pandemic. Cyberpsychology, Behavior, and Social Networking. [DOI] [PubMed] [Google Scholar]
- Fletcher TL, Hogan JB, Keegan F, et al. (2018). Recent Advances in Delivering Mental Health Treatment via Video to Home. Current psychiatry reports. 20(8):56. [DOI] [PubMed] [Google Scholar]
- Freeman MA, Pleis JR, Bornemann KR, et al. (2017). Has the Department of Veterans Affairs Found a Way to Avoid Racial Disparities in the Evaluation Process for Kidney Transplantation? Transplantation. 101(6):1191–1199. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Gilmore AK, Ward-Ciesielski EF. (2019). Journal of Telemedicine and Telecare. Perceived risks and use of psychotherapy via telemedicine for patients at risk for suicide. 25(1):59–63. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Gould CE, Loup J, Kuhn E, Beaudreau SA, Ma F, Goldstein MK, … & O'Hara R (2020). Technology use and preferences for mental health self-management interventions among older veterans. International journal of geriatric psychiatry, 35(3), 321–330. [DOI] [PubMed] [Google Scholar]
- Grady B, Myers KM, Nelson EL, et al. (2011). Evidence-based practice for telemental health.Telemedicine and e-health. 17(2):131–148. [DOI] [PubMed] [Google Scholar]
- Gray DM, Joseph JJ, Olayiwola JN. (2020). Strategies for Digital Care of Vulnerable Patients in a COVID-19 World—Keeping in Touch. JAMA Health Forum. [DOI] [PubMed] [Google Scholar]
- Guinart D, Marcy P, Hauser M, Dwyer M, & Kane JM (2020). Patient Attitudes Toward Telepsychiatry During the COVID-19 Pandemic: A Nationwide, Multisite Survey. JMIR mental health, 7(12), e24761. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Heyworth L, Kirsh S, Zulman D, Ferguson JM, Kizer KW. (2020). Expanding Access through Virtual Care:The VA’s Early Experience with COVID-19. NEJM Catalyst Innovations in Care Delivery. 1(4). [Google Scholar]
- Jaklevic MC. (2020). Telephone Visits Surge During the Pandemic, but Will They Last? JAMA. [DOI] [PubMed] [Google Scholar]
- Kim SS, Darwish S, Lee SA, et al. (2018). A randomized controlled pilot trial of a smoking cessation intervention for US women living with HIV: telephone-based video call vs voice call. International Journal of Womens Health. 10:545–555. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Lam K, Lu AD, Shi Y, Covinsky KE. (2020). Assessing Telemedicine Unreadiness Among Older Adults in the United States During the COVID-19 Pandemic. JAMA Internal Medicine. 180(10):1389–1391. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Lewis AK, Harding KE, Taylor NF, O’brien TJ, Carney PW. (2020). The feasibility of delivering first suspected seizure care using telehealth: a mixed methods controlled study. Epilepsy Research. 106520. [DOI] [PubMed] [Google Scholar]
- Liebmann EP, Preacher KJ, Richter KP, et al. (2019). Identifying pathways to quitting smoking via telemedicine-delivered care. Health Psychology. 38(7):638–647. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Lovell K, Cox D, Haddock G, et al. (2006). Telephone administered cognitive behaviour therapy for treatment of obsessive compulsive disorder: randomised controlled non-inferiority trial. BMJ. 333(7574):883. [DOI] [PMC free article] [PubMed] [Google Scholar]
- McClellan MJ, Osbaldiston R, Wu R, Yeager R, Monroe AD, McQueen T, & Dunlap MH (2021). The effectiveness of telepsychology with veterans: A meta-analysis of services delivered by videoconference and phone. Psychological Services. Online ahead of print. [DOI] [PubMed] [Google Scholar]
- Mohr DC, Ho J, Duffecy J, et al. (2012). Effect of telephone-administered vs face-to-face cognitive behavioral therapy on adherence to therapy and depression outcomes among primary care patients: a randomized trial. JAMA. 307(21):2278–2285. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Morland LA, Mackintosh MA, Glassman LH, et al. (2020). Home-based delivery of variable length prolonged exposure therapy: A comparison of clinical efficacy between service modalities. Depression and Anxiety. 37(4):346–355. [DOI] [PubMed] [Google Scholar]
- Morland LA, Mackintosh MA, Rosen CS, et al. (2015). Telemedicine versus in-person delivery of cognitive processing therapy for women with posttraumatic stress disorder: a randomized noninferiority trial. Depression and Anxiety. 32(11):811–820. [DOI] [PubMed] [Google Scholar]
- Pingitore DP, Scheffler RM, Sentell T, West JC. (2002). Comparison of psychiatrists and psychologists in clinical practice. Psychiatric Services. 53(8):977–983. [DOI] [PubMed] [Google Scholar]
- Price LE, Shea K, & Gephart S (2015). The Veterans Affairs's corporate data Warehouse: uses and implications for nursing research and practice. Nursing Administration Quarterly, 39(4), 311–318. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Richter KP, Shireman TI, Ellerbeck EF, et al. (2015). Comparative and cost effectiveness of telemedicine versus telephone counseling for smoking cessation. Journal of Medical Internet Research. 17(5):e113. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Riviere P, Luterstein E, Kumar A, et al. (2020). Survival of African American and non-Hispanic white men with prostate cancer in an equal-access health care system. Cancer. 126(8):1683–1690. [DOI] [PubMed] [Google Scholar]
- Rodriguez JA, Clark CR, Bates DW. (2020). Digital Health Equity as a Necessity in the 21st Century Cures Act Era. JAMA. [DOI] [PubMed] [Google Scholar]
- Rosen CS, Morland LA, Glassman LH, et al. (2020). Virtual mental health care in the Veterans Health Administration’s immediate response to coronavirus disease-19. American Psychologist. [DOI] [PubMed] [Google Scholar]
- Rural Health Research Center. Using RUCA data. [Web Page]. 2020; http://depts.washington.edu/uwruca/ruca-uses.php. Accessed 11/20/2020.
- Shi Z, Huskamp HA, Souza J, Busch AB, Uscher-Pines L, Mehrotra A. (2019). Characteristics of Organizations That Provide Telemental Health. Healthcare Transformation. 11–13. [Google Scholar]
- Turner CM, Mataix-Cols D, Lovell K, et al. (2014). Telephone cognitive-behavioral therapy for adolescents with obsessive-compulsive disorder: a randomized controlled non-inferiority trial. Journal of the American Academy of Child and Adolescent Psychiatry. 53(12):1298–1307 e1292. [DOI] [PMC free article] [PubMed] [Google Scholar]
- University of Wisconsin Department of Medicine. Neighborhood Atlas. [Web Page]. 2020; https://www.neighborhoodatlas.medicine.wisc.edu/. Accessed 11/20/2020.
- US Department of Veterans Affairs. VA Video Connect. [Web Page]. 2020; https://mobile.va.gov/app/va-video-connect. Accessed 11/20/2020.
- Uscher-Pines L, Sousa J, Raja P, Mehrotra A, Barnett M, Huskamp HA. (2020). Treatment of opioid use disorder during COVID-19: Experiences of clinicians transitioning to telemedicine. Journal of Substance Abuse Treatment. 118:108124. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Varker T, Brand RM, Ward J, Terhaag S, Phelps A. (2019). Efficacy of synchronous telepsychology interventions for people with anxiety, depression, posttraumatic stress disorder, and adjustment disorder: A rapid evidence assessment. Psychological Services. 16(4):621. [DOI] [PubMed] [Google Scholar]
- Whitten PS, Mackert MS. (2005). Addressing telehealth's foremost barrier: provider as initial gatekeeper. International Journal of Technology Assessment in Health Care. 21(4):517–521. [DOI] [PubMed] [Google Scholar]
- Zulman DM, Wong EP, Slightam C, et al. (2019). Making connections: nationwide implementation of video telehealth tablets to address access barriers in veterans. JAMIA Open. 2(3):323–329. [DOI] [PMC free article] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.