Abstract
Background:
As a result of the COVID-19 public health emergency (PHE), telehealth utilization accelerated to facilitate health care management and minimize risk. However, those with mental health conditions and substance use disorders (SUD)—who represent a vulnerable population, and members of underrepresented minorities (e.g., rural, racial/ethnic minorities, the elderly)—may not benefit from telehealth equally.
Objective:
To evaluate health equality in clinical effectiveness and utilization measures associated with telehealth for clinical management of mental health disorders and SUD to identify emerging patterns for underrepresented groups stratified by race/ethnicity, gender, age, rural status, insurance, sexual minorities, and social vulnerability.
Methods:
We performed a systematic review in PubMed, Embase, Cochrane Central Register of Controlled Trials, and CINAHL through November 2022. Studies included those with telehealth, COVID-19, health equity, and mental health or SUD treatment/care concepts. Our outcomes included general clinical measures, mental health or SUD clinical measures, and operational measures.
Results:
Of the 2,740 studies screened, 25 met eligibility criteria. The majority of studies (n = 20) evaluated telehealth for mental health conditions, while the remaining five studies evaluated telehealth for opioid use disorder/dependence. The most common study outcomes were utilization measures (n = 19) or demographic predictors of telehealth utilization (n = 3). Groups that consistently demonstrated less telehealth utilization during the PHE included rural residents, older populations, and Black/African American minorities.
Conclusions:
We observed evidence of inequities in telehealth utilization among several underrepresented groups. Future efforts should focus on measuring the contribution of utilization disparities on outcomes and strategies to mitigate disparities in implementation.
Keywords: telemedicine, telehealth, health inequities, systematic review, COVID-19, vulnerable populations, rural population, mental health, substance-related disorders, opiate substitution treatment
Introduction
The COVID-19 pandemic had considerable immediate impact on health inequality in health care availability and delivery,1–4 disruptions to clinical care,5 and exacerbation of underlying mental health conditions.6–9 Increasing anxiety, isolation, decreased access to mental health support and services, family and relationship considerations, food insecurity, and financial instability resulting from the pandemic all contributed to worsening mental health.5,10–14 Providing care for those with substance use disorder (SUD) also presented a challenge, as physical distancing, quarantine, and public health measures disrupted utilization of treatment and support services during the pandemic.15,16 Mental health- and SUD-related outcomes worsened across all demographic groups,13 with evidence of disparities among minorities and underrepresented groups.
Several studies reported declining mental health- and SUD-related care and outcomes from prepandemic times, but those adverse outcomes were particularly exacerbated among younger adults (ages 18–24),15 adults over 30 years,13 those of Hispanic ethnicity,13,17 non-Hispanic Blacks,15 rural residents,18–20 and other underserved communities.21
To overcome challenges and disruptions from the pandemic, telehealth—delivery of health care via digital communication technology—was rapidly integrated into health care delivery across many institutions and systems.22–26 With the rise of telehealth utilization as an alternative means of delivering care,27–29 some have questioned whether existing disparities were exacerbated through a digital divide (gap in access to information and communications technology) in underserved communities.30,31 The digital divide has been identified as a new social determinant of health that may contribute to worsening social and economic barriers. Therefore, it is necessary to evaluate the impact of telehealth during the COVID-19 pandemic on mental health and SUD care across these social determinants of health as indicators of health inequities.
Our objective was to evaluate clinical effectiveness and utilization associated with telehealth for clinical management of mental health and SUD and to identify emerging patterns for underrepresented groups stratified by race/ethnicity, gender, age, rural status, insurance, sexual minorities, and social vulnerability that would suggest health inequities. We aimed to understand whether telehealth influenced differences in utilization and outcomes across vulnerable groups during the pandemic.
Methods
OVERVIEW
We conducted a systematic review and report on that work here according to relevant elements from the Preferred Reporting Items in Systematic Reviews and Meta-analyses (PRISMA) guidelines,32 a standardized method for systematic reviews. The review protocol was registered in PROSPERO (CRD42022383956). No ethical approval was obtained because all data used for these analyses were published previously.
DATA SOURCES, SEARCH STRATEGY, AND SELECTION CRITERIA
We developed a search, including four domains: telehealth, COVID-19, health equity, and clinical conditions (i.e., mental health and SUD). The following databases were searched for eligible citations from inception to November 9, 2022: PubMed, Embase (Elseviers), Cochrane Central Register of Controlled Trials (Wiley), and CINAHL (EBSCO). Duplicates in the search results were removed using a combination of automated and manual methods. A health sciences librarian trained in systematic literature searching developed and executed the search strategies. We also searched Telehealth.HHS.gov33 and the Rural Health Research Gateway websites on November 29, 202234 for research products on relevant topics published any time after the beginning of the COVID-19 pandemic (i.e., 2020–2022). The full search strategies for all databases are available in Supplementary Appendix SA. No language limits were applied to any of the strategies.
We included study designs of randomized trials, nonrandomized trials, quasi-experimental designs (e.g., before-and-after analyses, difference-in-difference studies), observational cohort studies (with or without a nontelehealth control group), or case-control studies. We excluded studies on tobacco cessation only or neurocognitive conditions because they were out of scope (e.g., dementia, Alzheimer's disease, autism). Within health equity concepts, we excluded articles that did not report at least one subgroup finding (e.g., by race, gender, rural/urban designations), but we included studies based on any social or demographic factor as a focus of study. We excluded case series with no control group, qualitative studies only, narrative reviews, and commentaries/editorials.
EXPOSURES
We included all synchronous direct-to-consumer or provider-to-provider telehealth applications, but we excluded those that used only computer applications or artificial intelligence technology without a synchronous telehealth provider connection. We included studies in which synchronous health care was delivered remotely by telehealth (phone and/or video) and by a clinician (e.g., physician, nurse practitioner, therapist/counselor) in any clinical setting. Activities performed by telehealth could include prescribing, treating, diagnosing, and/or managing a patient who was not physically present in the same location administered within the context of longitudinal care. We assessed telehealth care delivery compared with (1) in-person treatment/services, (2) no treatment/services, or (3) a combination of in-person and telehealth visits. If there was no nontelehealth comparison group, we compared subgroups within the cohort (e.g., for race, the comparison of the outcome between a racial minority group and White participants).
OUTCOMES
We included several outcomes in this study:
General clinical measures (e.g., medication adherence, quality of life, treatment retention);
Mental health clinical measures (e.g., medication adherence, quality-of-life, treatment retention);
SUD clinical measures (e.g., medication adherence, quality of life, treatment retention); or
Operational measures (e.g., hospitalizations, hospital readmissions, emergency department/urgent care visits, utilization, availability).
STUDY SELECTION, DATA EXTRACTION, AND RISK-OF-BIAS ASSESSMENT
Two authors (L.L. and S.T.) independently screened all titles and abstracts of initial articles in the first stage, full-text review of articles retrieved in the second stage, and gray literature sources. They then extracted data from included studies, including study characteristics (e.g., study design, time frame, exposure definitions, and measurements). We used the Downs & Black checklist for the risk-of-bias assessment, which is applicable to randomized and nonrandomized studies.35 These criteria include reporting characteristics (e.g., clearly described objectives, study population, interventions/exposures, outcomes), external validity characteristics (e.g., representativeness, recruitment), and internal validity (e.g., blinding, data dredging presence, variability in follow-up time, confounding assessment). Disagreements between the review authors (L.L., S.T.) on study inclusion, the data elements captured, or the risk of bias were resolved by discussion by the expert panel committee of three study investigators (J.P.V., M.M.W., N.M.M.).
DATA ANALYSIS/SYNTHESIS
After reviewing the included articles, we decided that we could not statistically pool the results because of high clinical heterogeneity. We therefore provided a synthesis of the findings from the included systematic review structured by the type of intervention, target population characteristics, type of outcome, and intervention content. This approach included identifying differences in studies that reported outcomes stratified by at least one demographic subgroup, and by varying definitions of outcomes to address the heterogeneity of included studies. We also analyzed reasons for inconsistency in treatment effects across studies by evaluating differences in the study population, intervention, comparator, outcomes, missingness of data, and covariates assessed.
Results
OVERALL STUDY CHARACTERISTICS
Of the initial 2,740 references in our initial search, 25 met final inclusion criteria for the study (Fig. 1). The majority of excluded studies from the full-text review were due to design and/or publication type issues (e.g., qualitative studies only). One additional reason that several studies were excluded that would have otherwise met study criteria included those that provided global measures of association across an entire study component, with no underserved component specifically.36–38
Fig. 1.
PRISMA 2020 flow diagram of included studies. PRISMA, Preferred Reporting Items in Systematic Reviews and Meta-analyses.
Nearly all (n = 24) studies were observational in design (Table 1). Nine studies focused on Veterans specifically. All studies evaluated synchronous, direct-to-consumer video telehealth (n = 25); however, some studies included audio or text message-based synchronous telehealth interactions (n = 6).
Table 1.
Table of Evidence and Characteristics of Included Studies
| AUTHOR (YEAR) | STUDY DESIGN | SETTING | TIME PERIOD | CONDITIONS | POPULATION | EXPOSURE CHARACTERISTICS | OUTCOMES | KEY FINDINGS |
|---|---|---|---|---|---|---|---|---|
| Ainslie et al. (2022) | Observational study, pre-post design | New Hampshire CMHC | December 1, 2019 through June 30, 2020; additional analysis—December 1, 2018 through June 30, 2019 | SMIs (Schizophrenia, bipolar disorder, major depression, PTSD, anxiety disorders, and all other conditions) | Population that was Medicaid-eligible and received treatment from a CMHC | Telemedicine use | Telemedicine use dosage (low [<25%], medium [25–75%], or high [>75%]) | The integration of telemedicine supported care continuity for most CMHC patients; yet, retention varied by subpopulation, as did telemedicine utilization. |
| Campos-Castillo and Laestadius (2022) | Cross-sectional | US, online survey | March to May 2021 | Mental health care | 13–17 Years old from the National Opinion Research Center's AmeriSpeak Teen Panel | Telehealth use for mental health care (voice, video, text/chat, internet support group, privacy for telehealth use) | Telehealth visits (yes/no) | Text-based communication/chat was most prevalent among minoritized racial and ethnic groups. Parental support was positively associated with finding private space for telehealth visits. Black adolescents were less likely to report in-person visits. Among those unable to receive care, Black adolescents preferred in-person visits. |
| Chakawa et al. (2021) | Quasi experimental study | Pediatric primary care clinic within a children's hospital located in a Midwestern metropolitan city | April 2019 and October 2019; April 2020 and October 2020 | Externalizing issues, internalizing issues, developmental delay, medical concerns, obsessive/habitual behavior, feeding/elimination, and trauma/adjustment | Pediatric (<19 years) | Pediatric IPC services delivered via telehealth | Attendance for scheduled visits; referral concerns | Although telehealth has helped provide IPC continuity during COVID-19, findings from this study show troubling preliminary data regarding reduced attendance, increased internalizing concerns, and disparities in scheduling for Black patients. |
| Chang et al. (2022) | Quasi experimental study | REACH Medical is a low-threshold harm reduction medical practice located in Ithaca, New York | March and December 2020 | Opioid use disorder | Patients from nonurban integrated substance use disorder treatment site in New York utilizing tele-MOUD care | Tele-MOUD use | Predictors of majority televideo visits; predictors of new patient inductions | Telemedicine may increase access to MOUD, although certain patients may rely on different forms of telemedicine. |
| Connolly et al. (2022) | Quasi-experimental study, pre-post | Department of Veterans Affairs | October 1, 2017 to July 10, 2020 | Anxiety disorder, bipolar disorder, depression, PTSD, substance use disorder, or schizophrenia, and history of MH hospitalization | Veterans at VA MH outpatient clinics | Any video experience (delivered to the patient's home or another non-VA location) | Use (yes/no) | Findings demonstrate a digital divide, such that older and lower income patients, and older providers, engaged in less video care. |
| Ehmer et al. (2022) | Retrospective observational | Colorado. PROMISE Clinic, an integrated obstetric behavioral health program that serves pregnant women; HEART program, an integrated behavioral health program embedded in a primary care clinic for adolescent mothers and their babies | September 1, 2019 to February 29, 2020; April 1, 2020 to September 30, 2020 | Mental health | Young mothers <25 years and their children | Telehealth use | Service utilization measures; presenting problem, missed/canceled rates of appointments | Significantly more White and Hispanic perinatal women were seen during COVID and telehealth adaptations, while significantly fewer Black perinatal women were seen during this period. |
| Ferguson et al. (2023) | Quasi-experimental, retrospective cohort study | Nationwide, VHA | March 10, 2020 to February 28, 2021 | Mental health | Veterans active in VA health care | Video-based care utilization | Likelihood of having a video visit and rate of video visits among those who had any video visits by care type (primary care, mental health care, and specialty care) | Variation in video care utilization patterns by type of care identified Veteran populations that might require greater resources and support to initiate and sustain video care use. Our data support service specific outreach to homeless and American Indian/Alaska Native Veterans. |
| Frost et al. (2022) | Cross-sectional study | Nationwide, VHA | March 23, 2020, to March 22, 2021 | Opioid use disorder | Veterans receiving VHA care aged at least 18 years | Buprenorphine treatment retention via telephone or video-only or in-person visit | Treatment modality | Many patients accessed buprenorphine via telephone and some were less likely to have any video visits. These findings suggest that discontinuing or reducing telephone access may disrupt treatment for many patients, particularly groups with access disparities. |
| Guajardo et al. (2023) | Retrospective cohort study | Nationwide, VHA | October 2019 to September 2020 | Mental health services | Veterans | VVC | Mean percentage encounters through VVC; effect modification (time and ethnicity) | Hispanic veterans access VTH at higher rates than their non-Hispanic counterparts. |
| Haderlein et al. (2022) | Retrospective cohort study | VA PC-MHI clinics | March 1, 2018 to October 29, 2021 | Mental health | Veterans; PC-MHI patients | PC-MHI through telehealth | Telehealth use; Same day primary care access among new PC-MHI mental health patients | New PC-MHI patients who were seen via telehealth were less likely to receive same-day primary care access than patients seen in person. |
| Hogan et al. (2022) | Time series | Nationwide, VHA | July 2019 to October 2020 | Mental health | Veterans | VVC | Change in virtual mental telehealth encounters | Before COVID-19, rates of VVC use as percentages of all mental health care were higher among rural veterans. After implementation of pandemic restrictions, rural veteran VVC use continued to increase, but this increase was surpassed by that of urban veterans. |
| Hughes et al. (2021) | Retrospective cohort study | Singly family medicine clinic in a rural/micropolitan region in Appalachia | Pre-Covid: January 16, 2020 to March 15, 2020; Transition: to March 15, 2020 to April, 15 2020; COVID—April 15, 2020 to June 15, 2020 | Opioid use disorder | Patient visits from the electronic health records system at a single family medicine clinic with a high concentration of providers that offer OBOT services in a primarily rural and micropolitan region | Telehealth use during transition period and COVID period | The number and type of visits (in-person vs. telehealth) | Total MOUD visits increased during COVID while overall new patient visits remained constant. The clinic's overall catchment area increased in size, with new patients coming primarily from rural areas. |
| Leung et al. (2022) | Quasi-experimental study, pre-post | Nationwide, VHA | (March 16, 2019 to March 15, 2020) and 21 months after pandemic onset (March 16, 2020 to December 16, 2021) | Primary care with mental health | Veterans who received primary care and mental health integration services | Telehealth (and video) use for all primary care related encounters | Use of PC-MHI | Compared to urban VAs, telehealth expansion lagged for rural ones, especially in mental health integration services. |
| Lindsay et al. (2022) | Retrospective cohort study | Nationwide, VHA | October 2019 to September 2020 | Mental health | Veterans treated within the VHA | VVC encounter for mental health care | Use of telehealth for mental health services (at least one encounter) | VVC use for mental health care is greater in women veterans compared to male veterans and may reduce gender-specific access barriers. |
| McKee et al. (2022) | Quasi-experimental, pre-post | Nationwide, VHA | October 1, 2017 to March 10, 2020; March 11, 2020 to December 31, 2020 | Mental health | Veterans—couple or family therapy through the VA Health Care | VVC, The VA's telehealth platform | Telehealth Use | Pre-COVID predictors of TMH-V utilization were limited to obsessive-compulsive disorder diagnosis and history of psychiatric hospitalization, suggesting that TMH-V usage was largely related to clinical indications. In the COVID-19 era, older and rural veterans were less likely to attend appointments via TMH-V than younger and suburban/urban veterans, while Hispanic veterans were more likely to do so than non-Hispanic veterans. |
| Miu et al. (2021) | Retrospective cohort study | Outpatient psychiatry clinic, urban academic medical center, Texas | January 16, 2020 to April 30, 2020 | SMI including Schizophrenia spectrum Disorders, Bipolar Disorder (moderate or severe), Major Depressive Disorders (severe), Substance Use Disorders, BPD, suicidality, and PTSD | Patients with and without SMI | Teletherapy—Telehealth offered remotely by providers (who worked remotely) | Conversion to teletherapy, number of teletherapy sessions, new patients starting therapy via telehealth | SMI group had a significantly greater number of teletherapy visits compared to non-SMI patients, indicating that SMI group utilized teletherapy regularly after conversion to teletherapy. |
| Poulsen et al. (2023) | Cross-sectional survey | Five health systems with MOUD programs about tele-MOUD models | May 2021 | Opioid use disorder | Patients with OUD, referral varies | Telehealth delivery (tele-MOUD programs)—varied by site | (1) Compare tele-MOUD program models, (2) evaluate sociodemographic characteristics of patients who had utilized tele-MOUD versus those who had not, (3) describe trends in tele-MOUD use, and (4) explore the relationship between tele-MOUD use and patient engagement | Sociodemographic differences, with a greater proportion of female, White, and non-Hispanic patients using tele-MOUD. |
| Rosenthal et al. (2022) | Observational study | Rhode Island | May 2020 to October 2020 | Depressive disorder; anxiety disorder; Suicide ideation; harmful drinking | Young adults, 18–25 | Telehealth service (Phone or web-based) | Telehealth prevalence | Sexual and gender minorities and those with low social status were more likely to access telehealth services for mental health symptoms, highlighting its effectiveness at reaching disadvantaged young adults. |
| Ruprecht et al. (2021) | Cross-sectional survey | Chicago community | April 16, 2020 to May 15, 2020 | Mental health services | General population | None | Use of telehealth for mental health services | Black, Latinx, sexual minority, and gender minority group reported significantly lower levels of use of telehealth for mental health services. |
| Sizer et al. (2022) | Retrospective cohort study | Twelve mostly rural parishes in northeast Louisiana—outpatient behavioral health clinics served by NEDHSA | April 2020 to March 2021 | Mental health challenges, developmental disabilities, and addictive disorders; Anxiety, bipolar and depressive disorders, schizophrenia and psychotic disorders, trauma and stressor related disorders | Patients with telehealth visits who received psychiatric care in NEDHSA outpatient clinics | Telehealth service utilization | Intensity of treatment (total number of visits) | Being younger, female, and more educated were associated with a higher number of telehealth visits. The prevalence of other chronic conditions increased telehealth visits by 10%. Patients diagnosed with the schizophrenia spectrum and other psychotic disorders utilizing 15% fewer telehealth visits than patients diagnosed with depressive disorders. |
| Tobin et al. (2023) | Quasi-experimental study | General Internal Medicine Clinic within a large urban health system | January to December 2020 | Integrated psychology team (i.e., senior staff psychologist and/or psychology doctoral student | Patients seeking care in clinic | Telehealth services (audio and video) | Use of video versus (audio, in-person visits) | Older patients, Black patients, and those with Medicare and Medicaid were more likely to complete audio only telehealth visits versus video visits. |
| Ward et al. (2022) | Cross-sectional study | Schools in Alabama, Arkansas, Connecticut, Indiana, Kansas, Kentucky, Minnesota, New Mexico, New York, North Carolina, South Dakota, Tennessee, Virginia, West Virginia | Fall 2019 to Spring 2020 | Adjustment disorders, anxiety, OCD, bipolar, depression, ADD, ADHD, impluse control disorders, trauma and stress disorders | <18 Years | Tele-behavioral health services in schools | Change in total students participating | Tele-behavioral health programs significantly increased both behavioral services provided to their ongoing schools and increased the number of schools served. |
| Weintraub et al. (2021) | Interventional study | Maryland communities | June to October 2020 | Opioid use disorder | Patients who entered treatment with TM-based medications for OUD | TM-MTU | 0, 30, 60, and 90 Days after treatment intake | Data demonstrate the feasibility of combining TM with mobile treatment, with outcomes (retention and opioid use) similar to those obtained from office-based TM MOUD programs. |
| Williams et al. (2023) | Quasi-experimental, pre-post | Outpatients Divisions, Childrens' Hospitals (Philadelphia, Boston) | March 15, 2019 to April 15, 2019; March 15, 2020 to April 15, 2020 | Pediatric mental health and psychiatry | All patients at the childrens' hospitals | Transition to telemedicine services | Changes in the characteristics of patients | Racially minoritized patients receiving services in urban areas may be particularly at risk for losing access when telemedicine is implemented. |
| Xue et al. (2022) | Quasi-experimental design, pre-post | Twenty-seven hospitals in 24 counties in North Carolina (NC Statewide Telepsychiatry Program) | January 2019 to March 2021 | Mental health care | Patients ED visits with mental health complaints and patients receiving telepsychiatry consultations | ED Telepsychiatry consultation counts | ED telepsychiatry counts | COVID-19 crisis has led to a heightening demand for telepsychiatry consultations in NC, and there is a possible race disparity in these demands between black and white mental health patients. |
ADD, attention deficit disorder; ADHD, attention-deficit/hyperactivity disorder; BPD, Borderline Personality Disorder; CMHC, community mental health center; IPC, integrated primary care; MH, mental health; MOUD, medications for opioid use disorder; MTU, mobile treatment unit; NEDHSA, Northeast Delta Human Services Authority; OBOT, office-based opioid treatment; OCD, obessive compulsive disorder; OUD, opioid use disorder/dependence; PC-MHI, Primary Care-Mental Health Integration; PTSD, posttraumatic stress disorder; REACH, Respectful Equitable Access to Compassionate Healthcare; SMI, Serious Mental Illness; TM, Telemedicine; TMH-V, Telemental Health-Videoconferencing; VHA, Veterans Health Administration; VTH, Video-to-home Telehealth; VVC, VA Video Connect.
STUDY CONDITIONS, OUTCOMES, AND SUBGROUPS
Most studies (n = 20) evaluated telehealth for mental health conditions,39–57 while the remaining (n = 5) studies evaluated telehealth for opioid use disorder/dependence (OUD).58–62 Conditions within mental health included general mental health (e.g., anxiety), serious mental health (e.g., schizophrenia, major depressive disorder), and primary care/mental health integration (e.g., psychiatry and family medicine linking services).
The most common study outcomes focused on utilization (n = 19), but other outcomes included visit attendance (n = 1), primary care visit attendance (n = 1), treatment retention (n = 1), and demographic characteristics of telehealth utilization (n = 3). Apart from treatment retention in one study, no other studies evaluated the relationship between telehealth and clinical effectiveness outcomes among the included studies incorporating a health equity component.
Table 2 captures key findings by each subgroup/underrepresented group and condition assessed. Overall, the most common dimensions captured included race/ethnicity (n = 21) or gender (n = 20). The least common dimensions of diversity included sexual orientation/identity (n = 2) or social vulnerability (n = 4).
Table 2.
Summary of Key Study Findings by Underrepresented Group and Condition
|
RURALITY
There were 10 studies with a geographical component included, of which 7 reported that rural residents had less telehealth use than urban residents42,44,45,47–50 and 3 indicated no differences in utilization outcomes by urban-rural designation.39,58,59 All studies that observed rural/urban disparities during COVID-19 were reported from the Veterans Health Administration (VHA).42,44,45,47–50
AGE
Sixteen studies included data on age, of which 10 demonstrated that older patients were less likely to use telehealth than younger patients39,42,44,45,49,50,53,58,59,63 and none showed increased use in older patients.
GENDER
Of the 20 studies that examined telehealth outcomes by gender, we observed 10 studies that demonstrated differences in telehealth utilization,39,42,44,45,49,51,53,57,59,61 and of these, nine reported increased utilization among women. Five of these studies were conducted within the VHA, but there were no other discernable similarities in these studies.
RACE/ETHNICITY
There were 21 studies that included at least 1 stratified analysis of race and/or ethnicity indicators, and the plurality of studies (n = 7) reported lower telehealth utilization among Black/African American participants when compared to white participants.41,42,49,52,53,57,59 We did not detect any discernable patterns in studies that reported differences in telehealth utilization by other racial/ethnic group comparisons.
SEXUAL ORIENTATION
Two studies included telehealth utilization conducted in the COVID-19 era that were stratified by sexual orientation and/or gender identity. Both studies reported increased telehealth use among gender minorities (e.g., gay, lesbian) or sexual identity minorities (e.g., nonbinary, transgender).51,52
INSURANCE
Seven studies reported telehealth-related outcomes by insurance categories, of which most studies (n = 5) reported no difference in telehealth utilization by indicators of public insurance.41,43,58,60,61 Among the two studies that showed mixed results,54,56 there was no identifiable pattern in observed findings due to variability in condition (e.g., mental health vs. SUD), utilization measures (e.g., utilization, predictors of telehealth users, attendance), or study population (e.g., children, Veterans, women, community-based).
SOCIAL VULNERABILITY
Four studies examined social vulnerability.42,44,58,59 Results from these studies were mixed and identified decreased utilization among those in areas with a higher area deprivation index,42 mixed results due to multiple utilization markers assessed,44,59 or no difference by markers of social vulnerability.58
RISK-OF-BIAS ASSESSMENT
Findings by each component of the Downs and Black Risk-of-Bias Assessment are presented in Figure 2. Total scores ranged between 8 and 20, with four studies with a risk of bias score of 18 or greater. Scores averaged 14.2 points (out of 28), while other subdomain averages included reporting bias (7.5/11), external validity (1.1/3), internal validity—bias (3.9/7), internal validity—confounding (1.6/6), and power (0/6).
Fig. 2.
Risk-of-bias assessment.
Discussion
The COVID-19 pandemic precipitated stress on health care systems and led to disruptions in clinical care.1–3 Due to these circumstances, vulnerable populations (e.g., women, racial/ethnic minorities, sexual minorities, Medicaid beneficiaries, older patients, rural patients) who were already at greater risk for worse health outcomes now faced additional burdens in addressing physical and mental health needs. In this systematic review, we examined health equity in telehealth studies that emerged from the COVID-19 pandemic among vulnerable populations, and we observed overall that vulnerable patient groups utilized telehealth less for mental health and SUD when compared to those with less vulnerability.
A priori, we aimed to include studies with multiple clinical and operational outcomes; however, 76% (n = 19) of identified studies focused on telehealth utilization as an outcome (with the remaining assessing attendance, retention, or within-group characteristics of telehealth users with no comparator). This finding was both surprising and important, as numerous pre-COVID-19 studies examined clinical effectiveness between telehealth and in-person care. For both mental health and SUD in the pre-COVID-19 era, prior work examined key outcomes of clinical effectiveness (with or without a health equity component), including medication adherence,64,65 treatment retention,66–68 and health care utilization69–76 such as hospitalizations, emergency department visits, and operational metrics (e.g., length of stay).
Telehealth utilization as an outcome is critical, as limited adoption of technological advancements is an indicator of disparities; yet, this outcome alone may mask other disparities in health outcomes between clinical services (e.g., at the innovation and health care system levels) and the vulnerable populations they serve.
Second, many COVID-19 era studies reported the experience with interventions that were designed and implemented very quickly, which reflected the reality of the public health emergency (PHE), but leaves findings open to risk of bias as we found in our synthesis of the quality of included studies. Without accounting for selection or unmeasured confounding bias thoroughly, we are limited in our understanding of what factors are truly contributing to telehealth-based inequities. Nonetheless, capitalizing on the natural experiment of the COVID-19 experience allows us to infer how increased telehealth use might function at scale—even outside the context of the pandemic. While we are approaching a postpandemic stage, it is purported that psychological morbidity will peak later and last longer than physical health consequences of the COVID-19 pandemic.8,77 Therefore, there still exists an unmet need to examine clinical effectiveness outcomes comprehensively through rigorous design and analytical approaches moving forward.
The COVID-19 PHE shifted our traditional models in health care delivery by rapidly expanding use of telehealth—partially through relaxing policies surrounding mental health and SUD care.29,78–80 For example, significant changes to Medicare emerged as a result of the PHE, including, but not limited to, enabling Federally Qualified Health Centers and Rural Health Clinics to serve as distant site providers for behavioral and mental health services, enabling Medicare patients to receive telehealth services from their homes, use of audio-only communication platforms for behavioral/mental health services, and provision of telehealth services by all eligible Medicare providers.81 Before the PHE, telehealth services were limited to rural beneficiaries (including where these services could be rendered), thus excluding urban residents from these benefits.82
These policies may have partially contributed to increased utilization in some groups (e.g., urban populations, since rural populations were able to use telehealth before the PHE), but it did not translate to reduced disparities among underserved groups. Rather, they could have exacerbated existing disparities particularly for rural residents, older populations, and Black/African American minorities.
Understanding barriers that contributed to these widening disparities may also be better understood through future work focused on (1) underlying systems of care, (2) mental health and SUD uniquely as a subset of health conditions, and (3) the patient populations telehealth successfully or unsuccessfully reaches.83 When examining the underlying systems of care, one key finding in our review was from the subset of VHA-based studies that accounted for over one-third of included studies. This is not surprising or unique as the VHA was an early adopter of telehealth services, serves as a mature telehealth network, and is committed to telehealth expansion to serve the needs of rural Veterans.
However, a distinct finding among several VHA studies was that telehealth utilization was similar between urban and rural Veterans before the COVID-19 pandemic, but the growth in telehealth use among rural Veterans after the onset of the pandemic lagged behind their urban counterparts—despite a well-organized and implemented mature telehealth program.47–50 Commonly cited barriers included structural aspects of using telehealth such as internet bandwidth, lack of smart phones/tablets in rural areas, and longer time frames associated with requesting, receiving, and learning these services.49,84,85 The VHA had been working toward developing the telehealth infrastructure before the COVID-19 pandemic by distributing tablets and creating regional hubs to provide telehealth visits for understaffed clinics,86,87 experiences which may not be reflected outside of the VHA system. However, the pandemic onset still prompted logistical constraints, which could have led to lags in adopting changes rapidly in health system practices particularly among more vulnerable groups.
A particularly important facet to this work is that we focused on mental health and/or SUD, which often reflects a uniquely underserved and marginalized population. To understand potential barriers within mental health and/or SUD, the condition itself may compound individual-level characteristics (e.g., older, rural, socially disadvantaged patients) that represent a different user group than those who quickly adopted telehealth for other conditions. Personal attributes such as motivation toward change in practices, knowledge/awareness about telehealth, optimism, and beliefs in self-efficacy (i.e., confidence in being able to successfully use telehealth) may be different among vulnerable subgroups, which may drive ongoing disparities in observed utilization metrics.23,54,84,85,88,89
Our study has several limitations. First, the underlying studies exhibited substantial heterogeneity in outcomes and inclusion criteria. Much of this variation was related to the setting in which health system innovation occurred, but it makes pooling of findings challenging. Second, much of the published research reported results from very early in the COVID-19 experience. As health systems and programs matured, standard procedures may have changed to respond to widening observed disparities. Finally, variability in programs under study that were published might have changed over time, such that a program's function evolved, or that comfort with telehealth and technology may have changed patients' willingness to participate in telehealth-enabled care.
Conclusions
In this systematic review of studies evaluating health equity in telehealth use for mental health and SUD care after the onset of the COVID-19 PHE, we found that most studies identified that telehealth implementation during this phase suffered from significant and widening disparities for disadvantaged populations, including rural populations, older patients, and racial/ethnic minorities. If the technological vehicle used to address inequities further propagates a digital divide, policymakers should examine individual-, innovation-, and system-level implementation processes and policies that promote or hinder equity in adoption, utilization, and clinical effectiveness. Future efforts should focus on measuring the contribution of utilization disparities on outcomes and strategies to mitigate disparities in implementation.
Disclosure Statement
No competing financial interests exist.
Funding Information
This study was supported by funding from the Office for the Advancement of Telehealth, Health Resources and Services Administration (U3G RH40003).
Supplementary Material
REFERENCES
- 1. Buikema AR, Buzinec P, Paudel ML, et al. Racial and ethnic disparity in clinical outcomes among patients with confirmed COVID-19 infection in a large US electronic health record database. EClinicalMedicine 2021;39:101075. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2. Magesh S, John D, Li WT, et al. Disparities in COVID-19 outcomes by race, ethnicity, and socioeconomic status: A systematic-review and meta-analysis. JAMA Netw Open 2021;4(11):e2134147. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3. Robinson-Lane SG, Sutton NR, Chubb H, et al. Race, ethnicity, and 60-day outcomes after hospitalization with COVID-19. J Am Med Dir Assoc 2021;22(11):2245–2250. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4. Wang SM, Dalal K. Safe communities in China as a strategy for injury prevention and safety promotion programmes in the era of rapid economic growth. J Community Health 2013;38(1):205–214. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5. Singh S, Bedi D. Financial disruption and psychological underpinning during COVID-19: A review and research agenda. Front Psychol 2022;13:878706. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6. Khan KS, Mamun MA, Griffiths MD, et al. The mental health impact of the COVID-19 pandemic across different cohorts. Int J Ment Health Addict 2022;20(1):380–386. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7. Cullen W, Gulati G, Kelly BD. Mental health in the COVID-19 pandemic. QJM 2020;113(5):311–312. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8. Gavin B, Lyne J, McNicholas F. Mental health and the COVID-19 pandemic. Ir J Psychol Med 2020;37:156–158. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9. Kumar A, Nayar KR. COVID 19 and its mental health consequences. J Ment Health 2021;30(1):1–2. [DOI] [PubMed] [Google Scholar]
- 10. Cheng Z, Mendolia S, Paloyo AR, et al. Working parents, financial insecurity, and childcare: Mental health in the time of COVID-19 in the UK. Rev Econ Househ 2021;19(1):123–144. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11. Panchal N, Saunders H, Rudowitz R, et al. The Implications of COVID-19 for Mental Health and Substance Use. KFF; 2023. Available from: https://www.kff.org/coronavirus-covid-19/issue-brief/the-implications-of-covid-19-for-mental-health-and-substance-use [Last accessed: November 16, 2023].
- 12. Purtle J. COVID-19 and mental health equity in the United States. Soc Psychiatry Psychiatr Epidemiol 2020;55:969–971. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13. Swaziek Z, Wozniak A. Disparities old and new in US mental health during the COVID-19 pandemic. Fisc Stud 2020;41(3):709–732. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14. The Academy of Medical Sciences, MQ: Transforming Mental Health. Survey Results: Understanding People's Concerns about the Mental Health Impacts of the COVID-19 Pandemic. The Academy of Medical Sciences; 2020. Available from: https://acmedsci.ac.uk/file-download/99436893 [Last accessed: January 8, 2024].
- 15. Czeisler M, Board A, Thierry JM, et al. Mental health and substance use among adults with disabilities during the COVID-19 pandemic—United States, February–March 2021. MMWR Morb Mortal Wkly Rep 2021;70(34):1142–1149. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 16. National Institute on Drug Abuse. COVID-19 & Substance Use. NIH; 2023. Available from: https://nida.nih.gov/research-topics/comorbidity/covid-19-substance-use. [Last accessed: November 16, 2023].
- 17. McKnight-Eily LR, Okoro CA, Strine TW, et al. Racial and ethnic disparities in the prevalence of stress and worry, mental health conditions, and increased substance use among adults during the COVID-19 pandemic—United States, April and May 2020. MMWR Morb Mortal Wkly Rep 2021;70(5):162–166. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 18. Cacari Stone L, Roary MC, Diana A, et al. State health disparities research in Rural America: Gaps and future directions in an era of COVID-19. J Rural Health 2021;37(3):460–466. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 19. Mueller JT, McConnell K, Burow PB, et al. Impacts of the COVID-19 pandemic on rural America. Proc Natl Acad Sci U S A 2021;118(1):2019378118. doi: 10.1073/pnas.2019378118. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 20. Mueller JT, Merdjanoff AA. Mental Health in Rural America During COVID-19 and Beyond. In: COVID-19, Frontline Responders and Mental Health: A Playbook for Delivering Resilient Public Health Systems Post-Pandemic (Horney JA. ed.) Emerald Publishing Limited: Leeds, England; 2023; pp. 119–133. [Google Scholar]
- 21. Chacon NC, Walia N, Allen A, et al. Substance use during COVID-19 pandemic: Impact on the underserved communities. Discoveries (Craiova) 2021;9(4):e141. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 22. Hollander JE, Carr BG. Virtually perfect? Telemedicine for Covid-19. N Engl J Med 2020;382(18):1679–1681. [DOI] [PubMed] [Google Scholar]
- 23. Luo J, Tong L, Crotty BH, et al. Telemedicine adoption during the COVID-19 pandemic: Gaps and inequalities. Appl Clin Inform 2021;12(4):836–844. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 24. Ohannessian R, Duong TA, Odone A. Global Telemedicine implementation and integration within health systems to fight the COVID-19 pandemic: A call to action. JMIR Public Health Surveill 2020;6(2):e18810. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 25. Portnoy J, Waller M, Elliott T. Telemedicine in the era of COVID-19. J Allergy Clin Immunol Pract 2020;8(5):1489–1491. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 26. Office for the Advancement of Telehealth, Health Resources and Services Administration. What is Telehealth? HRSA; 2022. Available from: https://www.hrsa.gov/telehealth/what-is-telehealth [Last accessed: November 16, 2023].
- 27. Friedman AB, Gervasi S, Song H, et al. Telemedicine catches on: Changes in the utilization of telemedicine services during the COVID-19 pandemic. Am J Manag Care 2022;28(1):e1–e6. [DOI] [PubMed] [Google Scholar]
- 28. Cantor J, McBain RK, Kofner A, et al. Telehealth adoption by mental health and substance use disorder treatment facilities in the COVID-19 pandemic. Psychiatr Serv 2022;73(4):411–417. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 29. Mulvaney-Day N, Dean D Jr., Miller K, et al. Trends in use of telehealth for behavioral health care during the COVID-19 pandemic: Considerations for payers and employers. Am J Health Promot 2022;36:1237–1241. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 30. Clare CA. Telehealth and the digital divide as a social determinant of health during the COVID-19 pandemic. Netw Model Anal Health Inform Bioinform 2021;10(1):26. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 31. Ramsetty A, Adams C. Impact of the digital divide in the age of COVID-19. J Am Med Inform Assoc 2020;27(7):1147–1148. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 32. Moher D, Liberati A, Tetzlaff J, et al. Preferred reporting items for systematic reviews and meta-analyses: The PRISMA statement. PLoS Med 2009;6(7):e1000097. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 33. Telehealth.HHS.gov. Research Articles and Papers. Telehealth.HHS.gov; 2023. Available from: https://telehealth.hhs.gov/research-articles [Last accessed: November 16, 2023].
- 34. Rural Health Research Gateway. Rural Health Research Gateway. 2023. Available from: https://www.ruralhealthresearch.org [Last accessed: November 16, 2023].
- 35. Downs SH, Black N. The feasibility of creating a checklist for the assessment of the methodological quality both of randomised and non-randomised studies of health care interventions. J Epidemiol Community Health 1998;52(6):377–384. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 36. Szmulewicz AG, Benson NM, Hsu J, et al. Effects of COVID-19 pandemic on mental health outcomes in a cohort of early psychosis patients. Early Interv Psychiatry 2021;15(6):1799–1802. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 37. Busch AB, Mehrotra A, Greenfield SF, et al. A cohort study examining changes in treatment patterns for alcohol use disorder among commercially insured adults in the United States during the COVID-19 pandemic. J Subst Abuse Treat 2023;144:108920. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 38. Deng H, Raheemullah A, Fenno LE, et al. A telehealth inpatient addiction consult service is both feasible and effective in reducing readmission rates. J Addict Dis 2023;41(3):225–232. [DOI] [PubMed] [Google Scholar]
- 39. Ainslie M, Brunette MF, Capozzoli M. Treatment interruptions and telemedicine utilization in serious mental illness: Retrospective longitudinal claims analysis. JMIR Ment Health 2022;9(3):e33092. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 40. Campos-Castillo C, Laestadius LI. Mental healthcare utilization, modalities, and disruptions during Spring 2021 of the COVID-19 pandemic among U.S. adolescents. J Adolesc Health 2022;71(4):512–515. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 41. Chakawa A, Belzer LT, Perez-Crawford T, et al. COVID-19, telehealth, and pediatric integrated primary care: disparities in service use. J Pediatr Psychol 2021;46(9):1063–1075. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 42. Connolly SL, Stolzmann KL, Heyworth L, et al. Patient and provider predictors of telemental health use prior to and during the COVID-19 pandemic within the Department of Veterans Affairs. Am Psychol 2022;77(2):249–261. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 43. Ehmer AC, Scott SM, Smith H, et al. Connecting during COVID: The application of teleservices in two integrated perinatal settings. Infant Ment Health J 2022;43(1):127–139. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 44. Ferguson JM, Wray CM, Jacobs J, et al. Variation in initial and continued use of primary, mental health, and specialty video care among Veterans. Health Serv Res 2023;58(2):402–414. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 45. Guajardo E, Amspoker AB, Stanley MA, et al. Patterns of telehealth use for mental health treatment among Hispanic Veterans. Telemed J E Health 2023;29(5):788–792. [DOI] [PubMed] [Google Scholar]
- 46. Haderlein TP, Dobalian A, Raja PV, et al. Association between virtual care use and same-day primary care access in VA primary care-mental health integration. J Prim Care Community Health 2022;13:21501319221091430. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 47. Hogan J, Amspoker AB, Walder A, et al. Differential impact of COVID-19 on the use of tele-mental health among Veterans living in urban or rural areas. Psychiatr Serv 2022;73(12):1393–1396. [DOI] [PubMed] [Google Scholar]
- 48. Leung LB, Yoo C, Chu K. Widening rural-urban divide in telehealth use during the COVID-19 pandemic: A national observation study of primary care delivery in the Veterans Health Administration. J Gen Intern Med 2022;37:S346–S347. [Google Scholar]
- 49. Lindsay JA, Caloudas A, Hogan J, et al. Getting connected: A retrospective cohort investigation of video-to-home telehealth for mental health care utilization among women Veterans. J Gen Intern Med 2022;37(Suppl 3):778–785. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 50. McKee GB, Knopp K, Morland LA, et al. Use of telemental health for VA family services before and during the COVID-19 pandemic. Psychol Serv 2023;20(Suppl 2):20–32; doi: 10.1037/ser0000704. [DOI] [PubMed] [Google Scholar]
- 51. Rosenthal SR, Sonido PL, Tobin AP, et al. Breaking down barriers: Young adult interest and use of telehealth for behavioral health services. R I Med J (2013) 2022;105(1):26–31. [PubMed] [Google Scholar]
- 52. Ruprecht MM, Wang X, Johnson AK, et al. Evidence of social and structural COVID-19 disparities by sexual orientation, gender identity, and race/ethnicity in an urban environment. J Urban Health 2021;98(1):27–40. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 53. Sizer MA, Bhatta D, Acharya B, et al. Determinants of telehealth service use among mental health patients: A case of rural Louisiana. Int J Environ Res Public Health 2022;19(11):6930. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 54. Tobin ET, Hadwiger A, DiChiara A, et al. Demographic predictors of telehealth use for integrated psychological services in primary care during the COVID-19 pandemic. J Racial Ethn Health Disparities 2023;10(3):1492–1498. [DOI] [PubMed] [Google Scholar]
- 55. Ward MM, Ullrich F, Merchant KAS, et al. Describing changes in telebehavioral health utilization and services delivery in rural school settings in pre- and early stages of the COVID-19 public health emergency. J Sch Health 2022;92(5):452–460. [DOI] [PubMed] [Google Scholar]
- 56. Williams JC, Ball M, Roscoe N, et al. Widening racial disparities during COVID-19 telemedicine transition: A study of child mental health services at two large children's hospitals. J Am Acad Child Adolesc Psychiatry 2023;62(4):447–456. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 57. Xue Y, Saeed SA, Liang H, et al. Investigating the impact of Covid-19 on telepsychiatry use across sex and race: A study of North Carolina emergency departments. Telemed J E Health 2022;28(10):1431–1439. [DOI] [PubMed] [Google Scholar]
- 58. Chang JE, Lindenfeld Z, Thomas T, et al. Patient characteristics associated with phone versus video telemedicine visits for substance use treatment during COVID-19. J Addict Med 2022;16(6):659–665. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 59. Frost MC, Zhang L, Kim HM, et al. Use of and retention on video, telephone, and in-person buprenorphine treatment for opioid use disorder during the COVID-19 pandemic. JAMA Netw Open 2022;5(10):e2236298. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 60. Hughes PM, Verrastro G, Fusco CW, et al. An examination of telehealth policy impacts on initial rural opioid use disorder treatment patterns during the COVID-19 pandemic. J Rural Health 2021;37(3):467–472. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 61. Poulsen MN, Santoro W, Scotti R, et al. Implementation of telemedicine delivery of medications for opioid use disorder in Pennsylvania treatment programs during COVID-19. J Addict Med 2023;17(2):e110–e118. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 62. Weintraub E, Seneviratne C, Anane J, et al. Mobile telemedicine for buprenorphine treatment in rural populations with opioid use disorder. JAMA Netw Open 2021;4(8):e2118487. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 63. Miu AS, Vo HT, Palka JM, et al. Teletherapy with serious mental illness populations during COVID-19: Telehealth conversion and engagement. Counsell Psychol Q 2021;34(3–4):704–721. [Google Scholar]
- 64. Basit SA, Mathews N, Kunik ME. Telemedicine interventions for medication adherence in mental illness: A systematic review. Gen Hosp Psychiatry 2020;62:28–36. [DOI] [PubMed] [Google Scholar]
- 65. Guille C, Simpson AN, Douglas E, et al. Treatment of opioid use disorder in pregnant women via telemedicine: A nonrandomized controlled trial. JAMA Netw Open 2020;3(1):e1920177. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 66. Lin YH, Chen SY, Lin PH, et al. Assessing user retention of a mobile app: Survival analysis. JMIR Mhealth Uhealth 2020;8(11):e16309. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 67. Spruill TM, Friedman D, Diaz L, et al. Telephone-based depression self-management in Hispanic adults with epilepsy: A pilot randomized controlled trial. Transl Behav Med 2021;11(7):1451–1460. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 68. Vakkalanka JP, Lund BC, Ward MM, et al. Telehealth utilization is associated with lower risk of discontinuation of buprenorphine: A retrospective cohort study of US Veterans. J Gen Intern Med 2022;37(7):1610–1618. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 69. Kothadia RJ, Jones K, Saeed SA, et al. The impact of the North Carolina Statewide Telepsychiatry Program (NC-STeP) on patients' dispositions from emergency departments. Psychiatr Serv 2020;71(12):1239–1244. [DOI] [PubMed] [Google Scholar]
- 70. Mohr NM, Wu C, Ward MJ, et al. Potentially avoidable inter-facility transfer from Veterans Health Administration emergency departments: A cohort study. BMC Health Serv Res 2020;20(1):110. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 71. Roberge J, McWilliams A, Zhao J, et al. Effect of a virtual patient navigation program on behavioral health admissions in the emergency department: A randomized clinical trial. JAMA Netw Open 2020;3(1):e1919954. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 72. Natafgi N, Mohr NM, Wittrock A, et al. The association between telemedicine and emergency department (ED) disposition: A stepped wedge design of an ED-based telemedicine program in critical access hospitals. J Rural Health 2020;36(3):360–370. [DOI] [PubMed] [Google Scholar]
- 73. Fairchild RM, Ferng-Kuo SF, Laws S, et al. Telehealth decreases rural emergency department wait times for behavioral health patients in a group of critical access hospitals. Telemed J E Health 2019;25(12):1154–1164. [DOI] [PubMed] [Google Scholar]
- 74. Vakkalanka JP, Harland KK, Wittrock A, et al. Telemedicine is associated with rapid transfer and fewer involuntary holds among patients presenting with suicidal ideation in rural hospitals: A propensity matched cohort study. J Epidemiol Community Health 2019;73(11):1033–1039. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 75. Vakkalanka JP, Nataliansyah MM, Merchant KAS, et al. Telepsychiatry services across an emergency department network: A mixed methods study of the implementation process. Am J Emerg Med 2022;59:79–84. [DOI] [PubMed] [Google Scholar]
- 76. Vakkalanka JP, Nataliansyah MM, Merchant KAS, et al. Evaluation of telepsychiatry services implementation in medical and psychiatric inpatient settings: A mixed-methods study. Telemed J E Health 2023;29(8):1224–1232. [DOI] [PubMed] [Google Scholar]
- 77. Maunder RG, Leszcz M, Savage D, et al. Applying the lessons of SARS to pandemic influenza: An evidence-based approach to mitigating the stress experienced by healthcare workers. Can J Public Health 2008;99(6):486–488. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 78. Whaibeh E, Mahmoud H, Naal H. Telemental health in the context of a pandemic: The COVID-19 experience. Curr Treat Options Psychiatry 2020;7(2):198–202. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 79. Heath S. DEA Okays Telehealth to Prescribe Opioids Amid COVID-19 Emergency. TechTarget, Inc., Xtelligent Healthcare Media; 2020. Available from: https://mhealthintelligence.com/news/dea-okays-telehealth-to-prescribe-opioids-amid-covid-19-emergency [Last accessed: November 16, 2023].
- 80. Verma S. Early Impact of CMS Expansion of Medicare Telehealth During COVID-19. Health Affairs; 2020. Available from: https://www.healthaffairs.org/content/forefront/early-impact-cms-expansion-medicare-telehealth-during-covid-19 [Last accessed: November 16, 2023].
- 81. Health Resources & Services Administration. Telehealth Policy Changes after the COVID-19 Public Health Emergency. HRSA; 2023. Available from: https://telehealth.hhs.gov/providers/telehealth-policy/policy-changes-after-the-covid-19-public-health-emergency [Last accessed: November 16, 2023].
- 82. Koma W, Cubanski J, Neuman T. Medicare and Telehealth: Coverage and Use During the COVID-19 Pandemic and Options for the Future. KFF; 2021. Available from: https://www.kff.org/medicare/issue-brief/medicare-and-telehealth-coverage-and-use-during-the-covid-19-pandemic-and-options-for-the-future [Last accessed: November 16, 2023].
- 83. CFIR. Consolidated Framework for Implementation Research. Available from: https://cfirguide.org [Last accessed: January 8, 2024].
- 84. Morales DA, Barksdale CL, Beckel-Mitchener AC. A call to action to address rural mental health disparities. J Clin Transl Sci 2020;4(5):463–467. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 85. Myers CR. Using telehealth to remediate rural mental health and healthcare disparities. Issues Ment Health Nurs 2019;40(3):233–239. [DOI] [PubMed] [Google Scholar]
- 86. Cannedy S, Bergman A, Medich M, et al. Health system resiliency and the COVID-19 pandemic: A case study of a new nationwide contingency staffing program. Healthcare (Basel) 2022;10(2):244. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 87. Zulman DM, Wong EP, Slightam C, et al. Making connections: Nationwide implementation of video telehealth tablets to address access barriers in veterans. JAMIA Open 2019;2(3):323–329. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 88. Andrews JA, Brown LJ, Hawley MS, et al. Older adults' perspectives on using digital technology to maintain good mental health: Interactive group study. J Med Internet Res 2019;21(2):e11694. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 89. Wuthrich VM, Frei J. Barriers to treatment for older adults seeking psychological therapy. Int Psychogeriatr 2015;27(7):1227–1236. [DOI] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.


