Abstract
Objective:
To conduct a mixed-methods evaluation of an emergency telehealth intervention in unscheduled settings (emergency department [ED] and urgent care clinic [UCC]) within the Veterans Health Administration (VHA).
Materials and Methods:
We used the Reach, Effectiveness, Adoption, Implementation, Maintenance (RE-AIM) framework to conduct a mixed-methods evaluation of a novel telehealth program implemented in the VHA (Hospital System) in March 2020. We compared the 3 months preimplementation (December 1, 2019 through February 29, 2020) with the 3 months postimplementation (April 1, 2020 through June 30, 2020), then followed sustainability through January 31, 2021. Qualitative data were obtained from surveys and semistructured interviews of staff and providers and analyzed with thematic analysis.
Results:
Patient demographics and dispositions were similar pre- and postimplementation. The telemental health intervention was used in 319 (83%) unscheduled mental health consultations in the postimplementation phase. After implementation, we did not detect adverse trends in length of stay, 7-day revisits, or 30-day mortality. Use remained high with 82% (n = 1,010) of all unscheduled mental health consultations performed by telemental health in the sustainability phase. Staff and clinician interviews identified the following themes in the use of telemental health: (1) enhanced efficiency without compromising quality and safety, (2) initial apprehension, (3) the COVID-19 pandemic, and (4) sustainability after resolution of the COVID-19 pandemic.
Conclusions:
This mixed-methods evaluation of unscheduled telemental health implementation found that its use was feasible, did not impact the safety and efficacy of mental health consultations, and was highly acceptable and sustainable in unscheduled settings.
Keywords: emergency care, telehealth, mental health, implementation
Introduction
Mental health conditions are the most common cause of disability worldwide and account for 1-in-8 emergency department (ED) visits in the United States.1,2 Mental health-related conditions are the fastest growing reasons for ED visits, with a 41% increase in adult psychiatric ED visits between 2009 and 2015.2 Within the Veterans Health Administration (VHA), mental health ED visits are the sixth most common reason for ED visits and the most common reason for interfacility transfers, comprising 41% of all VHA ED transfers. More than 11% of VHA ED transfers for mental health conditions are considered potentially avoidable3 and represent a preventable source of cost and psychological burden for patients and their families.4,5
Use of telehealth, or telemental health, may improve access and quality of care across large distances by providing a real-time, high quality video connection between a mental health professional and patient. Telemental health is noninferior to in-person care for post-traumatic stress disorder and major depression and is cost-efficient.4 Within the VHA, access to telehealth expanded rapidly in the first months of the COVID-19 pandemic for outpatients6 and scheduled mental health care.7 As nearly one-third of all Veterans are rural and the accompanying shortage of rural mental health providers,8 telehealth represents an opportunity to deliver mental health care to rural Veterans.
The COVID-19 pandemic underscored the need for alternative access for unscheduled mental health care. Many patients delayed9 or were restricted in seeking in-person care due to the pandemic. The Centers for Disease Control and Prevention (CDC) recommended telehealth as a strategy to enhance patient access during the pandemic,9 but key knowledge gaps for broader dissemination remain. These include the feasibility and acceptability of telemental health in unscheduled settings (i.e., ED and Urgent Care Clinics [UCCs]). Furthermore, determining how the implementation of such programs impacted care and potential sustainability are important to understand for successful broader dissemination. The VA Office of Rural Health funded the development and implementation of a provider-to-provider telemental health intervention in unscheduled settings within the VA (Healthcare System). We sought to describe and evaluate this intervention based on the Standards for Reporting Implementation Studies Statement.10
Materials and Methods
SETTING
The VA (Healthcare System) has two 24/7 unscheduled care locations, one in (city) and the other in (city 2). The (city) campus has 26,000 annual ED patient visits and accepts ambulances, whereas (city 2) has 15,000 annual UCC visits and does not accept ambulances. Both facilities are staffed with emergency and mental health providers (advanced practice providers and board-certified psychiatrists) with expertise in emergency psychiatry. Before the intervention, mental health providers were available for in-person consultations during operating hours.
INTERVENTION
Telemental health carts consisted of an Apple (Cupertino, CA) iPad and stand and used FaceTime for videoconferencing. Preparation for intervention implementation began before the COVID-19 pandemic in January 2019 with the development of standard operating procedures and workflow and stakeholder engagement. In April 2019, simulated scenarios were used to refine the workflow, and in May 2019, volunteer providers and low-acuity patients were using telemental health during weekday daytime hours (8 AM–4.30 PM). Use was then expanded to all mental health providers, and in March 2020 telemental health became the default means for conducting mental health evaluations in unscheduled settings.
All patients presenting to the VA (Hospital System) with a mental health consultation were eligible for telemental health. The finalized workflow process was similar to in-person care: once a treating provider determined that a mental health consultation was necessary, a phone call was placed to the mental health provider, and a consultation was ordered. Behind the electronic security firewall, the mental health provider then used their iPad to call the ED/UCC's unique iPad, which was brought into the patient's room. An ED/UCC nurse was assigned to each case to ensure that the FaceTime connection was made, and then the patient was oriented to the technology and introduced to the virtual provider. The nurse then left the examination room but remained in close proximity for the management of any acute situation. Documentation and clinical care were otherwise unchanged.
DATA COLLECTION
We took a mixed methods approach to data collection. Quantitative data were obtained from the VHA Corporate Data Warehouse to identify in-person and telemental health encounters along with patient, demographic, and operational data (e.g., visit duration). Next, anonymous paper feedback forms were attached to the telemental health carts. Patients and providers were asked to complete a four question survey based on the Telehealth Usability Questionnaire11 and included a 5-point Likert scale about the access, sound quality, satisfaction, and perceived value of telemental health (Supplementary Fig. S1). Open-ended feedback about the intervention was also solicited. Staff were reminded monthly to complete the cards and to request patients to do so as well. Qualitative data were collected through semistructured interviews conducted by two investigators (XXX, XXX) from October to December 2020 from providers who used the intervention. Interview questions were structured according to the Reach, Effectiveness, Adoption, Implementation, Maintenance (RE-AIM) framework.12
PARTICIPANTS
Participants for this project were Veteran patients and VA staff who used the telehealth technology. For interviewees, we worked with leadership at both sites to identify staff, nurses, emergency, and mental health providers who regularly used the telemental health system. This project was reviewed by the Institutional Review Board at the VA (Hospital System) and was classified as “non research.” We obtained verbal consent before all interviews, and the interview guide (Supplementary Fig. S2) was reviewed and approved by the local labor unions for both campuses.
ANALYSIS
The telemental health intervention was categorized into the following phases: preimplementation (December 1, 2019–February 29, 2020), wash-in (March 2020), postimplementation (April 1, 2020–June 30, 2020), and sustainability (July 1, 2020–January 31, 2021). The wash-in phase coincided with physical distancing restrictions enacted as part of the COVID-19 pandemic response but was planned a priori. Thematic analysis guided by RE-AIM was conducted by both investigators (M.J.W. and C.D.M.) and reviewed by the investigative team. Representative quotes for each domain were also identified and reviewed using audio recordings to ensure fidelity.
RE-AIM domains
We assessed Reach, that is, access to mental health care in the emergency setting, from the patient's orientation. Effectiveness was assessed by comparisons of ED/UCC lengths of stay stratified by disposition (e.g., discharged), return visits, hospitalizations, and outpatient clinic follow-ups within 7 days, 30-day mortality, and satisfaction through the surveys. Adoption was evaluated as the number of eligible mental health providers who conducted emergency telemental health consultations at either campus. Interviews focused on staff, ED, and mental health providers' experiences with and perceptions of delivering care using telemental health. Interviews also sought to understand staff and provider perception of timeliness, satisfaction, and the quality of telemental health care delivered. Implementation was assessed through a survey question addressing sound quality and through interviews on the implementation itself, barriers, and adaptations needed. Finally, use of the intervention at 10 months and interviews to understand perceptions of sustainability were used to assess Maintenance.
Results
Patient demographics, consults, and follow-up are presented in Table 1. We received 66 feedback cards from patients (n = 17) and providers (n = 49) postimplementation. Representative quotes from 16 interviews, including staff and nurses (n = 10), emergency physicians (n = 3), and mental health providers (n = 3), are presented in Table 2 and summarized by RE-AIM dimension below.
Table 1.
Patient Demographics for Consults Performed Pre- and Postimplementation of Telemental Health
| IN-PERSON |
TELEMENTAL HEALTH |
|||
|---|---|---|---|---|
| PREIMPLEMENTATION |
POSTIMPLEMENTATION |
PREIMPLEMENTATION |
POSTIMPLEMENTATION |
|
| n = 502 | n = 67 | N/A | n = 319 | |
| Total mental health ED/UCC visits | ||||
| ED, n (%) | 242 (48) | 36 (54) | — | 162 (51) |
| UCC, n (%) | 262 (52) | 31 (46) | — | 157 (49) |
| Visits by age, year | ||||
| 18–44, n (%) | 154 (31) | 16 (24) | — | 102 (32) |
| 45–64, n (%) | 247 (49) | 30 (45) | — | 158 (50) |
| ≥ 65, n (%) | 101 (20) | 21 (31) | — | 59 (18) |
| Visits by sex | ||||
| Female, n (%) | 42 (8) | 9 (13) | — | 29 (9) |
| Male, n (%) | 460 (92) | 58 (87) | — | 290 (91) |
| Visits by Race/Ethnicity | ||||
| American Indian or Alaska Native, n (%) | 4 (1) | 0 (0) | — | 0 (0) |
| Black, n (%) | 120 (24) | 16 (24) | — | 68 (21) |
| Native Hawaiian or other Pacific Islander, n (%) | 1 (0) | 0 (0) | — | 5 (2) |
| Unknown, n (%) | 25 (5) | 2 (3) | — | 14 (4) |
| White, n (%) | 352 (70) | 49 (73) | — | 232 (73) |
| Visits by rurality | ||||
| Rural, n (%) | 195 (39) | 20 (30) | — | 118 (37) |
| Urban, n (%) | 278 (56) | 44 (66) | — | 191 (60) |
| Missing, n (%) | 29 (6) | 3 (4) | — | 10 (3) |
| Disposition | ||||
| Discharge, n (%) | 142 (28) | 15 (22) | — | 79 (25) |
| Admission (medicine), n (%) | 63 (13) | 6 (9) | — | 37 (12) |
| Admission (mental health), n (%) | 133 (26) | 16 (24) | — | 124 (39) |
| Transfer (VA), n (%) | 83 (17) | 13 (19) | — | 28 (9) |
| Transfer (non-VA), n (%) | 2 (0) | — | — | 2 (1) |
| Other, n (%) | 2 (0) | — | — | — |
| Missing, n (%) | 77 (15) | 17 (25) | — | 49 (15) |
| Length of stay | ||||
| Discharge, median (IQR) | 7.5 (5.7–11.6) | 9.6 (7.7–11.0) | — | 6.2 (4.7–8.2) |
| Admission (medicine), median (IQR) | 13.5 (9.1–17.5) | 12.9 (12.6–14.4) | — | 14.1 (9.1–17.5) |
| Admission (mental health), median (IQR) | 6.3 (4.6–8.2) | 7.5 (4.8–9.1) | — | 5.8 (4.7–8.8) |
| Transfer (VA), median (IQR) | 15.4 (11.3–18.0) | 19.0 (15.0–19.2) | — | 14.5 (11.6–17.0) |
| Transfer (non-VA), median (IQR) | 19.3 (17.6–20.9) | — | — | 32.1 (31.8–32.4) |
| Other, median (IQR) | 11.5 (11.5–11.5) | — | — | — |
ED, emergency department; IQR, interquartile range; N/A, not applicable; UCC, urgent care clinic.
Table 2.
Reach, Effectiveness, Adoption, Implementation, Maintenance Domain Representative Quotations
| Reach |
| “Probably 95 to 96% of the patients like [tele-mental health] better”—ED/UCC Nurse/Staff 1 “I think patients get more time with the doc, they aren't rushed.”—Mental Health Provider 3 “I think that they like it because they put their mask down in the room and talk”—ED/UCC Provider 1 “Folks who are being seen in the regular [behavioral health clinic] a lot of them are being seen by telephone so when they come into the ER I've had a multiple anecdotal reports that they've really enjoyed seeing their providers and interacting with them”—Mental Health Provider 3 “I was surprised, even psychotic patients do okay with the iPad.”—ED/UCC Nurse/Staff 7 “I would say only a handful of patients refused and are predominately the ones who refused are alcohol detox who we have a more challenging time with getting them to be accepting to the iPad, if anger issues arise…mostly patients we would have had a difficult time with generally”—ED/UCC Nurse/Staff 2 “[Telehealth] was something new so they were not embracing it and [now] they realize it works, it's better, patients accept it. I've not had a single patient who has declined to be seen by telehealth. I think it's a win-win situation.”—Mental Health Provider 2 |
| Effectiveness |
| “The physical transit time has been eliminated for the psychiatrists, which improves timeliness for everything.”—Mental Health Provider 1 “Quicker access to the doctors instead of waiting for them to come down; they're calling on the iPads much quicker so the turnaround time is quicker for the patients.”—ED/UCC Nurse/Staff 1 |
| Adoption |
| “I think without coronavirus occurring we would've gotten here eventually but it would've taken much longer”—Mental Health Provider 1 “During the pandemic it's allowed us to have higher risk providers to work from home and perform the same work so that's been a huge benefit…the amount of sick leave providers take has gone down dramatically.”—Mental Health Provider 1 “Every step in the process seems to be a little faster.”—Mental Health Provider 1 “If one provider at the [alternative site] is a little backed up or if they have numerous consults then a provider at [original site] can help them out..”—Mental Health Provider 1 “I don't think [telemental health has] affected any clinical judgement or decision-making at all.”—Mental Health Provider 1 “It's just like seeing a patient in person. The fears were just like ‘how am I going to do this, am I going to press the right buttons, how am I going to connect this' you know those fears like buying a new car where you don't know where the knobs and buttons are”—Mental Health Provider 2 |
| Implementation |
| “The only problem is if the patient is really hard of hearing. Then I have to tell the [mental health provider] to speak louder…Sometimes it's a problem if I miss when they call the IPad”—ED/UCC Nurse/Staff 6 “We've been faxing [physical documents] into the emergency department and I think that's worked pretty well.”—Mental Health Provider 1 “Face-to-face interactions are more personal than iPads”—ED/UCC Nurse 8 “For me initially there was an awkward period where it was difficult to develop a rapport with the patient on the other end…but I no longer feel that.”—Mental Health Provider 1 “When they come out of the room from speaking with the patient [staff] can have that face-to-face [about] the next step and plan for the patient rather than if they hang up then we either wait for them to call back or we have to re-page them to see is the patient going to be discharged or admitted”—ED/UCC Nurse/Staff 3 “We've had a couple of patients walk out upset because they couldn't see somebody in-person and we don't want to lose those patients to not being seen.”—ED/UCC Nurse/Staff 1 |
| Maintenance |
| “We've lost quite a few psychiatrists that have taken full-time telepsych jobs elsewhere”—Mental Health Provider 1 “I personally don't want to go back to the old situation.”—Mental Health Provider 2 “Almost all of [mental health providers] have told me that if this is available for either a part-time or full-time option after the pandemic they would choose to continue with telepsych.”—Mental Health Provider 1 “Ultimately I feel it rests with administration…and the number of [relative value units] and things like that that can be generated from it is going to be the determining factor as to whether it continues”—Mental Health Provider 3 |
REACH
Number and proportion of mental health consultations by month and modality can be seen in Figure 1. During preimplementation, there were 502 mental health consultations, compared with 386 postimplementation (n = 319, 83% by telemental health). Demographics of the consultations were similar between preimplementation in-person compared with telemental health consultations postimplementation. The distribution of visits by site (ED vs. UCC), age group, sex, race, and rurality was unchanged. However, in-person visits postimplementation were older, more likely to be female, Black, and urban.
Fig. 1.
Proportion of telemental health consultations conducted during implementation phases: preimplementation (December 1, 2019–February 29, 2020), wash-in (March 1, 2020–March 31, 2020), postimplementation (April 1, 2020–June 30, 2020), and sustainability (July 1, 2020–January 31, 2021) phases.
Feedback from 64 responders, including 16 Veterans, indicated that 97% agreed that the intervention “improved access” to mental health services. Interviews (Table 2) identified that staff and providers felt that Veterans had positive experiences with telemental health, likely due to the timeliness of care. Furthermore, patients were willing to try telemental health because of the ongoing COVID-19 pandemic. Telemental health also offered advantages over other forms of unscheduled mental health care provided during the pandemic (e.g., telephone) such as the ability to take down one's mask during the virtual encounter.
EFFECTIVENESS
Among 66 survey respondents, 99% agreed that they were very satisfied with the intervention, and 96% agreed that the intervention added value to their care. Safety and efficacy outcomes (Fig. 2) did not decline postimplementation. Post-ED/UCC follow-up in any VHA outpatient clinic was high; 65% preimplementation and 79% postimplementation. Return 7-day ED/UCC visits were 5% preimplementation and 9% (in-person) and 5% (telemental health) postimplementation. Among patients who received a mental health consultation, there was one 30-day death among the 502 encounters preimplementation compared with three in the postimplementation group among the 386 ED/UCC encounters. While one of the deaths in the telemental health group was the result of suicide, this patient had been admitted to the Psychiatry inpatient service after emergency telemental health evaluation.
Fig. 2.
(A) Seven-day outpatient clinic and (B) 7-day ED revisit and hospitalization rates and 30-day deaths among patients seen in-person versus telemental health pre- and postimplementation. ED, emergency department.
There were substantive changes in the number of patients who sought ED care and changes in operations overall due to the COVID-19 pandemic; therefore, it is difficult to separate potential impact of implementation of telemental health from changes due to COVID-19 for these measures. However, we did not detect differences in the proportion of patients discharged, hospitalized with the mental health or medical services, or transferred. ED/UCC lengths of stay (Table 1) were similar before and after implementation. Among patients admitted to the mental health service, ED/UCC length of stay was 6.3 (interquartile range [IQR] 4.6–8.2) h preimplementation, while it was 5.8 (IQR 4.7–8.8) h for those who received telemental health. Among patients who were discharged from the ED/UCC, length of stay was 7.5 (IQR 5.7–11.6) h preimplementation, compared with 6.2 (IQR 4.7–8.2) h postimplementation. Finally, interviewees identified that the perception of improved speed and efficiency of care contributed to increased patient satisfaction (Table 2).
ADOPTION
Out of 24 physicians, 5 advanced practice providers, and 15 resident physicians (n = 44) scheduled to work during these periods, 100% participated in telemental health consultations. Both patients (94%) and providers (98%) agreed that telemental health improved access to mental health care. Interviewees (Table 2) identified the ongoing COVID-19 pandemic as an important facilitator of adoption, but would have otherwise occurred eventually. Using telemental health allowed some mental health providers with clinical comorbidities to reduce their exposure risk through remote work. Mental health providers commented on how telemental health improved their efficiency and benefited patients by facilitating multitasking (e.g., electronic health record review), improving response time to initiate consultations, and through a reduction in interruptions. Telemental health also improved the ability to quickly respond across institutions to a surge in consultations, and mental health providers felt that the quality of care was similar to in-person care. Prior telehealth experience facilitated provider interest in adopting telemental health, and initial apprehension to switch to telemental health was rapidly overcome by the COVID-related policy change requiring its use.
IMPLEMENTATION
Among 66 survey respondents, 99% agreed that the sound quality was good. Interviewees (Table 2) identified minor technical barriers that were easily overcome. Mental health providers identified work-arounds to handle physical paperwork, particularly for involuntary psychiatric holds. Mental health providers found cognitive assessments (e.g., drawing clocks) more challenging, but addressable with minor modifications. Interviewees identified that the timeliness benefit from telemental health could be compromised when multiple patients were waiting, and they noted that this was addressed through the addition of a second telehealth unit.
One area where staff and providers had mixed perceptions was the ability to build rapport through virtual consultations. Some nurses reported that the switch to virtual care impeded their awareness of disposition planning for the patient, while others said it improved communication with consultants. Finally, several staff interviewed said that they felt that a mental health provider should be available in-person, if needed.
MAINTENANCE
Following implementation, we tracked use for an additional 10 months and a total of 1,010 telemental health consultations. Use of telemental health remained steady at 82% of all consultations ranging from 73% (December 2020) to 88% (June 2020; Fig. 1). Mental health providers (Table 2) reported a desire to continue with telehealth after the pandemic due to enhanced job satisfaction. Furthermore, its use provided an alternative to private sector jobs that similarly offered telehealth. Better integration of the telemental health units with other VHA software applications was identified as an area for improvement, while long-term viability may be determined by how productivity is measured.
Discussion
Our mixed-methods evaluation of the implementation of a telemental health intervention found that provider-to-provider telemental health in unscheduled settings was well received by patients, staff, and providers and was sustainable. Our evaluation identified four major themes, including timeliness, initial apprehension to try telemental health, the COVID-19 pandemic, and sustainability.
Timely responsiveness without compromising care quality was a major contributor to patient and staff satisfaction along with ongoing use. Brief surges in patient volume and to a lesser degree minor technical issues threatened to compromise these benefits but were easily addressed through work-arounds and the addition of extra iPads. Improved timeliness was achieved through reduced travel time and multitasking not feasible previously. Moreover, the use of telemental health created “virtual capacity” for mental health providers to see consults across clinical sites. An important consideration is the safety and efficacy of psychiatric assessment by telehealth, particularly among patients at-risk for suicide. The fact that the only patient in this study who died by suicide was appropriately referred for Psychiatric admission based on the telemental health evaluation supports the safety and efficacy of evaluating high risk patients by telehealth.
In the context of COVID-19, telemental health also provided a source of safety by protecting providers in a form of “electronic personal protective equipment”13 while still allowing them to perform their duties. Considering that 46% of physicians experience at least one symptom of burnout at a higher rate than the general U.S. worker population,14 and nearly half of health care workers reported serious psychiatric symptoms during the pandemic,15 telemental health became an unexpected source of satisfaction and support during the COVID-19 pandemic.
Another theme was the initial reluctance of staff and providers to use telemental health, particularly among those who had never used telehealth before. While not necessary for eventual implementation, the pandemic accelerated adoption by providing universal acceptance of the benefits of telemental health to limit viral spread. However, future interventions are unlikely to be facilitated by a global pandemic. This work identified that getting staff and providers to use an intervention, even briefly, may help overcome provider reticence. Moreover, this work underscored the importance of effective communication about the significance of an intervention.
Sustainability in the 10 months postimplementation was high and likely due to widespread satisfaction with the intervention. Further areas for development include enhanced usability, expansion to other sites, addressing minor technical problems, and performance measurement. Most important is how sustainability of the intervention will be impacted once the pandemic resolves. While decreasing viral spread may diminish enthusiasm to use telehealth, widespread use of telehealth during the pandemic may be a tipping point for ongoing use.
The next consideration is whether in-person mental health providers are needed as backup. Although in-person care was infrequently needed, one solution may be through the use of emergency care providers who are already on-site. Emergency providers can sign involuntary holds and order temporizing measures for emergency situations (e.g., combative patients). Thus, on-site mental health providers may not be needed in emergency care settings. Finally, while mental health providers felt that the quality of care and decision-making were comparable and follow-up data were broadly supportive of this sentiment, evaluating this question is an important future direction.
Limitations
Two limitations warrant consideration in the evaluation of our results. First, while we provide a diverse set of implementation outcomes guided by RE-AIM, our results may not generalize to other unscheduled settings. Second, the ongoing pandemic may have substantially altered behavior of providers and patients. Potential impact from telemental health may be confounded by changes in patient need for services and altered clinical operations due to the COVID-19 pandemic. However, we did not find evidence that these outcomes were negatively impacted by telemental health, and it is possible, for example, that telemental health was beneficial (e.g., shorter length of stay). How this impacts sustainability as the incidence of SARS-CoV-2 wanes remains an important topic for future investigation.
Conclusions
Implementation of telemental health in acute unscheduled settings represents an important direction toward providing such services virtually and more broadly. Telemental health was well-liked and felt to be comparable in quality to in-person care. These likely contributed to the high rates of sustained use. The resolution of the pandemic and expansion to other sites represent important future directions for research.
Supplementary Material
Acknowledgments
The authors thank the many ED and UCC nurses who were instrumental in the implementation of this program; in particular, Amanda Moon, RN, and Jayme Jarvis, RN.
Authors' Contributions
All authors meet the definition of authorship for this article. Each has made substantial contributions to the conception and/or design of the work, the acquisition, analysis, or interpretation of data for the work. Furthermore, they have either drafted or made critical revisions for important intellectual content and have made final approval of the version to be published and agree to be accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved.
Content
The content is solely the responsibility of the authors and does not necessarily represent the official views of the United States Government or any of the funding sources.
Disclosure Statement
No competing financial interests exist.
Funding Statement
This project was supported by funding from the VA Office of Rural Health (VA ORH-10808). Both Drs. Ward and McNaughton receive support from the VA Geriatric Research Education Clinical Center, Tennessee Valley Healthcare System, VA Health Services Research and Development (IIR-19-134), and NIH (R21HL140381). In addition, Dr. Ward was supported by K23 HL127130, and Dr. McNaughton reports receiving support from Pfizer. Dr. Mohr additionally received support from the Agency for Healthcare Research and Quality (K08HS025753) and the Rural Telehealth Research Center, funded by the Federal Office of Rural Health Policy (U1C-RH29074), Health Resources and Services Administration (HRSA).
Supplementary Material
REFERENCES
- 1. Owens PL, Mutter R, Stocks C. Mental health and substance abuse-related emergency department visits among adults, 2007: Statistical brief #92. In: Healthcare cost and utilization project (HCUP) statistical briefs. Rockville (MD), 2006. Available at https://www.hcup-us.ahrq.gov/reports/statbriefs/sb92.pdf (last accessed May 13, 2021). [PubMed]
- 2. Santillanes G, Axeen S, Lam CN, Menchine M. National trends in mental health-related emergency department visits by children and adults, 2009–2015. Am J Emerg Med 2020;38:2536–2544. [DOI] [PubMed] [Google Scholar]
- 3. Mohr NM, Wu C, Ward MJ, et al. Potentially avoidable inter-facility transfer from Veterans Health Administration emergency departments: A cohort study. BMC Health Serv Res 2020;20:110. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4. Egede LE, Acierno R, Knapp RG, et al. Psychotherapy for depression in older veterans via telemedicine: A randomised, open-label, non-inferiority trial. Lancet Psychiatry 2015;2:693–701. [DOI] [PubMed] [Google Scholar]
- 5. McNaughton CD, Bonnet K, Schlundt D, et al. Rural interfacility emergency department transfers: Framework and qualitative analysis. West J Emerg Med 2020;21:858–865. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6. Baum A, Kaboli PJ, Schwartz MD. Reduced in-person and increased telehealth outpatient visits during the COVID-19 pandemic. Ann Intern Med 2021;174:129–131. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7. Connolly SL, Stolzmann KL, Heyworth L, et al. Rapid increase in telemental health within the department of veterans affairs during the COVID-19 pandemic. Telemed J E Health 2021;27:454–458. [DOI] [PubMed] [Google Scholar]
- 8. Mott JM, Grubbs KM, Sansgiry S, Fortney JC, Cully JA. Psychotherapy utilization among rural and urban veterans from 2007 to 2010. J Rural Health 2015;31:235–243. [DOI] [PubMed] [Google Scholar]
- 9. Czeisler ME, Lane RI, Petrosky E, et al. Mental health, substance use, and suicidal ideation during the COVID-19 pandemic—United States, June 24–30, 2020. Mmwr-Morbid Mortal W 2020;69:1049–1057. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10. Pinnock H, Barwick M, Carpenter CR, et al. Standards for reporting implementation studies (StaRI) statement. BMJ 2017;356:i6795. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11. Parmanto B, Lewis AN Jr., Graham KM, Bertolet MH.. Development of the telehealth usability questionnaire (TUQ). Int J Telerehabil 2016;8:3–10. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12. Glasgow RE, Vogt TM, Boles SM. Evaluating the public health impact of health promotion interventions: The RE-AIM framework. Am J Public Health 1999;89:1322–1327. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13. Turer RW, Jones I, Rosenbloom ST, Slovis C, Ward MJ. Electronic personal protective equipment: A strategy to protect emergency department providers in the age of COVID-19. J Am Med Inform Assoc 2020;27:967–971. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14. Shanafelt TD, Boone S, Tan L, et al. Burnout and satisfaction with work-life balance among US physicians relative to the general US population. Arch Intern Med 2012;172:1377–1385. [DOI] [PubMed] [Google Scholar]
- 15. Young KP, Kolcz DL, O'Sullivan DM, et al. Health care workers' mental health and quality of life during COVID-19: Results from a mid-pandemic, national survey. Psychiatr Serv 2021;72:122–128. [DOI] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.