TELEHEALTH IN EMERGENCY MEDICINE: A CONSENSUS CONFERENCE TO MAP THE INTERSECTION OF TELEHEALTH AND EMERGENCY MEDICINE

Abstract

Introduction

Telehealth has the potential to significantly change the specialty of Emergency Medicine (EM) and has rapidly expanded in EM during the COVID pandemic; however, it is unclear how EM should intersect with telehealth. The field lacks a unified research agenda with priorities for scientific questions on telehealth in EM.

Methods

Through the 2020 Society for Academic Emergency Medicine’s annual consensus conference, experts in EM and telehealth created a research agenda for the topic. The multi-year process used a modified Delphi technique to develop research questions related to telehealth in EM. Research questions were excluded from the final research agenda if they did not meet a threshold of at least 80% of votes indicating “important” or “very important”.

Results

Round 1 of voting included 94 research questions, expanded to 103 questions in Round 2, and refined to 36 questions for the final vote. Consensus occurred with a final set of 24 important research questions spanning five breakout group topics. Each breakout group domain was represented in the final set of questions. Examples of the questions include: “Among underserved populations, what are mechanisms by which disparities in emergency care delivery may be exacerbated or ameliorated by TH” (Health Care Access) and “In what situations should the quality and safety of TH be compared to in-person care and in what situations should it be compared to no care” (Quality and Safety).

Conclusion

The primary finding from the process was the breadth of gaps in the evidence for telehealth in EM, and telehealth in general. Our consensus process identified priority research questions for the use of and evaluation of telehealth in EM to fill the current knowledge gaps. Support should be provided to answer the research questions to guide the evidenced-based development of telehealth in EM.

Introduction

Telehealth—the use of various telecommunication technologies to provide a broad range of healthcare services (Figure 1)— was an emerging platform of Emergency Medicine (EM) care delivery prior to the COVID pandemic. Using video-based telehealth, experienced emergency nurses and board-certified EM physicians provide guidance to remote clinicians, in the continental United States (US) and to other medical providers or staff in remote locations outside of the US, such as in airline and maritime medicine. These provider-to-provider programs leverage EM expertise to fill gaps in care at sites external to the brick-and-mortar Emergency Departments (EDs). Emergency physicians staff several of the direct-to-patient or direct-to-consumer programs, either part of their group practice or as independent contractors. Tele-triage facilitates the triage process and can reduce the time from ED arrival to provider evaluation. In tele-triage, an EM physician provides clinical guidance or order entry following a live-streamed video conversation with the patient in ED triage. Tele-triage enables a physician to oversee triage at multiple sites, e.g., multiple EDs in the same health care system or multiple triage locations within a large ED. Most of the EM use of telehealth prior to COVID was limited to video-based visits with little use of asynchronous telehealth modalities such as store-and-forward technology or mHealth.

Figure 1.

Description and examples of types of telehealth.

Even prior to the pandemic, the importance of telehealth in EM care delivery was increasing, with many in the field seeking to answer broad questions with regards to how telehealth fits into EM care and how EM will shape the future of telehealth. On September 22 and 24, we held the 2020 Society for Academic Emergency Medicine (SAEM) Consensus Conference, Telehealth in Emergency Medicine: A Consensus Conference to Map the Intersection of Emergency Medicine and Telehealth, the culmination of a multi-year process. During this conference, experts from EM and telehealth convened virtually to create consensus around the highest telehealth research priorities. While the consensus conference was planned prior to the discovery of the novel coronavirus, the COVID pandemic highlighted the importance of telehealth in EM. Adoption and use accelerated dramatically during the COVID pandemic; telehealth visits throughout the US medical system increased more than 175 times the pre-COVID baseline.¹ Specifically in emergency medicine (EM), telehealth rapidly expanded during COVID, with novel uses such as video-conference via electronic tablets in patient rooms to decrease personal protective equipment (PPE) usage^2,3 and widespread expansion of existing practices such as remote connection of specialists and EM patients.⁴ Innovative uses of telehealth in EM also controlled the flow of patients into the ED further protecting ED staff and patients.^2–4 Some EDs even provided remote patient monitoring to COVID+ patients at ED discharge.⁵

Research to date has rendered mixed results for the efficacy and impact of telehealth services, which makes a robust research agenda essential to provide the scientific basis to guide the development of telehealth in EM. Given the pressures from patients, payors, and healthcare institutions to provide increased access to high quality care at decreased costs, along with the accelerated use of telehealth during the pandemic, further study is warranted as to whether telehealth can provide solutions. Emergency medicine serves as a microcosm of the healthcare system and can provide valuable insight to how telehealth could address current and future problems in healthcare. Emergency Medicine leaders must understand telehealth and advocate for telehealth-related legislation and policies that enable rather than obstruct the use of telehealth in EM.

In seeking to create a comprehensive telehealth research agenda in EM, the consensus process involved a diverse interdisciplinary group that discussed themes and priorities informed by national experts both within and outside of EM. By bringing together researchers, educators, and telehealth experts, we sought to examine the intersection of telehealth and EM to inform a future research agenda and guide messaging to policy makers regarding regulatory policy related to telehealth delivery.

Methods

We used a nine-step modified Delphi method⁶ to identify and reach agreement amongst a group of relevant stakeholders in EM and telehealth on key research priorities in telehealth and EM. With this iterative process, we aimed to reach consensus on the key questions from the breakout topics, as well as eliminate areas that stakeholders identified as lower priority. Details of the steps can be found in Figure 2.

Figure 2.

Timeline and description of the 9-step modified Delphi consensus process.

The first step began in 2016 and included a background literature search and consensus between content experts. This culminated in the five breakout group topics: (1) Educational Needs and Outcomes, (2) Healthcare Access, (3) Quality and Safety, (4) Research Facilitation, and (5) Workforce.

The second step began in the Fall of 2019 with the creation of breakout groups with an assigned content expert for each of the five groups. Emails to the SAEM Telehealth Interest Group recruited breakout group members. Breakout group leaders led discussions with their members and developed questions in their respective topics via electronic media and conference calls. The breakout group leaders and conference co-chairs determined an overarching conceptual framework did not exist for the large domain of telehealth. Instead, the breakout groups used conceptual frameworks that were specific to their domains. No limitations or requirements existed for the number of questions generated; however, the planning committee recommended a final 3–5 research questions for each breakout group. The breakout groups submitted to the co-chairs their initial summaries of the state of the science and potential research questions. Co-chairs reviewed the five summaries and ensured no overlap occurred.

The third step occurred three months preceding the original conference date. Relevant stakeholders within and outside of EM (Figure 3) provided feedback on the breakout groups’ summaries. Using SurveyMonkey (San Mateo, California) the following questions provided stakeholder feedback: (1) Are there any subject areas within this topic that are not represented here that you suggest we add, (2) Are there any subject areas within this topic that you suggest we remove, (3) Are there any important resources or seminal research articles that are not listed here, and (4) Any further comments?

Figure 3.

List and timeline of stakeholders who reviewed and/or voted on the research questions.

Due to the cancellation of the original conference due to the COVID Public Health Emergency, we added a fourth step to incorporate any new research or insights from the explosion of telehealth during the COVID pandemic. The breakout group leaders revised their summaries using the initial stakeholder survey feedback and a virtual conference “Telehealth in Emergency Medicine during COVID: Lessons Learned”.⁴ A second stakeholder survey administered in July 2020, identical to the first stakeholder survey, provided targeted feedback to the breakout groups. The stakeholders who provided feedback in this round can be found in Figure 4.

Figure 4.

List of stakeholder representatives providing feedback to the Breakout Group summaries.

The fifth step occurred the month preceding the rescheduled conference date. After the breakout groups revised their summaries with feedback from step four, each preregistered conference participant received all five of the breakout group summaries and a survey to rank the importance of the questions developed by the breakout groups. The ranking used a 5-point Likert scale with the following question stem, “As an area of research, the following question is:” with the following potential responses: “not important”, “somewhat important”, “neutral”, “important”, “very important”. Participants could suggest new questions or gaps and provide comments in a free-text section. Breakout group leaders revised their summaries and research questions in light of the participant feedback from the preconference survey. Prior to the first voting (preconference survey), the Co-chairs determined a priori that questions receiving >80% responses with “important” or “very important” would remain on the research agenda and dropped the questions that did not meet this threshold.

The sixth step occurred on Day 1 of the virtual conference. The conference included three keynote speakers who provided different perspectives on EM telehealth. Former SAEM President and nationally recognized telehealth expert, Dr. Judd Hollander, discussed the myths surrounding the use of telehealth. Dr. Bisan Salhi, an EM physician researcher and expert on homelessness and high utilizers of ED care challenged the participants’ assumptions on how to engage those who may be least included in telehealth programs. Aaron Martin, Executive Vice President and Chief Digital Officer of Providence St. Joseph Healthcare provided a snapshot of telehealth use in current progressive healthcare systems.

Along with the keynote presentations, two facilitated panel discussions included a diverse array of stakeholders including patients, providers, and representatives from the American Association of Medical Colleges, National Quality Foundation, Society for Education and Research in Connected Health Society, Telehealth Resources Centers and many others. The panelists provided a broad stakeholder perspective for the conference participants to consider when voting on the research agenda. Patients and patient advocates gave critical patient perspectives on the topic throughout the consensus planning process as well as during the conference breakout group discussions.

On Day 1 of the virtual conference, the breakout group leader led the breakout sessions. Due to resource constraints, the Healthcare Access and the Quality and Safety groups met concurrently as did the Educational Needs and Outcomes, Research Facilitation, and Workforce groups. Each attendee participated in two separate breakout group discussions on each of the conference days.

The breakout group leader began the sessions with a presentation of the topic and the research questions. Telehealth content experts further described the breakout group topic with short presentations followed by open dialog amongst the participants. During the pre-conference rehearsals with the breakout group leaders, the planning committee reviewed clear goals and objectives of the sessions and voting to mitigate inter-group variation. The following guidelines were established: each breakout group could not be subdivided into smaller discussion groups, discussions should be managed so that all participants were heard, and priority should be given to the patient and patient representatives.

Scribes documented discussions during the sessions (Scribe America, Fort Lauderdale, Florida). The scribes shared the notes with the respective breakout groups leaders at the end of the live virtual conference days.

At the end of the Day 1 breakout session, participants completed a survey of the breakout group’s research questions using Poll Everywhere (Poll Everywhere, San Francisco, CA). Not all breakout groups were able to discuss each of the research questions; therefore, participants completed a separate survey on the evening of Day for Round 2 of voting on the importance of the current questions for the research agenda.

The seventh step occurred on Day 2 of the conference. The same five breakout groups met again, and the breakout group leader led a brief discussion of those items that had not reached >80% in Round 2 of voting. Participants could make a case to rescue the questions from being discarded. After that, the floor opened to discussion on the questions that met the 80% threshold. That evening, the breakout group participants completed a survey as a last round (Round 3) of voting.

The eighth step occurred after the conference. The post-conference evaluation included the following question, “Do you have any further feedback or concerns about the final research agenda presented on September 24th?” This question served to surface any disagreement that may not have been captured in the conference day.

The ninth step occurred after feedback from the post-conference evaluations were received. The breakout groups revised their summaries according to feedback from the conference day and the post-conference evaluation. The conference planning committee reviewed and incorporated these summaries into these conference proceedings.

Various methods obtained participants’ agreement on the research questions. The preconference stakeholder surveys and Round 1 of voting (preconference survey) used Survey Monkey (Survey Monkey, San Mateo, CA). Initially we planned to use Poll Everywhere (Poll Everywhere, San Francisco, CA) for voting within the breakout groups; however, it was not effective in the virtual conference setting. Specifically, the breakout sessions were too short for a detailed videocall-based discussion of all the research questions and the simultaneous use of Poll Everywhere to vote on each item took time away from the discussion. Instead, real-time discussions were prioritized. For any question posed on the wording of a potential research question, the breakout group leader would open the discussion and edit immediately. Ideally ample time would have been given for both discussion and voting. As most groups could not vote on priorities in the time allotted, the planning committee decided on the evening of Day 1 to use Survey Monkey for the voting for the remainder of the process, including for Rounds 2 and 3 of voting, as well as for the post-conference evaluation.

Results

The preconference survey included 94 candidate items for priority research questions developed by the five breakout groups. Of the 47 pre-registered participants, 38 (81%) responded to the preconference survey (step 5 from Methods) for Round 1 of voting (Pre-conference Survey, Appendix). Most of the preregistered participants came from teaching hospitals (89.5%) in urban settings (68.4%) (Table 1). Most preregistered participants practiced clinical EM (71%). Seventy-one per cent reported that they use or have used telehealth in their clinical practice where only 7.9% reported that they do not use telehealth, nor did they have any plans to create a program. Of the preregistered participants, less than a quarter reported having telehealth training programs for trainees. Participants rated the importance of all questions on the preconference survey and carried forward for the next round of voting.

Table 1.

Demographics of pre-registered participants. n=47

Characteristic	n (%)
Primary clinical setting
Rural	1 (2.6)
Suburban	5 (13.1)
Urban	26 (68.4)
N/A	6 (15.8)
Teaching Hospital
Yes	34 (89.5)
No	0 (0)
N/A	4 (10.5)
Practice Clinical EM
Yes	27 (71)
No	11 (28.9)
Experience in TH
I use/have used TH in my practice	27 (71)
There is no TH in my practice but there are plans to create a program	5 (13.1)
There is no TH practice and no plan to create a program	3 (7.9)
TH Training Program
Student	7 (18.4)
Resident	9 (23.7)
Fellow	9 (23.7)
Other	8 (21.1)

N/A: Not applicable

EM: Emergency Medicine

TH: Telehealth

At the live virtual conference, 93 unique attendees (excluding SAEM staff) participated; 88 attendees on Day 1 and 65 attendees on Day 2. Please see Table 2 for the attendance and Table 3 voting response rates for Breakout Group Day 1 and Day 2. Round 2 of voting included 103 research questions and Round 3 of voting included 36 research questions on the survey (Appendix).

Table 2.

Voting response rate and attendance for Breakout Groups on Day 1 (Round 2 of voting) and Day 2 (Round 3 of voting)

Breakout Group	Day 1	Day 2
Educational Needs and Outcomes	51.7 (15/29)*	55 (11/20)
Healthcare Access	72.2 (26/36)	75 (21/28)
Quality and Safety	66.7 (18/27)	61.9 (13/21)
Research Facilitation	72.2 (26/36)	76.9 (20/26)
Workforce	50 (9/18)	50 (7/14)

^*Denominator=attendance

Table 3.

Number of potential research questions per breakout group.

Breakout Group	Round 1	Round 2	Round 3
Educational Needs and Outcomes	16	17	10
Healthcare Access	28	31	9
Research Facilitation	4	7	2
Quality and Safety	20	21	5
Workforce	26	27	10
Total	94	103	36

For the post-conference survey, we had a 68.9% response rate (64/93). General feedback included the rushed feeling of the breakout group discussions. No participants objected to the final list of research questions. The final 24 questions for the Telehealth in Emergency Medicine Consensus Conference Research Agenda is listed in Figure 5.

Figure 5.

List of final research questions for research agenda

Discussion

While telehealth provided solutions during the COVID pandemic in delivery of acute unscheduled care, it is an emerging care platform with strengths and weaknesses that must be better delineated. This consensus process highlighted several key research gaps; accordingly, these gaps will require focused effort to close. Telehealth innovations in EM are being widely implemented, often without convincing evidence in the literature. We did not intend to halt the infusion of telehealth into EM with this conference, but to underscore the need for parallel research processes while these innovations occur.

Below are the findings from each breakout group, including the current state of the science and the final research questions. As stated in the Methods, an overarching conceptual framework did not exist for all five breakout group themes; however, each breakout group chose their own framework. We present the summaries in alphabetical order.

Educational Needs and Outcomes

The educational needs and outcomes for telehealth in EM are divided into these four domains: (1) core competencies and best practices, (2) best strategies to educate all EM providers, (3) assessment of clinical impact, and (4) use of telehealth for education.

Competencies

While a variety of societies and professions have published on general competencies in telehealth,^7–12 publications delineating telehealth competencies specific to EM are relatively few.^13,14 These publications in EM speak to the competencies for practicing physicians, medical students, and other clinicians using telehealth for providing patient care. Recognizing that EM telehealth is relatively new, there is a need for evaluation studies of EM health care professionals’ perception to validate and gain acceptance as part of EM practice.^15–17

In 2016 the American College of Emergency Physicians (ACEP) indicated that, with regard to telehealth, “an appropriate and adequate examination to establish a diagnosis or underlying condition” should be performed.¹⁸ Further, the “technology used must be adequate to enable an examination similar to that possible in a face-to-face encounter”. No other details outline the specifics of the remote exam to ensure similarity to an in-person patient encounter. The American Telemedicine Association (ATA) published practice guidelines noting that the remote provider’s examination may include “an explicit physician-guided self-examination which may include peripheral devices.”¹⁹ There are many publications on use cases for telehealth in EM. However, there is little mention of best practices of a modified exam for telehealth.

Educational Strategies

Training Modalities and Curriculum Integration

The need for telehealth training for medical students and residents is recognized by the American Medical Association (AMA) and AAMC^11,12,20 as well as learners.²¹ Training modalities in the literature are predominantly short didactics via online modules or face-to-face sessions. Blended methods that incorporate experiential learning in addition to didactics often involve either simulation with trainees role playing as a provider caring for a patient via telehealth using standardized patients.²² Other experiential learning included shadowing in clinical telehealth shifts or conducting a telehealth visit under supervision.²³ Formal telehealth education should incorporate both didactics and experiential learning; however, further research is needed to determine the most effective training modalities for telehealth education, especially as it applies to different aspects of telehealth education (e.g., “webside” manner, choosing most appropriate telehealth modality, etc.). Furthermore, a paradigm shift is needed to true mastery and competency of telehealth skills and concepts, which can be achieved through longitudinal integration telehealth education to existing curricula.

Training Content

Technical knowledge, such as how to use the telehealth video platforms, is cited as a key barrier to telehealth utilization in both the general telehealth and EM literature.²⁴ One evaluation of a telehealth-enhanced emergency care program noted that inadequate telehealth technician training led to low confidence levels and performance difficulties.²⁵ Literature on EM-specific use of tele-stroke and tele-ultrasound similarly cite inconsistent or lack of technical training as barriers to adoption and utilization.^26,27 A formalized training program has the potential to address key factors previously cited as barriers to provider adoption and engagement in telehealth.^28,29 Furthermore, providers familiar with technology are more inclined to use telehealth. No targeted studies in EM literature exist focusing on training in technical knowledge of telehealth.³⁰

Little empirical research is available on the critical soft-skills or unique telehealth etiquette skills necessary to conduct a successful telehealth visit. Currently there is no literature specific to EM practitioner training related to telehealth etiquette. While empathy and communication skills, including bedside manner, are part of healthcare training, these skills may not transfer intuitively to a telehealth encounter.³¹ Poor telehealth etiquette is often the cause of unsuccessful telehealth encounters.³² Emergency Medicine physicians should receive early training and assessment related to telehealth etiquette.³³

EM physicians regularly engage in interprofessional practice. Despite this, little EM research has focused on how to further develop collaboration through telehealth technologies. Other non-EM literature describes how telehealth facilitates collaboration across professions³⁴ and details an interprofessional telehealth curriculum for third year medical students.^35,36

Educational Outcomes

To ensure trainees are satisfying core competencies, we must assess the quality and impact of the telehealth training outside of satisfaction surveys. No EM literature has assessed if training in telehealth skills translate to improvements in clinical or financial outcomes. In the ACEP 2016 Policy paper on “Emergency Medicine Telemedicine”, guidelines describe provider orientation and training that include 1) understanding current local and state laws for the practice of telehealth, 2) “maintaining their technical and clinical competence”, 3) creating and following contingency plans for technology failures and 4) following telehealth protocols for escalating care as appropriate.¹⁸ However, no described minimum standard training requirement exists in EM literature.

Telehealth for Education

Tele-education including tele-simulation and tele-mentoring have been used increasingly by EM physicians.^37,38 A number of applications have been described for tele-mentoring, for example, Project ECHO and ultrasound skill training.^39–42 The literature describes applications of tele-simulation for resuscitation training, acute trauma care, and primary care.^37,38 During the COVID-19 pandemic, tele-simulation and tele-mentoring played an integral role in rapid education of providers.^43,44 While descriptive studies have been published regarding use of telehealth for education, no EM studies have evaluated the effectiveness of tele-education beyond self-report. No best practices or guidelines of tele-education have been published.

During the virtual synchronous consensus conference, discussion focused on wording of each individual question, specifically the breadth of each question. Discussion focused on broadening the scope of questions to encompass a larger research agenda while still providing definite guidance. Another consideration discussed was a desire to provide guidance without creating a framework that would become a checklist or “merit badge” for telehealth education/training.

It is notable that while all four domains had representative questions during the final vote (Figure 5), the seven questions that ultimately passed were focused on the first two domains, Core Competencies and Educational Strategies. Given the paucity of EM research to date in this field, this focus on core competencies and educational curricula likely represents a desire to focus the research agenda on establishing a solid base of research in these key areas before advancing to other aspects of telehealth education.

• Competencies

  • What are the core competencies in telehealth that are common to all providers, regardless of role, specialty, or level of training?
  • What gaps, if any, in current EM training need to be addressed to adapt practice to telehealth?
  • In patient-provider telehealth encounters, what are the components of the video-based physical exam?
• Educational Strategies

  • What types of educational experiences and instructional modalities are effective to teach telehealth to EM practitioners?
  • How do we train emergency practitioners in virtual presence (webside manner) for patient-to-provider and provider-to-provider encounters?
  • What are the best ways to integrate telehealth skills into both UME and GME EM curricula?
  • How do we train interprofessional EM teams to provide collaborative care via telehealth?

Healthcare Access in Emergency Care Delivery

Telehealth can improve access and quality of healthcare for patients in remote and other settings. For this Consensus Conference, we focused our review of this expansive topic on literature examining telehealth for healthcare access in emergency care delivery. The review is organized in 4 distinct content areas: (1) Access to Specialists; (2) Access to Specialists in Out-of-Hospital and Resource-Limited Settings; (3) Optimizing ED Utilization and Disposition Coordination; and (4) Enhancing Access for Underserved Populations.

We used a framework developed by the Supporting Pediatric Research on Outcomes and Utilization of Telehealth (SPROUT) group to identify gaps to guide future investigative efforts.⁴⁵ Priority research questions are categorized into the SPROUT domains: (1) Impact of telehealth on patient-level and population-level health outcomes; (2) Impact of telehealth on the quality of healthcare delivered; (3) Outcomes of the telehealth encounter and program (related to people, process, and tools); and (4) Implementation process measurements. Discussions during the consensus conference events underscored the importance of a patient-centered approach to the use and study of telehealth for healthcare access, and the need to ensure equity across populations.

Access to Specialists

Telehealth is frequently used to connect ED patients with specialists, most often for time-sensitive emergencies.^46–48 Telestroke has been a particularly successful model system with widespread implementation and has improved access to stroke specialists and reperfusion therapy.^{49,50,59–67,51–58} In trauma and critical illness, evidence suggests telehealth can optimize triage, reduce transfers, and better stabilize transferred patients, with decreased mortality and ICU admission rates.^{68,69,78–84,70–77} Among pediatric critical care patients, telehealth has been shown to decrease ICU admissions, transfers, medication errors, and improve provider satisfaction.^{29,85,94–100,86–93} In behavioral health, telehealth has reduced length of stay, improved transfer processes, and enabled “psychiatry clearance” and discharge planning.^101–109 Telehealth has also been used to expand access to buprenorphine treatment.¹¹⁰ Among nearly all these clinical applications, intermediate and process measures are often surrogates, but the evidence is otherwise fairly consistently lacking in studies evaluating the impact on patient- and population-level outcomes.

Access to Specialists & Specialist-guided Emergency Interventions in Out-of-Hospital & Resource-Limited Settings

Most prehospital telehealth research has focused on the impact on feasibility, reliability of prehospital technology, and its effect on time-sensitive illnesses, system resource utilization, and cost-effectiveness.^{111,112,121–123,113–120} High quality efficacy trials are limited; most studies are observational, descriptive, or simulation studies and demonstrations. Simulation studies suggest a range of benefits from prehospital telehealth including improved guideline/protocol adherence, shorter time-to-treatment for procedures, improved quality of prehospital notification, and fewer errors in handoffs. Technology may be used for remote supervision of EMS personnel in routine care or tasks with which they are familiar. In addition, the idea of tele-mentoring -- remote mentoring of unfamiliar procedures to less experienced caregivers -- has also been demonstrated to be feasible in simulated settings.^124–128 However, data on real-world implementation are needed. Additionally, only a few prehospital studies assess patient or population-level outcomes. In meta-analyses, prehospital ECG transmission was associated with improved in-hospital mortality, and mobile stroke units were associated with reduced time-to-treatment and hospital length of stay.^129–131 Evidence related to trauma management and pediatric care in the prehospital setting is limited. Across conditions, evidence evaluating longer-term patient outcomes is lacking. Evidence is also limited in more resource-limited and disaster settings. Published literature describes operational models for disaster and humanitarian response,^132–137 barriers to implementation and use, current and potential uses of telehealth applications in acute and recovery phases, and feasibility of novel technologies (e.g., robots, drones).^138–140 Future studies should identify potential points of system failure, quantify reliability of system performance in order to optimize implementation of systems with infrequent but high-stakes use, and evaluate patient- and population-level health outcomes.

Optimizing ED Utilization and Disposition Coordination

Telehealth is commonly used to reduce or improve ED utilization, crowding, disposition coordination, and transfer processes.^141–145 The use of telehealth for acute illness in non-acute care settings (e.g., nursing homes, correctional facilities, and schools) tends to lead to fewer ED transfers without negatively impacting patient outcomes.^146,147 Telehealth in skilled nursing facilities (SNF) has been shown to reduce transfers, lower costs, and be favorably perceived by stakeholders. However, research on this topic is limited and merits further investigation. As with the broader literature on remote patient monitoring and home health care delivery, it is challenging to craft an overview of interventions directly involving emergency care by EMS and the ED. Telehealth is also used by prehospital providers to avert unnecessary ED visits and improve resource utilization with favorable cost, quality, provider and patient satisfaction. Direct-to-consumer (DTC) telehealth for low acuity conditions may contribute to a small reduction in ED utilization, but some studies also suggest an additive rather than substitutive effect of DTC; more data are needed.^148,149 With respect to ED crowding, telehealth provides a potential solution through reducing volume, utilizing remote providers, or improving ED processes of care.^150–155 Results are mixed but evidence suggests that telehealth applications lead to reduced ED length of stay, and improved patient satisfaction. Reducing transfers and improving pre-transfer management are primary outcomes for many telehealth interventions as discussed above.^156–161 Evidence suggests that telehealth is infrequently used for transfer coordination in connecting a remote patient with a different hospital than the telehealth ‘hub’.¹⁶² Finally, in the setting of the ongoing COVID-19 pandemic, there has been increasing use of telehealth for post-ED discharge monitoring. Literature on implementation and outcomes of this application is limited.

Enhancing Access for Underserved Populations

Telehealth has been associated with improved access to specialists for patients in rural or under-resourced settings and has been associated with improvements in metrics such as length of stay, reduced transfers, increased discharges, improved costs, and similar quality of care, as discussed above.^163–172 Barriers to telehealth in rural EDs have previously been described and cost is the most frequently cited.^173–178 In addition to patients in rural settings, telehealth may also play a role in improving access for other underserved populations.^179–182 It is important to note, however, that there are limited data devoted to populations with limited English proficiency, low health literacy, residence in WIFI deserts, or other health-related social determinants that may influence accessibility.¹⁸³ Disparities along racial or ethnic lines in relation to access to emergency telehealth have not been well studied. Differences in telehealth uptake, adoption and use may be present amongst different groups, and there may certain health disparities may even be exacerbated in certain clinical situations, necessitating additional research to explore the intersection of social determinants of health and technology.¹⁸⁴

In summary, the evidence to date supports the feasibility of telehealth applications to improve emergency healthcare access in a variety of settings and conditions, as well as some measures of care efficiency (e.g., length of stay, transfer rates).^144,185 Despite the vast potential for telehealth to improve access to emergency care, there are limited objectively controlled data describing benefits to patient-level or population-level health outcomes. As a tool with potential to mitigate disparities in access, improved understanding of how telehealth influences patient access and health outcomes in rural and otherwise underserved settings is central.¹⁸⁶ Future research is needed to ensure that a technology intended to narrow disparities does not, in fact, widen them. We also noted in our review that there are limited data on the optimal approaches to educate the public about the benefits and downsides of telehealth encounters, or strategies for interacting with a provider during a telehealth encounter. More patient-centered approaches to telehealth access research will be valuable.

As mentioned previously, we categorized the research questions using the SPROUT framework. The final research questions are the following:

• Patient-, population-level health outcomes

  • How does emergency telehealth access vary by patient or population characteristics?
  • When considering the impact of telehealth for improving access, what are the appropriate patient-level outcomes to evaluate?
  • When considering the impact of telehealth for improving access, what are the appropriate population-level outcomes to evaluate?
  • What are costs and cost-effectiveness of telehealth from the perspective of the patient and the system, and relatedly, what is the appropriate approach to differentiate value of increased access vs excessive low-value utilization?
• Quality of healthcare delivery

  • Among underserved populations, what are mechanisms by which disparities in emergency care delivery may be exacerbated or ameliorated by telehealth?
• Outcomes of the telehealth encounter and program

  • What are the barriers and facilitators of implementation of telehealth in EDs (e.g., barriers such as payment models or healthcare delivery systems)?
• Implementation process measurements

  • What lessons can we learn from the expansion of telehealth during the COVID-19 pandemic?
  • What are the barriers and facilitators to improving access via telehealth and quality of care for underserved populations?

Quality and Safety

We sought to evaluate potential research gaps in quality and safety of emergency telehealth. Given the potential for overlap in content areas and in the setting of the COVID-19 pandemic rapidly accelerated adoption of telehealth in emergency care, we defined the scope and definition of these areas using organizing framework(s) to evaluate quality and safety. We used the Agency for Healthcare Research and Quality’s (AHRQ) simplified framework for quality which was adapted from the original Institute of Medicine’s (IOM) definition of quality.¹⁸⁷ AHRQ’s framework reduces the number of domains from six to three – effectiveness, patient- centeredness, and safety.

Effectiveness is care that is proven to work and maps to the original IOM dimensions of effectiveness, timeliness, and, in some cases, to efficiency. Patient-centeredness is care that is responsive to a patient’s needs and preferences and maps to the patient-centered and equity domains. Finally, Safety involves care that protects patients from medical errors and does not cause harm and maps to the safety dimension.

Effectiveness in Emergency Telehealth

We identified seven broad categories of models of emergency telehealth evaluating effectiveness: (1) Prehospital: Emergency Department (ED) to Outpatient Clinic, (2) Prehospital Emergency Medical Services (EMS): ED to ambulance staff, (3) ED-ED Consultations: Emergency Medicine (EM) Physician to rural emergency advanced practice provider for general ED care consultations, (4) Specialist Consultations to ED: Specialty Physician to EM Physician specialty consultation for specific conditions, (5) Direct to consumer; and (6) International telehealth. Effectiveness studies addressed the following categories of care most frequently: (1) cardiovascular (e.g., acute myocardial infarction); (2) trauma; (3) ophthalmological; (4) neurological (e.g., stroke); (5) mental health; (6) pediatric; and (7) critical care.

There are three types of effectiveness studies performed to date: (1) Randomized controlled trials (RCT); (2) Non-controlled comparative- effectiveness studies, including non-inferiority studies; and (3) Non-controlled, non-comparative, single intervention reported outcomes. Beyond several RCT evaluations of telestroke, effectiveness evaluations are primarily descriptive without controls lacking rigorous randomized evaluation. Finally, effectiveness outcomes studied to date have been primarily operational (e.g., time-to-events such as primary percutaneous coronary intervention), disposition (AMA, transfer rates), clinical decision-making (e.g., diagnostic accuracy), and ED return visits. A paucity of studies evaluated clinical outcomes of telehealth models.

Patient-Centeredness in Emergency Telehealth

Three broad categories of telehealth studies in which patient- centeredness was studied: (1) prehospital; (2) patient experience with direct-to-patient telehealth visits; and (3) studies evaluating the provision of ancillary care services through telehealth (e.g. social work).

Use of telehealth in prehospital settings allows EMS to transport patients to different levels of care, potentially to a non-ED healthcare facility. The studies related to prehospital and direct-to-patient telehealth measured absolute reduction in transport to the ED and triage to appropriate level of care needed as surrogates for patient-centeredness. While few measures of patient-centeredness exist (e.g., Consumer Assessment of Healthcare Providers and Systems [CAHPS^®]), even fewer were validated despite the number of measurements under consideration: patient confidence, comprehension, and adherence to recommendations after telehealth encounters.

Studies of ancillary services (social work and case management [CM]) have examined the delivery of discharge instruction teaching and patient teaching. While several studies included CM as the primary reason for the telehealth visit, no studies described the implementation of these ancillary services into an unscheduled telehealth visit for a primarily medical concern where ancillary services were subsequently needed. Outcomes for CM telehealth visits included rates of adherence to being present for the telehealth visit and adherence to the services provided (e.g., wound care).

Safety in Emergency Telehealth

Assessing quality includes evaluation of the protection from errors and harm. One framework of safety in EM proposes the following dimensions: (1) patient harm; (2) application of appropriate interventions; (3) error identification and correction; and (4) ED safety culture.¹⁸⁸ In 2019, AHRQ released a systematic review to identify and summarize the available evidence about telehealth consultation.¹⁸⁹ Consultation referred to the sharing of information or collaboration between providers regardless of the patient’s involvement.¹⁸⁹ They asked, “Do telehealth consultations result in harms, adverse events, or negative unintended consequences?”¹⁸⁹ Results from this systematic review along with the proposed safety framework were used to organize our approach to safety of emergency telehealth, namely product safety studies and safety studies of emergency telehealth.

Product safety studies looked at products that patients or health care workers can use remotely to interface with health care providers and are relevant for acute events that may result in an ED visit. Smart phones with the addition of software can be used for physical exam, information communication (e.g., ECGs), performance feedback (e.g., CPR), and consultation (e.g., orthopedic injuries).^190–199 While promising, these studies lacked follow-up to demonstrate safety with wider implementation.^200–204 Several studies evaluated the quality of patient-reported data (e.g., blood pressure) and concluded that these were reliable means to collect such data.^205,206 Studies showed that tele-dermatology consultation directed by health care providers (e.g., a primary care provider requests a tele-dermatology consult for their patient) is safe, with errors toward conservative treatment.⁹⁴ Direct-to-patient tele-dermatology may not be safe as there is currently little oversight on direct-to-consumer telemedicine. One study reviewed five different dermatology apps and found a large variation in the quality in history taking, medication use, allergies. None of these dermatology telehealth apps gave feedback on picture quality.¹⁹⁵ These studies in tele-dermatology may inform how telehealth in EM is accessed and used, specifically, if it is safe for patients to choose tele-emergency care or if it is safer for primary care providers to refer patients to tele-emergency care. Also noted by these tele-dermatology studies is the variability of clinical oversight in the different programs, which also can inform best practices in emergency telehealth program oversight.

Safety studies of emergency telehealth were categorized as: pharmacy, non-stroke EM consultation, and stroke care. Pharmacy tele-support by real time to ED providers and medication monitoring after discharge both show reduction in errors and harm.^207,208 Studies using EM providers for tele-triage in the ED and prehospital setting showed changes to length of stay and length of ambulance service, lacked significant reporting of patient-centered outcomes, but did not find evidence of harm.^207,209 The AHRQ review found 19 studies on ED specialty consultation, none reporting harms.¹⁸⁹ They found 22 EMS and urgent care studies, one of which reported data that could be interpreted as harm, but was not defined as such by the authors.¹⁸⁹ Telestroke was the most common EM-related telehealth application both in the AHRQ and our review. The AHRQ review found 29 studies (two RCTs), one of which only had four patients. Compared with hub hospitals managing stroke patients, spoke hospitals that used telehealth found no difference in the emergency care of acute ischemic stroke patients with telehealth (compared with hub hospitals).²¹⁰ Eleven studies reviewed by AHRQ reported harm outcomes, specifically intracranial hemorrhage (ICH), which was not different between groups. At least one study stated specifically in its limitations that it was not powered to detect harm.^131,189

The final three questions (Figure 5) represent a reduction of the much larger group of questions by representing the three quality and safety domains (effectiveness, patient-centeredness, and safety) by the more collective “quality and safety”. Individual questions tended to focus on individual quality and safety domains thereby reducing potential redundancy. Further, these questions were refined to allow for generalizability depending on the situation or population. Other research question topics of interest not included on the final list involved the quality and safety of specific populations and conditions of interest, performance of virtual physical exams, necessary study designs, and the impact of emergency telehealth on patient and provider communications. The final three questions involve patient outcomes, comparison of in-person and virtual care, and care transitions and the role emergency telehealth plays in all three.

• How can telehealth be used to augment safe transitions of care?

• In what situations should the quality and safety of telehealth be compared to in-person care and in what situations should it be compared to “no care”?

• In which clinical conditions, populations, and settings does emergency telehealth improve patient and operational outcomes?

Use of Telehealth in Emergency Medicine Research

Telehealth and related digital health tools (e.g., mHealth, eHealth) are advancing patient care with respect to quality, access, cost and a wide variety of related metrics. It may also offer an opportunity to improve research, including clinical trials, conducted in the emergency medicine setting – potentially improving the quality of the research and increasing access to research participation for individuals or groups of individuals not reached by current methods. The consensus conference narrowed the scope of important research questions to two broad areas: the use of telehealth for EM research facilitation and special considerations for the use of telehealth in research.

Use of Telehealth for EM Research Facilitation

While the literature in this area is growing, there is still an opportunity for additional research to ensure efficacy in utilizing telehealth and technology to facilitate emergency medicine research. Potential research topics include but are not limited to: assessing the receptiveness of patients seen by telehealth only to participate in research studies; developing strategies to increase receptiveness; determining if the use of telehealth for recruitment or obtaining informed consent affects representativeness of participants; devising effective strategies to improve representation; and determining if tele-recruiting and consent can protect and reduce attrition and support staff.

Studies in other medical disciplines, including those that likely collaborate with emergency medicine (e.g., neurology trials for stroke treatment) have found telehealth to be valuable in facilitating or enhancing recruitment, informed consent, and data collection. Lane and colleagues²¹¹ conducted a literature review of studies examining online recruitment and retention for mHealth studies. Online recruitment was the most frequently used modality and studies focused primarily on smoking cessation and mental health interventions. Among the 12 studies included, none focused on emergency medicine.

While the online recruitment methods were considered promising, areas for additional research were identified, including participant retention and generalizability to other settings.²¹¹ Mastellos et al. found that eHealth-supported recruitment was successful in the primary care research setting.²¹² Telehealth networks have also been identified as being helpful for subject recruitment in areas such as stroke^213,214, multiple sclerosis²¹⁵, and sexually transmitted infections²¹⁶.

Abujarad et al. developed a mHealth tool to enhance informed consent for clinical procedures.²¹⁷ The toolbox they created allowed for informed consent process to include tablets (e.g., iPads) utilizing virtual coaching with text-to-speech automated translation and interactive multimedia elements (e.g., graphics, video clips, animations, presentations). They concluded that mHealth can be effective in delivering health communications including informed consent. Importantly, the authors noted that ensuring patient comprehension should be central to any informed consent approach.²¹⁷

To our knowledge, only one study within EM has examined the use of telehealth to facilitate informed consent. Bobb et al tested whether telehealth‐enabled research informed consent provided non-inferior comprehension compared with a standard informed consent process in the setting of an academic emergency department.²¹⁸ They found that comprehension of research informed consent via telehealth was not inferior to face-to-face informed consent with respect to subjective understanding of informed consent or parent trial study accrual rates.²¹⁸ More work is needed in this area to determine the usefulness of using telehealth informed consent for research across different types of institutions.

Varma et al. explored telehealth-enhanced monitoring (automatic remote monitoring) vs. in-person follow-up for patients receiving an implantable cardioverter defibrillator (ICD).²¹⁹ They found that the telehealth-enhanced monitoring improved retention and adherence to scheduled follow-up compared with in-person follow-up visits.

In an emergency medicine-specific study, Varner et al explored text message reminders for study follow-up as a way to reduce attrition.²²⁰ They found that in their cohort of subjects consenting to participate in a randomized trial, text message reminders of upcoming telephone follow-up interviews were effective in reducing attrition.²²⁰

Taken together, these studies begin to suggest utility in using telehealth to enhance clinical research. Rigorous evaluation within emergency medicine, across types of centers (e.g., urban, rural, academic, community, trauma centers) and across the breadth of chief complaints and diagnoses treated in the emergency department is needed.

Special Considerations for the Use of Telehealth in Research

Although technology, access, and financial considerations are common barriers to telemedicine services, in general, additional barriers to patients engaging in research may be more related to reaching and engaging special populations. For example, there is a need for developing strategies for remote consenting of surrogates for pediatric populations, integrating American Sign Language into video-consent, and determining whether perceptions of internet and technology access are accurate in developed and developing countries?

Practice guidelines for telehealth (e.g., those developed by the American Telemedicine Association) often note special populations and the need for awareness regarding inclusion and access with respect to clinical care. This is an area that might also benefit from systematic investigation with respect to telehealth and emergency medicine research. For example, Greenwald et al. found that older adults with low acuity emergency department visits were comfortable receiving and satisfied with receiving care via telehealth.²²¹ The Telemental Health Guidelines for Children and Adolescents provides a useful framework for evaluating and providing guidance for the use of telehealth in emergency medicine research that is inclusive.²²²

The final wording of the two research questions for this breakout group are:

• How can/should telehealth be used for research facilitation, including recruitment, informed consent, reducing attrition, and data collection, for EM research?

• Which individuals or populations require special considerations as the role of telehealth in EM research is expanded and what are the key barriers for engaging these patients in telehealth-facilitated research studies?

Workforce

The use of telehealth in emergency care may have a profound effect on the future practice and makeup of the workforce. Four main workforce-related topics were chosen for this breakout group: (1) provider psychological health; (2) costs; (3) future staffing; and (4) rural versus urban workforce.

Provider Psychological Health

Provider mental health and burnout are significant issues in EM and with complex causes. Some of the EM-specific suspected causes of burnout are work hours, mix of shifts, Electronic Health Record (EHR) use, litigation stress, negative interactions with colleagues and patients, staffing needs, and sleep loss.^223–225 Whether the addition of telehealth to clinical practice will improve or worsen these is unknown. Some studies predict telehealth will improve burnout, but there were no studies found specific to EM physicians.¹³² A study by Romig et al. mentioned that a telehealth backup process improved burnout in critical care nurses in a self-reported survey.²²⁶ Burnout studies in EM do not mention telehealth or demonstrate how telehealth could alleviate burnout by increasing the variety of shifts, increasing the access to specialists, and improving the transfer process. Alternatively, there are no studies of burnout in EM about how telehealth can increase burnout by increasing the complexity of shifts, stretching the scope of the practitioner, et cetera. How aspects of telehealth will impact burnout is yet unknown.

Costs

We evaluated how telehealth’s costs to the individual, system and community will affect the future EM workforce. In general, telehealth is advertised as a cost saving measure, however barriers to reimbursement, implementation and staffing have significant investment costs and limit implementation.^{94,95,227–233} It is yet to be seen how the changes in telehealth reimbursement and regulations^234,235 impact the use of telehealth within EM. A RAND study noted that in a particular patient population studied, direct to consumer (DTC) telehealth increased utilization resulting in likely future increased costs to the healthcare system.²³⁶ Studies since have tried to determine whether longer outlook projections from established programs continue to demonstrate increased costs. Different EM staffing models, including the use of Advance Practice Providers (APPs), may require fewer physicians and affect costs (positively and negatively).^77,169,171 Using telehealth to streamline processes, by bringing patients to the appropriate level of care or improving throughput metrics could also impact costs.^{95,150,163,169,207,233,237–239} While studies in remote consults show cost savings due to lack of transfer or earlier interventions, its use is not standardized.^163,240

Future Staffing

Current staffing of telehealth covers a wide spectrum and includes physicians, APPs, nurses, emergency medical technicians (EMTs), and clinical telepresenters (i.e., those aiding with telehealth encounters).^241–245 Each group may have a different role (e.g., EMTs may call for prehospital telehealth, while telepresenters aid EM physicians with specialty consults). The pre-hospital use of Emergency Triage, Treat and Transport (ET3) intends to decrease ED volume by encouraging non-transport when appropriate; although, it is unclear if ED volumes will decrease.

Most of the literature on telehealth staffing models include descriptive studies of direct to consumer programs staffed by either physicians or APPs, or EMS oversight by EM physicians.^{117,246–250} Staffing descriptions in the literature typically were specific to the program being described as there are many variations of telehealth in emergency medicine.^{152,245,251,252} While most studies described how current emergency care processes were supported, the literature did not include future states of ED shifts with the addition of virtual care, ET3 models, and Emergency Medical Services (EMS).^253,254 A gap exists in understanding the future role that EM physicians will provide to support prehospital and hospital-to-hospital critical care transport, specifically using real-time videoconferencing for patient management decisions during transport, including decisions to direct paramedic care in the absence of protocols.^{164,255–260}

Also missing in the literature is the role of EM physicians in telehealth in relation to other types of practitioners.^{249,253,256,261} Depending on how telehealth will exist in EM practice and the staffing models chosen, the future EM workforce composition will likely change.

Rural versus Urban Workforce

Rural and urban EM practitioners have different needs and this likely translates to their telehealth use. For example, most studies in rural areas noted telehealth use for remote consults for access to specialty evaluation and transfer.²⁵⁵ Gaps exist in how rural and urban EM differ and it is important to further understand this topic, including the future of regional care access and effects of population migration.^{161,262–268} As more rural centers are closed and centralized metropolitan areas stretch their healthcare resources, there may be a greater need for trained EM telehealth providers.^{74 29,262,265,266,269}

During the consensus conference days, discussion included recommendations to broaden initial research questions to include a wider variety of health care providers in EM, including EMS, APPs, telepresenters, administrative teams, family members and care partners. While another breakout group focused on education, the use of family and care partners in telehealth may require new instruction and assessments telehealth and healthcare in the future.²⁷⁰

The final research questions for this breakout group are:

• How effective can TH be as a solution for hospitals, particularly rural and critical access, that are unable to staff with board-certified EM physicians at the bedside?

• What types of training will be required for current practicing providers?

• What kinds of staffing will be best suited for emergency TH in different settings (e.g., APP versus rural physician, rural versus urban, etc.)?

• What kinds of staffing and systems are required to ensure provider efficiency in emergency TH?

The gaps in evidence are broad, from defining appropriate educational outcomes to determining when to use a comparison of telehealth to in-person care versus no care. Many of these questions are focused on EM; however, they are applicable to telehealth in general. The research stimulated by the agenda we laid out will inform future policy and regulations. This agenda will also clarify a new value proposition for EM, that of providing acute, unscheduled care wherever a patient is located. As telehealth in EM expands in scope and volume, the answers to the research questions on this agenda are critical for effectively educating EM telehealth providers, maintaining or increasing access to healthcare through telehealth, ensuring safe and quality care for patients using telehealth, implementing telehealth thoughtfully into the research process, and appropriately managing the EM workforce with the integration of telehealth into EM practice.

Limitations

The results of this consensus conference have limitations. Initially, we planned for an in-person conference with opportunities for in-person discussion. Compared to a conference conducted in person, areas that may have been affected by the virtual format include the following: a less rich discussion, lack of familiarity of the virtual voting methods, and less opportunity for networking. Attendance at the conference potentially would have been higher at the in-person conference as it is traditionally scheduled as a pre-conference day to the SAEM Annual Meeting, which typically brings in additional EM providers who are already attending the annual meeting. With the need to reschedule due to the COVID pandemic, the planning committee chose to host a virtual meeting spanning two afternoons in the same week. This may have benefited the consensus process as it allowed two separate voting sessions after breakout group discussions; however, it may have limited the number of attendees.

The iterative cycle of remote voting and discussion may have limited our findings. For our first round of voting, we intentionally scheduled the survey administration and deadlines so there was time for the breakout groups to assimilate the survey results. However, we only captured those pre-registered participants who had registered by the time the survey was sent. It is clear from the number of preregistered participants and final list of attendees that we were not capturing all the attendees in the pre-conference survey. We attempted to mitigate this by maintaining all research questions from the list until after Round 2 of voting (after the live breakout group discussions on conference Day 1). Ad hoc feedback from both the conference planning committee and the participants confirmed that more time was needed for discussion of the research questions. This critique was strongly stated by many and the planning committee recommends future consensus conferences build in adequate time for discussion. While the planning committee intentionally scheduled two discussion sessions per topic, each separated by a full day for reflection and further asynchronous feedback and discussion, it is possible that some of the results would have been different if more time was allotted for the discussion.

The topic of telehealth in EM is broad and it is possible that there are domains that may not have been captured. The demographics of the attendees may have biased the final research agenda as the attendees were largely from urban teaching hospitals who are already use telehealth. It is possible that certain research questions were missed or not retained through the voting process due to the skewed attendee demographics. We solicited expert feedback throughout the process to ensure completeness; however, given the wide range of possible topics, sections may have been inadvertently left out, such as implementation science. While the consensus process covered a wide-ranging domain and all topics pertaining to telehealth were open for discussion, there are also key characteristics of telehealth in EM that did not ascend to the final priority research questions. Examples of these missing topics are reimbursement and privacy, both critical to the provision and receipt of telehealth. Reimbursement plays a significant role in telehealth, as both a barrier and driver of telehealth use. While the topic of reimbursement did not make it into the final questions, the evidence and lessons learned from answering the proposed research questions inform the arguments for or against reimbursement models or reimbursement options.

Conclusions

The use of telehealth is increasing due to the COVID pandemic, and Emergency Medicine has adopted many new telehealth modalities. The 2020 SAEM Consensus Conference on Telehealth in Emergency Medicine convened experts in Emergency Medicine and telehealth to generate research priorities. The 24 research areas identified can inform future research funding opportunities as well as healthcare policy and regulations. Future studies are needed to better understand how Emergency Medicine will shape telehealth and how telehealth will impact Emergency Medicine.

Appendix.

Round 1 (Preconference) Survey

Round 1 (Preconference) Voting Results

Round 3 (Final) Voting Results

Round 1 (Preconference) Survey Questions

For the question stem: As an area of research, (question/statement) is: [Likert scale from “not important at all”, “of little importance”, “of average importance”, “very important”, “absolutely essential”]

TH=Telehealth

EM=Emergency Medicine

APP=Advanced Practice Provider

Educational Needs and Outcomes

1. What are the core competencies in TH that are common to all providers?

2. What are the core competencies in TH for EM practitioners?

3. In patient-provider TH encounters, what is a standard approach to the video-based physical exam?

4. How does the video-based physical exam vary with modality and setting?

5. What modalities of training are most effective in TH education for emergency practitioners?

6. How can current EM curricula integrate TH education to allow for mastery of TH skills?

7. At what training levels should TH education be integrated for emergency practitioners?

8. How do we create technical knowledge training programs for emergency practitioners?

9. How do we teach technical TH knowledge to enable emergency practitioners to navigate technical difficulties during live encounters?

10. How do we train emergency practitioners in TH etiquette to ensure successful patient-to-provider and provider-to-provider encounters?

11. How do we train EM teams to provide care via TH?

12. Does training for EM providers in TH skills impact clinical outcomes or quality of care?

13. Does training for EM providers in TH skills impact patient satisfaction?

14. Should there be a minimum standard training requirement for EM practitioners to allow for independent practice of TH?

15. What are the best practices for tele-education and real-time tele-mentoring in EM?

16. How can TH platforms allow for more effective supervision of bedside learners?

Healthcare Access

1. What are the appropriate outcomes to evaluate the impact on patient-level outcomes?

2. What are the appropriate outcomes to evaluate the impact on population-level outcomes?

3. What is the impact of TH access on patient-centered outcomes for critically ill patients (e.g., intubation, ICU length of stay, morbidity, and mortality)?

4. What is the cost-effectiveness of TH in emergency care?

5. What are barriers and facilitators to more widespread implementation of TH in EDs?

6. What are effective strategies for incorporating medical specialists within emergency TH to improve access for patients?

7. High quality efficacy data is needed for prehospital applications for heart attack, trauma, pediatric applications. Which patient- and population-level outcomes should be prioritized?

8. How does TH affect outcomes when used in austere settings such as disaster response?

9. Does out-of-hospital TH improve the timeliness of interventions for patients with traumatic injury?

10. What is the best way to approach telementoring and communication? How can applications of educational and communication science be applied to TH encounters and interactions?

11. What role does Project ECHO play with respect to access and education in emergency care delivery?

12. What are the appropriate outcomes to study in research of TH use in austere settings?

13. What is the economic impact of programs in out-of-hospital settings?

14. What is the technical and operational effectiveness of TH programs for improving access to medical care in disaster response?

15. What implementation lessons learned are there from existing TH disaster response programs?

16. What are potential barriers to use of TH for disaster response?

17. What are the clinical outcome measures to capture for reduced patient transfers to EDs?

18. How should we measure appropriateness of TH utilization for diverting and triaging patients with urgent and emergent medical issues? Is there a parallel to the “ambulatory care sensitive ED visits” in this domain?

19. What is the appropriate way to differentiate between increased access to care (generally good) and excessive utilization (increased cost with little benefit)?

20. How does emergency TH access vary by patient/population characteristics?

21. Does TH improve access to care for vulnerable or remote populations by facilitating specialist follow-up post-ED discharge when travel times for in-person specialist evaluation would have otherwise been prohibitively long?

22. What are potentially unintended consequences of enhancing TH access for vulnerable patients, e.g., if access is more challenging related to interpreters, digital literacy, or technology ownership?

23. For people with inadequate broadband access, in what situations is audio-only TH a viable alternative (versus no care at all)?

24. Were there lessons learned from the COVID-19 pandemic to inform our understanding of situations in which audio-only may be a reasonable alternative for individuals with inadequate broadband access?

25. Among vulnerable populations, are disparities in emergency care delivery exacerbated by TH, and if so how? At the same time, are there ways in which TH ameliorates disparities in emergency care for vulnerable populations, and if so, how?

26. What is the impact of emergency TH access on disparities in care for outcomes related to stroke, heart attack, and trauma?

27. What is the appropriate strategy to educate the public, to ensure patients understand risks and benefits, and that patients have the tools required to ensure a valuable/high-quality encounter?

28. What are barriers/facilitators to improving access and quality of care for underserved populations? (i.e. funding for infrastructure vs billing/insurance issues vs patient education strategies)

Quality and Safety

1. How should effectiveness of emergency TH be defined? And should it differ by population (e.g., pediatrics, trauma, prehospital, disaster)?

2. How should the effectiveness of emergency TH be measured and what is their relationship with meaningful patient outcomes from a patient, hospital, and societal perspective?

3. What is the fidelity of a virtual physical exam and patient-reported vital signs with in-person exams?

4. What are the appropriate metrics to evaluate the quality of a TH exam?

5. In which clinical conditions and settings does emergency TH improve patient and operational outcomes?

6. Which study designs are necessary (and/or sufficient) to evaluate the effectiveness of emergency TH?

7. In what situations should the effectiveness of TH be compared to in-person care and in what situations should it be compared to “no care”? How can the COVID-19 pandemic experience help answer this question?

8. What are valid measures of patient (family and caregiver) experience in emergency TH and what is their association with patient outcomes?

9. Which measures are representative of patient-centeredness in emergency TH?

10. Are traditional patient-centered measures valid in emergency TH encounters?

11. Is it feasible to evaluate the patient (family and caregiver) experience in an emergency TH setting?

12. Should quality of TH care delivery be assessed differently based on the population being served (vulnerable or underserved, pediatric, geriatric, etc.)? And if so, how?

13. With the exponential increase in TH visits during the pandemic, what new data are available and relevant in terms of patient centeredness and emergency TH?

14. How do traditional clinical measures of safety translate to the emergency TH setting?

15. Are there unique safety measures that should be developed and used for emergency TH?

16. Is emergency TH safe compared with in-person emergency care? Are there times in which emergency TH is safer?

17. How should safety evaluation be implemented in the ongoing practice of emergency TH?

18. How should a safety culture around TH be fostered?

19. What are the best practice future applications of novel pandemic-related models of in-room to out-of-room emergency TH models designed to limit exposure and preserving PPE?

20. How can TH be used to augment safe transitions of care?

Research Facilitation

1. Can TH be used for research facilitation, including recruitment, informed consent, and data collection, for EM research?

2. Can TH be used as part of a study intervention in EM research?

3. Can TH be used to reduce attrition in EM research?

4. Are there individuals or populations that require special consideration if the role of TH in EM research is expanded: pediatrics, geriatrics, rural areas, low-income populations, populations with limited healthcare access (those with the potential to have access and technology gaps), limited English proficiency populations, individuals with disabilities?

Workforce

1. What factors in provider burnout (backup, additional shifts, time in career, urgent care component, etc.) can TH possibly alleviate?

2. How do we measure whether TH alleviates burnout symptoms?

3. What workforce provisions should be made for non-English speaking patients?

4. What does the use of TH on recipient hospitals (spoke) affect burnout on those physicians?

5. Is there an effect on physician burnout due to being at a distance in a TH encounter?

6. How will TH impact the clinician patient interaction/relationship, potentially affecting burnout? Can we design TH encounters to improve burnout?

7. What types of TH will be practiced in the future ED?

8. What types of TH will require EM physician oversight or direct involvement?

9. What types of oversight of TH will future EM physicians be required to know, and will it affect physician retention?

10. Is there data on APP v physician-staffed TH and patient perception?

11. What factors and information exist to inform recruitment and retention of TH providers in EM?

12. What types of factors will determine who will be practicing TH in EM in the future?

13. How will expanded role for APPs change the future workforce?

14. What relationship, if any, will there be between the “TH provider” and the patient’s physician? How do we ensure there is standardization in staffing, laws, and technology?

15. What types of training will be required for current practicing providers?

16. What effect does reimbursement have on utilizing EM TH?

17. Do current cost savings or studies inspire more or less investment? What future staffing costs will be needed to use TH in EM?

18. What is the anticipated near-term and long-term growth of the TH resource? How do remote consults, pre hospital consults, use of remote triage affect costs in EM?

19. Can TH triage patients to a lower level of care, decreased ED admissions or readmissions, therefore saving costs to the system?

20. What are the effects on costs of reduced transfers and hospitals without board certified physicians in rural areas?

21. What changes to the rural and urban healthcare will affect the future EM workforce? Where will most TH take place in rural vs. urban EDs?

22. What kind of staffing will be required for TH in rural vs. urban EDs?

23. What changes to rural vs. urban healthcare can be served best by EM TH? What noted differences are there between rural v urban EM TH?

24. How will the current use of TH affect the staffing of rural EDs and hospitals?

25. Will there be more reliance on patient’s family members and informal caregivers in rural TH?

26. To what degree can TH be a solution for hospitals, particularly rural and critical access, that are unable to staff with board-certified EM physicians at the bedside?

Demographics

1. Name: (free text)

2. Institution: (free text)

3. Primary clinical setting: (Rural, Suburban, Urban, N/A)

4. Teaching Hospital: (Y/N/n/a)

5. Do you practice clinical EM? (Y/N)

6. What is your experience in telehealth?

   1. I use/have used telehealth in my practice
   2. There is no telehealth in my practice but there are plans to create a program
   3. There is no telehealth practice and no plan to create a program
7. Do you have a telehealth training program? (Check all that apply):

   1. Student
   2. Resident
   3. Fellow
   4. Other

Round 1 Voting Results (see separate attachment)

Round 3 Voting Results (see separate attachment)

Consensus Conference Participants

David Amponsah	Emily Hayden, Chair	Jacob Ramseyer
Martina Anto-Ocrah	Daniel Herrmann	Junaid RazzakA
Carl Berdahl	Nancy Holland	Karen Rheuban
Tehnaz BoyleA	Judd Hollander	Natcha Rummaneethorn
Elizabeth BurnerR	Peter HouR	Margaret Sande
Hope Burnham	Darryl Jefferson	Michael Sardone
Kristie Busch	John Joseph	Dana SchinasiE
Christopher Carpenter	Aditi Joshi W	Elizabeth Schoenfeld
Shruti Chandra, Co-chair	David Kessler	Evan Schwarz
Wendy Chang	Ji Won KimQ	Sam ShenA
Sherita ChapmanA	Joshua Kim	Scott Shipman
Bradley Chappell	Andrew KirkpatrickA	Neal SikkaE
Dickson Cheung	Andrea Kitts*	Aggie Sikora
Jaron Christianson	Elizabeth Krupinski R	David Sklar
Sunday Clark R	Seth Krupp	Nina SolenskiE
Alexis Cole	Maria LameE	Tommy Stewart*
Christopher Davis, Co-chair	Kathleen LiA	Mindy Stimell-Rauch
Persis Dhas	Ana Maria Lopez	Elizabeth Suminski
Linley Dunn	Jennifer MacCready	Richard Summers
Robert Eisenstein	James Marcin	Thomas Terndrup
David Ernst	Jacob Mathew	K Noelle Tune
Barney Eskin	Christopher McDowell	Phyllis Vallee
Morgan Eutermoser	Colleen McQuownQ	Renoj Varughese
John Evanko	Lawrence MelnikerW	Marcia Ward
Pamela Flores-Sanchez	Nicholas Mohr	Michael Ward Q
Barbara Forney	Kevin Munjal	Tim Weir
Sagar GalwankarR	Neel Naik E	David Whitehead
Elizabeth Goldberg	Erica Olsen	Kathy WibberlyW
Jason GoldwaterQ	Christina OlsonQ	Bella Wong*
Peter Greenwald, Co-chair	Chris PalmerA	Thomas Yang
David HaoA	Dimitrios PapanagnouE	Kori Zachrison A
Stephen Hartsell	Divya Parikh

*Patient representatives

^EEducational Needs and Outcomes Breakout Group member

^AHealthcare Access Breakout Group member

^QQuality and Safety Breakout Group member

^RResearch Facilitation Breakout Group member

^WWorkforce Breakout Group member

Italicized names denote breakout group leaders

Breakout group members who were unable to attend the conference: Tina Gustin, Kendall Ho, Milania Trounce (Educational Needs and Outcomes); Sofian Berrouiguet and Jessica McKee (Healthcare Access); Todd Crocco, Michelle Lin, Oren Mechanic, Shashank Ravi, and Kimberly Rockwell (Quality and Safety); Carlos Camargo, AnnaMarie Chang, Vivek Chauhan, Brock Daniels, Scott Dresden, Jason Lowe, and James Victor Quinn (Research Facilitation); and Alex Forenko, Gordon Ngai, Karen O’Mara, Ryan Ribeira, and Chris Russi (Workforce).