Emergency medical services education research priorities during COVID‐19: A modified Delphi study

Abstract

Objective

Our objective was to identify research priorities to understand the impact of COVID‐19 on initial emergency medical services (EMS) education.

Methods

We used a modified Delphi method with an expert panel (n = 15) of EMS stakeholders to develop consensus on the research priorities that are most important and feasible to understand the impact of the COVID‐19 pandemic on initial EMS education. Data were collected from August 2020 to February 2021 over 5 rounds (3 electronic surveys and 2 live virtual meetings). In Round 1, participants submitted research priorities over 9 specific areas. Responses were thematically analyzed to develop a list of research priorities reviewed in Round 2. In Round 3, participants rated the priorities by importance and feasibility, with a weighted score (2/3*importance+1/3*feasibility) used for preliminary prioritization. In Round 4, participants ranked the priorities. In Round 5, participants provided their agreement or disagreement with the group's consensus of the top 8 research priorities.

Results

During Rounds 1 and 2, 135 ideas were submitted by the panel, leading to a preliminary list of 27 research priorities after thematic analysis. The top 4 research priorities identified by the expert panel were prehospital internship access, impact of lack of field and clinical experience, student health and safety, and EMS education program availability and accessibility. Consensus was reached with 10/11 (91%) participants in Round 5 agreeing.

Conclusions

The identified research priorities are an important first step to begin evaluating the EMS educational infrastructure, processes, and outcomes that were affected or threatened through the pandemic.

1. INTRODUCTION

1.1. Background

The COVID‐19 pandemic has affected the entire medical community,¹, ² and emergency medical services (EMS) are no exception. The need to understand the impact from the COVID‐19 pandemic on the EMS education pipeline was recognized by national organizations and stakeholders in the EMS community.³ In order to understand and address the impact, a national task force steering committee was formed by gathering representatives from 13 organizations involved in EMS education during the COVID‐19 pandemic.

1.2. Importance

The EMS workforce, and care they provide to ill and injured patients outside of the emergency department, is an integral part of the overall US emergency care system. Impacts from COVID‐19 have affected EMS service call volume,⁴, ⁵, ⁶, ⁷ increased EMS professional fatigue, and worsened well‐being,⁸, ⁹ and slowed the education pipeline.³ Specifically, for the EMS education pipeline, it is believed that the majority of EMS education programs experienced at least some form of temporary closure, modifications to delivery of content and program requirements, and limited or loss of access to simulated education as well as live field and clinical opportunities.³, ¹⁰, ¹¹, ¹², ¹³ More so, ability to test, certify, and recertify were delayed or extended with some states modifying the processes for EMS licensure.³, ¹³ Workforce shortages and poorly trained personnel can have downstream effects for emergency physicians.

1.3. Goals of this investigation

Quantifying the impact of the COVID‐19 pandemic on EMS education programs and developing mitigation strategies have been recognized as a priority, despite known challenges, such as resource sharing and standardized mechanisms for collecting data related to EMS initial education.³, ¹⁰ Our objective was to identify research priorities to understand how the COVID‐19 pandemic affected initial EMS education. We focused on initial EMS education for certification, rather than continued education after certification has been obtained, because the EMS education pipeline is directly related to workforce stability.

2. METHODS

2.1. Study design, setting, and participants

We used a modified Delphi method with a panel of national EMS stakeholder organizations (Table 1) to develop consensus on the research priorities that were most important and feasible to understand the impact of the COVID‐19 pandemic on initial EMS education. The Delphi methodology is a structured approach to consensus building with an expert panel that involves iterative rounds of data collection and feedback, typically in a face‐to‐face format.¹⁴, ¹⁵, ¹⁶ As an extension of this approach, the modified Delphi methodology leverages asynchronous electronic communication.¹⁵ The Delphi methodology has been widely used to develop research priorities in EMS and other healthcare settings.¹⁷, ¹⁸, ¹⁹, ²⁰

TABLE 1.

Stakeholder organizations involved in the task force

American Ambulance Association (AAA)
American College of Emergency Physicians (ACEP)
Commission on Accreditation for Prehospital Continuing Education (CAPCE)
Committee on Accreditation of Educational Programs for the Emergency Medical Services (CoAEMSP)
International Association of Fire Chiefs (IAFC)
International Association of Firefighters (IAFF)
National Association of Emergency Medical Technicians (NAEMT)
National Association of EMS Educators (NAEMSE)
National Association of EMS Physicians (NAEMSP)
National Association of State EMS Officials (NASEMSO)
National Highway Traffic Safety Administration (NHTSA), Office of EMS
National Registry of Emergency Medical Technicians (NREMT)a
National Volunteer Fire Council (NVFC)

^a Had no participant on the expert panel due to being part of the research team.

Participants for the expert panel were identified by national stakeholder organizations that formed the task force steering committee (Table 1). A total of 12 organizations were represented. The organizations included in the task force steering committee selected their own representatives. Ten organizations had a single representative on the task force, and 2 (National Association of State EMS Officials and National Highway Traffic Safety Administration) had multiple representatives on the task force and thus included as participants in the expert panel. The representative from the National Registry of Emergency Medical Technicians was a member of the national task force but did not participate in the expert panel due to being part of the research team. Participation and completion of the project were voluntary, and participants were asked to respond as individuals rather than on behalf of the organizations they represented. The American Institutes for Research institutional review board approved this project and waived documentation of consent.

2.2. Data collection and analysis

Five rounds of data collection occurred from August 2020 to February 2021 using 3 asynchronous electronic surveys and 2 live webinar meetings. Throughout data collection, identities of the participants and research team were known to each other; however, data were collected and reported anonymously.

2.2.1. Round 1

The first round of data collection consisted of an electronic survey via the Alchemer survey platform (Boulder, CO) to identify specific research topics of importance to understand the impact of the pandemic on initial EMS education. Participants were provided with the following prompt: “For each of the following areas, what are the top three priorities to study to assess the impact of COVID‐19 on initial EMS education in the US?” Up to 3 priorities could be entered for consideration within each of 9 areas of interest (ie, participants had the opportunity to provide up to 27 priorities each): general program sustainability and operations, achieving entry‐level competencies, delivering didactic instruction and lecture, use of simulation and lab, field experience, clinical experience, student perspective and experience, faculty perspective and experience, and other/miscellaneous. Participants also provided a short rationale or explanation for each topic.

Thematic analysis of the responses was used to develop a preliminary list of research priorities identified by the participants to continue to the next rounds. Five members of the research team (RC, WL, JP, KM, and PR) independently reviewed research priorities (2 reviewers per area of interest), with a sixth investigator (AP) consulted to resolve discrepancies. Rationales for the research priorities were then summarized using as much language as possible from participants’ responses.

2.2.2. Round 2

Round 2 was conducted via webinar in September 2020. The preliminary list of research priorities was presented to the panel with an opportunity for participants to disagree with any of priorities identified in Round 1, provide feedback on phrasing of the priorities or rationales, and suggest additional research priorities that were missed. We collected data anonymously during the meeting through the Poll Everywhere software (http://www.polleverywhere.com, San Francisco, CA), and the anonymous results were presented to participants in real time. Additional research priorities suggested by participants underwent thematic analysis as before.

The Bottom Line.

COVID‐19 affected emergency medical services (EMS) in many ways. Between August 2020 and February 2021, using a modified Delphi method with an expert panel of 15 EMS stakeholders, the top 4 EMS research priorities identified were as follows: prehospital internship access, impact of lack of field and clinical experience, student health and safety, and EMS education program availability and accessibility.

2.2.3. Round 3

In Round 3 conducted by electronic survey, participants were asked to rate each of the research priorities by importance and feasibility on 4‐point Likert‐type scales (1 = not at all important/feasible to 4 = very important/feasible). We defined importance as a priority that should be investigated within the next 6–18 months. To determine feasibility, we asked participants to consider availability or access to existing data sources, the need for collecting data, and the cost or funding required to conduct the work.

We calculated a mean importance and feasibility rating for each research priority. Using the calculated scores, we created an aggregated weighted importance score reflecting the combination of ratings for importance and feasibility, where:

$$\begin{array}{ccc}\hfill weightedimportancescore& =& \frac{2}{3}\ast meanimportance\hfill \\ & & +\frac{1}{3}\ast meanfeasibility\hfill \end{array}$$

We gave higher weight to importance ratings because some participants were more knowledgeable regarding importance of topics to EMS education rather than research feasibility. The top 12 priorities by weighted score were advanced to the Round 4 to determine consensus based on an a priori decision. The decision to use the top 12 priorities for final ranking was to enable participants to prioritize more effectively by limiting the cognitive burden.

2.2.4. Round 4

Round 4 was conducted via webinar in October 2020. We first presented a summary of the results from the previous rounds and the top 12 research priorities from Round 3. Participants were then asked to rank the research priorities in order from highest to lowest priority, considering both importance and feasibility in that ranking. A ranking score was determined by assigning points to each rank option, where a rank of 1 (most important) was equal to 12 points and a rank of 12 (least important) was equal 1 point.²¹, ²² The sum of points obtained was calculated. Based on an a priori decision, the top 8 research priorities were then used to create a final prioritized list, rank ordered by total points, and the percentage of participants ranking each in the top 4 was calculated.

2.2.5. Round 5

The final round was conducted by electronic survey in February 2021. Participants were asked if they agreed with the top 8 prioritized list of research priorities. If a participant did not agree with the prioritization as decided by the group, the reason for dissent was elicited. We defined majority consensus as at least 75% of participants agreeing with the prioritized list of the top 8 research priorities. Because of an initially low response rate, a final unplanned reminder in February 2021 to non‐responding participants was required.

3. RESULTS

3.1. Participants and response rates

A total of 15 participants were included from 12 national organizations (Table 1). Not all participants chose to respond to the electronic surveys; however, participants were involved in the live meetings where data were also collected and thus included. In Round 1, we received 11 responses to the electronic survey (response rate = 73%). We received 12 responses each for Rounds 3 and 4 (response rate = 80%). In the final round, we received 11 responses (response rate = 73%).

3.2. Rounds 1

During the first round, 117 research priorities were submitted by the panel, ranging from 3 to 29 priorities in each of the 9 areas. After initial thematic analysis, there were 49 distinct research priorities over the 9 areas, which were then reduced to 23 final research priorities owinge to overlap across the topic areas.

3.3. Round 2

During the live webinar, there was near universal agreement of the panel with the preliminary list of research priorities; only 1 participant reported being unsure about the list and no participant reported disagreement. An additional 18 potential research priorities were provided by the panel. After thematic analysis, this resulted in the addition of 4 research priorities to the preliminary list. The full list of research priorities identified by the panel, and their associated rationales, are included in Appendix 1.

3.4. Round 3

The 27 research priorities identified by the panel were then rated by importance and feasibility in Round 3. The mean importance rating ranged from 2.42 to 3.75, and the mean feasibility rating ranged from 2.33 to 3.45. The final weighted importance score ranged from 2.42 to 3.56. Table 2 shows the ranking of research priorities by weighted importance score.

TABLE 2.

Ranking of research priorities by weighted importance score

Research priority	Weighted importance scorea	Mean importance score	Mean feasibility score
Prehospital internship access	3.56	3.75	3.17
EMS education program availability and accessibility	3.50	3.58	3.33
Impact of lack of field and clinical experience	3.50	3.67	3.17
Student health and safety	3.47	3.58	3.25
Cognitive competency outcomes during COVID	3.42	3.50	3.25
Psychomotor competency outcomes during COVID	3.36	3.58	2.92
Hospital/ambulatory site access	3.36	3.50	3.08
Keeping EMS education accessible for all students	3.33	3.58	2.83
Simulation accessibility	3.32	3.25	3.45
Recruitment/enrollment	3.31	3.50	2.92
Program instruction changes because of COVID	3.25	3.33	3.08
Future of EMS education after the pandemic	3.25	3.42	2.91
Substitution of simulation for clinical/field contacts	3.22	3.50	2.67
Faculty health and safety	3.19	3.25	3.08
How is and how much simulation is being used	3.19	3.33	2.92
Affective competency outcomes during COVID	3.06	3.17	2.83
Medical director involvement	3.06	3.00	3.17
Alternatives to clinical rotations	2.94	3.08	2.67
Faculty stress and anxiety	2.94	2.92	3.00
Impact of program changes on future employment	2.89	3.17	2.33
Pandemic‐specific topics of education	2.87	2.64	3.33
Student stress and anxiety	2.86	2.91	2.75
Faculty availability, structure and size	2.85	3.00	2.55
Regulatory body requirements	2.75	2.75	2.75
Program funding	2.68	2.75	2.55
Student perception of competency	2.61	2.58	2.67
Changes in student characteristics	2.42	2.42	2.42

Abbreviation: EMS, emergency medical services.

^a Weighted importance score calculated as $\frac{2}{3}\ast \mathrm{mean}\mathrm{importance}+\frac{1}{3}\ast \mathrm{mean}\mathrm{feasibility}$.

3.5. Rounds 4 and 5

Participants were provided the top 12 research priorities based on preliminary ranking by weighted importance during the live meeting and asked to rank order the list. Table 3 shows the final ranking of the top 12 research priorities along with the percentage that were endorsed in the top 4. The top 4 research priorities identified by the expert panel were prehospital internship access, impact of lack of field and clinical experience, student health and safety, and EMS education program availability and accessibility. Majority consensus was reached, with 10/11 participants that responded in the final round (91%) agreeing with the list derived by the panel. The dissent from 1 participant was because he or she felt several of the research priorities overlapped.

TABLE 3.

Final ranking of the top 8 research priorities

	Frequency of occurrence for each ranka
Priority	1	2	3	4	5	6	7	8	9	10	11	12	Total score	Rank order	% in top 4
Prehospital internship access	5	2	0	0	2	0	1	1	0	1	0	0	112	1	58
Impact of lack of field and clinical experience	0	4	2	1	1	1	1	0	0	0	2	0	98	2.5	58
Student health and safety	1	3	0	1	1	2	2	2	0	0	0	0	98	2.5	42
EMS education program availability and accessibility	4	0	3	0	0	0	0	0	2	1	2	0	93	4	58
Psychomotor competency outcomes during COVID	0	0	2	2	0	4	1	0	2	0	0	1	81	5	33
Hospital/ambulatory site access	0	1	2	0	1	0	3	3	1	0	1	0	78	6	25
Cognitive competency outcomes during COVID	1	1	0	2	1	0	1	2	1	1	2	0	76	7	42
Keeping EMS education accessible for all students	1	0	1	2	1	1	0	0	3	2	0	1	74	8	33

Abbreviation: EMS, emergency medical services.

^a Each number represents the number of participants that selected a particular rank for that priority.

4. LIMITATIONS

This study has several limitations. The participants were selected as representatives of the stakeholder organizations included in the larger national task force; however, we asked participants to respond as individuals rather than on behalf of the organizations they represented. In asking organizations to select their representative and study participants in this manner, certain perspectives may have been excluded. This study was limited to impact on initial EMS education in the United States, though study participants and stakeholder organizations may have had international perspectives and membership. An EMS student representative was not sought, and this limitation is 1 example of a certain perspective that may have been excluded. Data collection was anonymous and only aggregated feedback was provided to the panel, but participants may have felt pressured to respond in kind with their organization's mission or chose not to participate fully. The varying level in round completion from the participants is acknowledged as a limitation of this study. We attempted to identify all relevant research priorities, but we recognize that rankings of importance and feasibility were based on the expert panel's experience and opinions, which may not reflect the wider EMS community. Finally, the response rate to the final round was initially lower than expected, leading to an unplanned reminder to attempt to obtain participation from the full expert panel.

5. DISCUSSION

The COVID‐19 pandemic hindered access to essential EMS educational activities, such as laboratory hands‐on learning and live patient encounters in hospital and field settings.¹¹, ¹², ¹³ The impact of the pandemic on EMS education and subsequent effects on the workforce pipeline created concern for stakeholders and national organizations. Understanding changes EMS education programs made to overcome challenges caused by the pandemic is critical to assessing impact and effects on learners and expected outcomes. EMS medical directors and emergency physicians should be cognizant of these changes and downstream effects—for example, EMS workforce shortages or newly certified EMS professionals who may need additional training and support as they enter the workforce. Understanding changes in response to the pandemic will likely identify opportunities for strengthening and ensuring the resilience of the EMS education during a future sustained disruption. Using a national panel of EMS education stakeholders in a modified Delphi process, we identified several important research priorities to guide future work in assessing the impact and effect of COVID‐19 on the education pipeline.

In total, the panel derived a list of 27 research priorities. The broad range of priorities likely represented the panel's recognition of the comprehensive impact and totality of the pandemic on EMS education. In recognizing various limitations in researching the 27 priorities, the panel reached consensus on the top 8 research priorities based on importance and feasibility. Research priorities weighted by importance and feasibility spanned a range of topics regarding access to educational activities and entry‐level competency. The intent of a manageable list of weighted research priorities is to aid in advocating for support and allocating resources from national stakeholders to support investigations deemed both important and feasible. Though we sought consensus on the prioritization of the top 8 research priorities, all topics identified by the panel represent potential areas with some level of identified importance from national stakeholders to guide future work.

In prior work from the Committee on Accreditation of Educational Programs for the EMS Professions and the National Association of EMS Educators, surveys of EMS educators identified concerns early in the pandemic consistent with many of the top 8 research priorities identified by the panel.¹¹, ¹² Access to education and the ability to complete continuing education were recognized as challenges affecting recruitment and retention in EMS, even before the COVID‐19 pandemic.²³, ²⁴, ²⁵ Many EMS education programs reported varying levels of continued operation,¹¹, ¹², ¹³ which potentially limits access to initial EMS education. Additionally, several programs reported increasing the use of virtual conferencing technologies to continue didactic education despite pandemic‐related restrictions.¹¹, ¹², ¹³ Increasing access to EMS education through virtual and hybrid approaches have been identified as ways to support access to initial and continuing education before and during the COVID‐19 pandemic.¹⁰, ²⁶ However, programs consistently reported concerns over loss of access to both simulation and live patient care opportunities,¹¹, ¹², ¹³ some of which were related to ongoing student and faculty health and safety concerns. Despite these reports of mitigation strategies and EMS education program modifications that address some of the priorities identified by the panel, little is known regarding concerns on student learning outcomes and achievement of entry‐level competency during the pandemic.

Guidelines advocate for states to obtain educational and institutional data including essential workforce data, understanding demand, and using information to understand the needs and performance of the EMS education pipeline.²⁷ Unfortunately, there is no single repository for data on EMS initial education to aid in investigating these research priorities. Data may exist at the paramedic level because of accreditation requirements through the Commission on Accreditation of Allied Health Education Programs. However, although emergency medical technicians (EMTs) represent the largest number of EMS learners, there is currently no centralized database or consistent national reporting mechanism to capture initial education program and student‐level data and outcomes. The lack of data at the EMT level hinders the feasibility of timely large‐scale studies and represents an opportunity for improvement in response to both the pandemic response and for the overall EMS initial education pipeline. This limitation is 1 specific example where there is a potential benefit from this national panel representing stakeholder organizations to raise through unified identification and advocacy. An example of a centralized repository of EMS continuing education does exist through the Commission on Accreditation for Prehospital Continuing Education (CAPCE). The CAPCE database may serve as a model for future work at the initial education level that would allow for better reporting and analysis of learning outcomes.

The pandemic has accelerated an ongoing trend toward increasing distance learning and hybrid programs for initial and continuing education.²⁶, ²⁸, ²⁹ Before the pandemic, programs had also begun using simulation‐based training to replace some of the traditional EMS education objectives of programs.²⁸, ²⁹ Several of the research priorities identified by the panel address this trend to examine if these changes in educational methods have affected the entry‐level competency of EMS clinicians.

One interesting finding that was noted by the panel in the second live meeting was regarding the perceived importance and ranking of student versus faculty health and safety. Although student health and safety was ranked in the top8, faculty health and safety were not. There may have been less concern regarding faculty safety as many faculty members are not exposed to live patient encounters; faculty members may have benefits and paid time off (eg, from the Families First Coronavirus Response Act) if exposed to COVID‐19,³⁰ whereas students may not; and because there is no consistent requirement that students hold health insurance in case they fall ill. At the beginning of the pandemic there was also a student‐specific concern regarding shortages of personal protective equipment during clinical assignments. Despite the recognition by the panel of the importance of student health and safety, and the ongoing personal burdens for many caused by the pandemic, both student and faculty stress and anxiety were ranked lower as research priorities. These topics remain potential research areas to promote EMS education program and student resilience.

Of note, there was 1 dissenting vote when final consensus was determined. This vote contended that the top 12 priorities had similarities that prevented them from reaching consensus. Although this vote is important and their perspective valued, it should not take away from 91% consensus being reached regarding these 12 research priorities. Future work in this area should recognize the potential for this occurring and take steps to ensure heterogeneity in themes.

We encourage continued research to address the identified research priorities in an effort to understand the impact of COVID‐19 on initial EMS education. In addition to investigating the impact from the COVID‐19 pandemic, this pandemic presents an opportunity to understand effects and lessons learned that could be translated to ways to increase resiliency in EMS education in preparation for the next sustained disruption. This includes identifying any changes in response to the pandemic with a positive impact that can be sustained or emerged as a new best practice. Ultimately, more work is needed to determine if the multitude of changes made by EMS education programs—such as modifications to curricula and terminal objectives, adjustments to instructional techniques, and the use of simulation in place of live patient encounters—enhanced or reduced entry‐level competency of EMS learners. Future research is encouraged to continue the collaborative approach among national stakeholder organizations and leaders as enjoyed during this Delphi panel.

In summary, understanding the impact of the COVID‐19 pandemic on initial education programs and the EMS educational pipeline is of national interest and critical to understanding the effects on learners and the EMS profession. The research priorities identified by this expert panel are an important first step to begin evaluating the educational infrastructure and processes that were threatened through the pandemic. Future task force work will focus on evaluating these priorities and identify solutions to strengthen the EMS education pipeline. Whether it is understanding the COVID‐19 pandemic or ways to prepare, collegial research projects supported collaboratively by national stakeholders are encouraged.

CONFLICTS OF INTEREST

WJL and PR are board members of the National Association of Emergency Medical Services Educators. KDM is a board member of the National Registry of Emergency Medical Technicians. JAM is a board member of the Commission on Accreditation for Prehospital Continuing Education. JRP, PR, and ARP are employees of the National Registry of Emergency Medical Technicians. The views expressed are the authors’ and do not necessarily represent the views of the stakeholder organizations or the United States government.

AUTHOR CONTRIBUTIONS

REC, WJL, and ARP conceived and designed the study. REC and JRP collected the data. REC, WJL, JRP, KDM, PR, and ARP analyzed the data. All authors provided interpretation of the data. REC drafted the initial manuscript. All authors critically reviewed the manuscript and approved the final version.

Biography

Rebecca E. Cash, PhD, MPH, is an Assistant Professor of Emergency Medicine at Harvard Medical School.

APPENDIX I. All research priorities identified by the panel (in alphabetical order)

I.1.

 

Priority	Rationale
Affective competency outcomes during COVID	Interpersonal relationship building and de‐escalation skills are necessary to ensure affective domain competency. The lack of in‐person assessment and patient interaction should be investigated to ensure we continue to meet affective competency.
Alternatives to clinical rotations	With limited access to clinical rotations, we should identify alternatives such as simulation lab, telemedicine, or field internship.
Changes in student characteristics	We should understand if student demographics and volume has changed from pre‐ to post‐pandemic.
Cognitive competency outcomes during COVID	Has COVID impacted the ability to reach competency in the cognitive domain? As programs adapted to COVID, it is unclear if student outcomes differ pre‐ and post‐pandemic, including impact on certification examination success, the role of distance learning and simulation, and program requirements for completion.
EMS education program availability and accessibility	Education programs should remain open and accessible to students. It is unclear how many are still operating, if class offerings or sizes differ (eg, only day vs day and night prior), and the impact of these changes.
Faculty availability, structure and size	We should understand if faculty demographics, size, and structure have changed from pre‐ to post‐pandemic. This includes availability of faculty to teach, education of faculty to teach on different platforms, shifting of lecturers to simulation instructors, and attrition of current educators.
Faculty health and safety	Faculty must be kept safe while performing in‐person instruction. Their perceptions of risk are unclear.
Faculty stress and anxietya	Understanding of faculty mental health concerns is necessary. COVID‐related stresses can lead to increased mental health concerns that should be recognized and explored.
Future of EMS education after the pandemic	The changes programs made in response to COVID have changed EMS education. How will future graduation rates be impacted? Will these changes continue beyond the pandemic?
Hospital/ambulatory site access	EMS programs rely on both hospital and ambulatory sites to meet continued competency. Programs need advocacy assistance in removing barriers (eg, liability, PPE, shortages of personnel and preceptors, and inherent value of EMS workforce) to keep these experiences available.
How is and how much simulation is being used	EMS education programs are increasingly using simulation technology in various ways. We need to understand the ways simulation is used, the simulation curriculum being developed, and impact on ability to interact with patients.
Impact of lack of field and clinical experience	It is unclear if there is an impact on student and patient outcomes if field and clinical experiences are limited.
Impact of program changes on future employment	The students who are completing programs adapted during the pandemic must still meet the needs of employers.
Keeping EMS education accessible for all students	Distance learning creates obstacles for students and educators, such as internet infrastructure and assuring access for students with different needs.
Medical director involvementa	Medical director approval of pandemic related changes is necessary. The extent to which medical directors are involved with educational changes and program advocacy is unclear.
Pandemic‐specific topics of education	Specific topics are unique to COVID and should be considered when educating students. These include airway management, cardiac arrest management, handling of death and dying patients, and appropriate use of PPE.
Prehospital internship access	EMS programs rely on pre‐hospital internship access to ensure competency. Programs need advocacy assistance in removing barriers (eg, liability, PPE, shortages of personnel and preceptors, and funding) to keep these experiences available.
Program funding	With the increase in alternative EMS education platforms, the impact on costs and funding models is unclear.
Program instruction changes due to COVID	Programs have made changes in response to the pandemic, including transitioning to online/distance learning, flipped classrooms, and independent study. The effectiveness and impact of these changes are unclear.
Psychomotor competency outcomes during COVID	Has COVID impacted the ability to reach competency in the psychomotor domain? As programs adapted to COVID, it is unclear if student outcomes differ pre‐ and post‐pandemic, including the role of simulation and requirements for completion.
Recruitment/enrollment	We need to understand current demands, barriers to recruitment and enrollment, and needs for the EMS education pipeline. These include perceptions of risk, accommodating diversity in student populations in recruitment efforts, and detailing the value of EMS education.
Regulatory body requirementsa	Programs are responsible to sometimes multiple regulatory bodies to provide evidence of student competency. How do programs continue to define competency in order to meet regulatory standards?
Simulation accessibility	All EMS education programs and students should have access to simulation. It is unclear if programs can afford simulation labs or have training to provide this type of education.
Student health and safety	Students must be kept safe during clinical and field internships. Students should be trained in proper use and have access to PPE. Students are also facing increased life stress.
Student perception of competencya	Confident student self‐perception is important to morale building. Do students feel competent to practice having missed significant portions of in‐person learning and practice?
Student stress and anxiety	Understanding of student mental health concerns is necessary. COVID‐related stresses can lead to increased mental health concerns that should be recognized and explored.
Substitution of simulation for clinical/field contacts	It is unclear how simulation can or should replace live patient encounters. What are the differences between learner and patient outcomes with simulation versus real world learning? Can minimum entry‐level competency be obtained with simulation alone? How does this compare to practices for physicians and nurses?

Abbreviations: EMS, emergency medical services; PPE, personal protective equipment.

^aAdded in Round 2.

Cash RE, Leggio WJ, Powell JR, et al., for the Pandemic Educational Effects Task Force . Emergency medical services education research priorities during COVID‐19: A modified Delphi study. JACEP Open. 2021;2:e12543. 10.1002/emp2.12543