Children’s Safety Initiative: A National Assessment of Pediatric Educational Needs amongEmergency Medical Services Providers

Abstract

Objective

Emergency Medical Services (EMS) providers may have critical knowledge gaps in pediatric care due to lack of exposure and training. There is currently little evidence to guide educators to the knowledge gaps most likely to improve patient safety. The objective of this study was to identify educational needs of EMS providers related to pediatric care in various domains in order to inform development of curricula.

Methods

The Children’s Safety Initiative-EMS performed a three-phase Delphi survey on patient safety in pediatric emergencies among providers and content experts in pediatric emergency care including physicians, nurses and pre-hospital providers of all levels. Each round included questions related to educational needs of providers or the effect of training on patient safety events. We identified knowledge gaps in the following domains: case exposure, competency and knowledge, assessment and decision making, and critical thinking and proficiency. Individual knowledge gaps were ranked by portion of respondents who ranked them “highly likely” (likert-type score 7–10 out of 10) to contribute to safety events.

Results

There were 737 respondents who were included in analysis of the first phase of the survey. Paramedics were 50.8% of respondents; EMT-Basics/first responders were 22% and physicians 11.4%. The top educational priorities identified in the final round of the survey include: pediatric airway management, responder anxiety when working with children, and general pediatric skills among providers. The top three needs in decision-making include knowing when to alter plans mid-course, knowing when to perform and advanced airway, and assessing pain in children. The top 3 technical or procedural skills needs were pediatric advanced airway, neonatal resuscitation, and intravenous/intraosseos access. For neonates, specific educational needs identified included knowing appropriate vital signs and preventing hypothermia.

Conclusions

This is the first large-scale Delphi survey related to pediatric pre-hospital education. Our results provide foundational information related to the educational needs of pre-hospital providers. Medical directors and educators can use the results to shape future curricular development.

Introduction

EMS providers must respond to emergencies of all kinds, and care for patients across the age spectrum. EMS providers spend the majority of their time caring for adults, though their impact on pediatric patients is potentially great. One child dies every hour in the US due to an injury with automobile accidents, suffocation, drowning, poisoning, fires, and falls being the most common causes of injuries.¹ EMS providers are often the first point of contact with the healthcare system for these children and they have the potential to save lives as well as limit pain and suffering.

Though the potential impact of EMS care is high, there is also the potential for knowledge gaps among EMS providers in pediatric care. Children represent between 4% and 13% of EMS transports, and critically-ill children are infrequently seen by any individual provider.^2–5 The 2006 Institute of Medicine report “Emergency Medical Care for Children, Growing Pains” identified the broad categories of limited initial training in educational programs, infrequent case exposure, and provider discomfort as critical barriers to quality pre-hospital pediatric care.^6,7 Several existing programs focus on pediatric EMS education including the Pediatric Education for Prehospital Providers (PEPP) Education program and the National Association of Emergency Medical Technicians Pediatric Prehospital Care Course.^8,9 Due to the limited existing literature, these curricula were largely based on the informally derived consensus of a limited group of experts in the field and did not include significant numbers of practicing paramedics or Emergency Physicians who receive pediatric EMS transports. A more rigorous and inclusive educational needs assessment is critical in order to refine the pediatric content in EMS training programs to target the most immediate needs, refine existing continuing education programs, as well as inform design of new curricula. Needs assessments should not only identify knowledge gaps, but prioritize them based on the potential for these gaps to adversely affect patient outcomes.

As an initial approach to help clarify educational needs and identify further research questions we conducted a broad-based national Delphi survey of EMS providers and experts. The Delphi survey process has been used in medical research and has been described elsewhere in depth.¹⁰ Delphi surveys use several survey rounds, each administered to the same group and evolve based on the results of the previous round, in order to reach consensus among respondents. The Delphi process has the potential to be more informative than a traditional survey and less prone to bias than in-person discussions. The objective of this study was to seek consensus among a large national panel regarding contributors to patient safety events and outline a specific prioritized list of educational needs or knowledge gaps which can be addressed to improve patient safety in the pre-hospital care of children.

Methods

Study setting and dates

We conducted a three-phase Delphi survey, enrolling a national panel of pediatric pre-hospital providers and content experts. All three phases were completed between August of 2011 and July of 2012.

Participants and inclusion/exclusion criteria

Respondents included Emergency Physicians, Emergency Nurses, Paramedics, EMT-Intermediate, EMT-Basic and First Responders. The respondents were recruited via email by the investigators using state Emergency Medical Services for Children (EMSC) contacts and various Emergency Physician, EMS physician, and EMS provider listserves, as well as the EMS provider Facebook page of a study investigator. Though this is a needs assessment for prehospital providers, Emergency Physicians and nurses were included as they are the primary group which receives patients from prehospital providers and performs the initial assessment on hospital arrival. In this role, they have a different perspective on prehospital care and may be able to identify errors which the field providers cannot. Exclusion criteria included age less than 18 and failure to complete the demographics section of the survey. Demographics of respondents were compared across the 3 phases of the study to see if there was a difference in those who dropped out and those who completed all phases. We used SurveyMonkey as the survey tool.

Main outcome measures and survey methodology

During the first round, participants were asked to rate certain factors according to their likelihood to contribute to patient safety events. Unstructured questions were asked in addition to the 9-point Likert-type scale questions to aid in refining future survey rounds. The domains of potential safety events included in round one were: assessment and decision making, technical and procedural skills, medication and equipment, experience and training in the care of children, communication, EMS cultural norms, clinical situations, scene characteristics, and resources available. Between 5 and 12 questions were asked within each of these domains regarding likelihood of these factors contributing to safety events in pre-hospital care of children. Data from the first round were ranked by median score and then percent of respondents who chose “highly likely” on the Likert-type scale (score of 7, 8, or 9). Twenty-two items from the survey were in the top third of “highly likely to contribute” category across all professional groups and were advanced to round 2.

In round 2, in addition to the 22 items advanced from round one, we asked additional questions in certain domains including assessment and decision making and procedural skills either to address comments raised by respondents in unstructured fields or to explore further detail in these areas. Respondents had the opportunity to provide free text answers regarding their top 3 most likely knowledge gaps which lead to safety events by patient age group from neonate to adolescent. Advancement to round 3 was based on similar criteria used for advancement to round 2. The qualitative responses were analyzed independently by trained research assistants using NVivo™ software to identify and code themes and rank responses per theme.

Round 3 of the survey applied the Likert-type scale to the questions advanced from the previous rounds. In addition, we added a ranking question regarding the likelihood of specific clinical situations leading to safety events.

We used results of all 3 phases of the survey to identify specific knowledge gaps, clinical scenarios, and challenging procedures which were felt to be likely to contribute to patient safety events. The 3 iterations of the Delphi survey and consultation with an expert advisory panel during the study led to the definition of global domains of EMS safety events which encompass the other specific categories. The expert advisory panel members included high level leaders from the National Association of State EMS Officials (NASEMSO), the National Association of EMS Physicians (NAEMSP), the National Registry of Emergency Medical Technicians (NREMT), and the US Human Resources and Services Administration (HRSA). These members helped us interpret the results of the study to inform subsequent Delphi rounds, categorize the results into specific domains, and contextualize them in the current state of the EMS system. The domains which are potentially amenable to educational intervention were included in this needs assessment and are: case exposure, competency and knowledge, assessment and decision making, and critical thinking and proficiency. Survey results were analyzed using SPSS. The Institutional Review Board of Oregon Health & Science University approved the study, consent was obtained online by all survey respondents prior to completion of the survey.

Results

Our survey respondents came from 44 of the 50 US States. Most of the states not represented were in the Southeastern region. Table 1 depicts the demographic characteristics of survey respondents. In round 1, 755 respondents consented to participate. Two respondents were excluded due to age less than 18 and 16 were excluded due to not specifying profession in the demographics section of the survey, resulting in 737 respondents continuing to the analysis phase of round 1. The 753 respondents who met age requirements were invited to participate in round 2; 614 (82%) completed the survey and were included in the round 2 analysis. Of the original 753 participants, 492 (65%) completed round 3 of the survey. EMT Paramedics comprised 50.8% of the original participants and 51.5% of final participants. The representation of all other professional groups included in the study was also stable and varied by less than 2% across the 3 survey phases.

Table 1.

Respondent demographics (n=737)

Characteristic	Number (percent)
Age, mean (sd)	41.7 (10.3)
Female, n (%)	284 (39.3)
Non-white, n (%)	63 (8.7)
Highest level of training
1st Responder/EMT Basic, n (%)	159 (22.0)
EMT Intermediate, n (%)	50 (6.9)
Paramedic, n (%)	367 (50.8)
RN/NP/LPN/Respiratory Therapist, n (%)	64 (8.9)
Physician, n (%)	82 (11.4)
Years of Experience at Current Training Level, mean (sd)	14.0 (9.2)
Years working in EMS, mean (sd)	16.8 (9.4)
Employment
Private Ambulance, n (%)	163 (22.6)
Public Ambulance, n (%)	278 (38.6)
Emergency Department, n (%)	86 (11.9)
State Office, n (%)	19 (2.6)
Urban, n (%)	244 (33.9)
Suburban, n (%)	226 (31.4)
Rural, n (%)	250 (34.7)
Number of respondents who have children, n (%)	558 (77.8)

Table 2 depicts the final consensus of knowledge gaps likely to contribute to safety events from round 3 of the Delphi survey. Lack of experience with pediatric airway management was consistently rated as the most important educational need across all three rounds of the Delphi followed by heightened anxiety when working with children.

Table 2.

Top 10 knowledge gaps related to patient safety events.

Knowledge gap	% Rated Highly Likely to Contribute to Safety Events
Lack of experience with pediatric airway management	73.4%
Heightened anxiety when working with children	72.5%
Lack of proficiency in pediatric skills among providers on scene	66.6%
Lack of experience with pediatric equipment	57.9%
Lack of ongoing pediatric training (CME) for EMS providers	48.6%
Knowing when to alter plans mid-course	47.3%
Determining whether a patient is sick or not sick	45.0%
Knowing when to perform advanced airway procedure (e.g. LMA, ETT, King, etc.)	44.5%
Making the decision to “Scoop and Run”/”Load and Go” or “Stabilize before Transport”	38.6%

Table 3 provides qualitative responses regarding specific competencies for pediatric EMS education by patient age group. The general themes of experience, training, and skills in pediatrics as well as assessment and monitoring emerged as the top 2 competencies for all age groups, from neonates to adolescents.

Table 3.

Prioritized list of knowledge gaps contributing to patient safety events by patient age.

Factor	Number of references within free-text responses (percent of total references)
Neonates (0 – 28 days)
In general, lack of experience, training, and skills	310 (26.6%)
Assessment and/or monitoring	80 (6.9%)
Medication calculation	63 (5.4%)
Prevention of hypothermia/knowing neonate vitals	24 (2.1%)
Infants (29 days - 11 months)
In general, lack of experience, training, and skills	202 (18.2%)
Assessment and/or monitoring	123 (11.1%)
Lack of exposure to infants	89 (8.0%)
Medication calculation	72 (6.5%)
Toddlers (12 – 24 months)
In general, lack of experience, training, and skills	179 (16.7%)
Assessment and/or monitoring	114 (10.6%)
Lack of exposure to toddlers	68 (6.3%)
Medication calculation	63 (5.9%)
School age (25 months - 11 years)
Assessment and/or monitoring	91 (10.1%)
Lack of exposure to school age children	84 (9.4%)
Medication calculation	57 (6.4%)
Adolescents (12 – 18 years)
In general, lack of experience, training, and skills	59 (8.6%)
Assessment and/or monitoring	60 (8.8%)
Medication calculation	41 (6.0%)
Lack of exposure to adolescents	20 (2.9%)

Table 4 displays what participants reported to be the 5 most important knowledge gaps within the domain of assessment and decision-making. Knowing when to alter plans mid-course and knowing when to perform an advanced airway were the top two factors felt likely to be associated with safety events. The five most challenging procedural skills were identified from round 2 when respondents were asked to rank procedures according to difficulty, and are listed in Table 5; advanced airway management and neonatal resuscitation were the top two. The five clinical scenarios at highest risk for safety events are respiratory failure/arrest, trauma, cardiac resuscitation, seizures and child abuse.

Table 4.

Top 5 Knowledge Gaps in the Domain of Assessment and Decision Making

Factor	% Ranked Highly Likely
1. Knowing when to alter plans mid-course	37.0%
2. Knowing when to perform advanced airway procedure (e.g. LMA, ETT, King, etc.)	36.6%
3. Assessing pain in pediatric patients	35.2%
4. Determining whether patient is sick or not sick	24.5%
5. Choosing the correct EMS protocol	19.1%

Table 5.

5 Most Challenging Procedural Skills in Children

Skill	n=614 n, (%)
Pediatric advanced airway	438 (71.3)
Newborn resuscitation	378 (61.6)
Pediatric IV/IO	269 (43.8)
Home ventilators	221 (36.0)
C-Spine	166 (27.0)

Discussion

This large national survey of EMS professionals revealed important knowledge gaps related to pediatric care and provides a roadmap for pediatric education. We found that airway management was the most critical knowledge gap and should be a focus of pediatric education. Airway management encompasses many skills, from basic positioning to advanced airway placement, and is a critical step in care of many pediatric emergencies. Provider anxiety when caring for children was also an important factor. We also found gaps in medical decision making including knowing when to alter care plans and when to perform procedures.

Experts in medical education have taught that needs assessment is a best practice for curriculum design though up to this point there has been little evidence to clarify the educational needs of EMTs in pediatric emergencies.¹¹ One potential way to incorporate the findings of our study in an educational intervention would be to use simulation to practice the procedural skills listed in Table 5 in the context of one of the high yield clinical scenarios we identified. The results of this study could also be utilized by national EMS educators and leaders to design an evidence-based standardized pediatric curriculum, or refine existing curricula. Medical education, including Emergency Medicine, is undergoing a significant transition towards competency based learning.¹⁴ The general concept of competency based learning is that the critical outcome of education is competency in the domains needed for practice, rather than number of hours or months spent pursuing education.¹⁵ These results could be used to identify important pediatric competencies.

Currently, only 41% of states require pediatric training for initial EMT certification for EMT-Basics and EMT-Paramedics. For recertification 63% and 67% of states require pediatric educational hours respectively with states requiring between 2 and 9 hours of training every 2 years.¹² The national EMSC program could advocate for more pediatric educational hours and even provide suggestions on how some of them are best used.¹³

Our findings are generally supported by the existing evidence. A recently published qualitative study based on focus groups of EMTs identified several provider level factors which were felt to contribute to safety events including heightened anxiety when working with children, lack of pediatric training experience, and difficulty in assessment and decision making.¹⁵ Anxiety related to pediatric care likely potentiates errors as stressful clinical scenarios have been shown to increase the chance of a medication error among experienced paramedics.¹⁶ Simulation-based studies have corroborated our results and identified errors in many facets of basic airway and ventilatory management including appropriate application of oxygen, use of oral airways, and bag mask ventilation.^17,18,19 In addition, EMS providers themselves have indicated they feel they would benefit from more pediatric education.²⁰

Based on experience in hospital medicine we expected medication and communication issues would be important contributors to errors though they did not make it to round 3 of the survey indicating a lack of consensus of their likelihood to contribute to safety events. The communication structure in EMS is different from hospital based communication; following a brief and protocol driven format, and may be less likely to contribute to safety events. It is possible that medications are a less important knowledge gap in prehospital medicine, or at least less perceived to be, since the majority of EMS calls may not involve medication administration, and the number of medications utilized is highly limited.

Our study has several limitations to consider. First, participation in the study required access to a computer, and the internet which may have caused selection bias in the group of respondents. Next, Paramedics were the most highly represented group of EMT respondents making the results less generalizable to other groups such as EMT-Basics. In addition, there was modest attrition across the three rounds, with 65% of original participants completing round 3; however, the composition of the respondents remained stable. The Delphi panel included a relatively experienced cohort of providers which may not reflect the “average” EMT. Finally, though the Delphi method is a rigorous survey method, any survey is an indirect assessment of actual behavior and carries inherent limitations compared to studies which directly observe behavior. However, given the resources required to perform sufficient direct observation to observe many pediatric safety events, we feel our methods are a reasonable initial step in this field.

Conclusions

This national Delphi survey identified several areas where targeted education may be most likely to have a positive impact on pediatric pre-hospital patient outcomes. Efforts to improve pediatric patient safety in EMS should focus on pediatric airway management, assessment and decision-making, and also work toward mitigating the anxiety when working with children.