A Modified Delphi Study for Development of a Pediatric Curriculum for Emergency Medicine Residents

Abstract

Objectives

Emergency medicine (EM) trainees are expected to learn to provide acute care for patients of all ages. The American Council for Graduate Medical Education provides some guidance on topics related to caring for pediatric patients; however, education about pediatric topics varies across residency programs. The goal of this project was to develop a consensus curriculum for teaching pediatric emergency care.

Methods

We recruited 13 physicians from six academic health centers to participate in a three‐round electronic modified Delphi project. Participants were selected on the basis of expertise with both EM resident education and pediatric emergency care. The first modified Delphi survey asked participants to generate the core knowledge, skills, and experiences needed to prepare EM residents to effectively treat children in an acute care setting. The qualitative data from the first round was reformulated into a second‐round questionnaire. During the second round, participants used rating scales to prioritize the curriculum content proposed during the first round. In round 3, participants were asked to make a determination about each curriculum topic using a three‐point scale labeled required, optional, or not needed.

Results

The first modified Delphi round yielded 400 knowledge topics, 206 clinical skills, and 44 specific types of experience residents need to prepare for acute pediatric patient care. These were narrowed to 153 topics, 84 skills, and 28 experiences through elimination of redundancy and two rounds of prioritization. The final lists contain topics classified by highly recommended, partially recommended, and not recommended. The partially recommended category is intended to help programs tailor their curriculum to the unique needs of their learners as well as account for variability between 3‐ and 4‐year programs and the amount of time programs allocate to pediatric education.

Conclusion

The modified Delphi process yielded the broad outline of a consensus core pediatric emergency care curriculum.

Emergency medicine (EM) physicians acquire proficiency in the emergent management of all patients including pediatric patients during their training. Despite the growth of pediatric EM as a subspecialty, pediatric EM (PEM) subspecialists only care for 10% to 20% of the pediatric patients in the emergency setting across the United States.1 The remaining 80% to 90% of pediatric emergency care patients are cared for by EM physicians and/or general practice pediatricians.2, 3, 4, 5 The Accreditation Council for Graduate Medical Education (ACGME) requires EM residents to have approximately 20% of their patient encounters with patients less than 18 years of age, including the critical care of infants and children.6 While time dedicated to pediatrics has increased in recent years,7 concerns remain as to whether this allows sufficient experience to develop the mastery level competency for the EM physician to effectively care for children.4, 7

Although the type of clinical experiences available is beyond the scope of this study, many studies support the need for a curriculum that does not solely rely on patient experiences for knowledge acquisition. Despite accreditation requirements designed to ensure sufficient pediatric education, concerns have been raised over the effectiveness of standards. A recent survey of EM residency directors revealed that EM residents spend 13% of their time on dedicated pediatric EM at tertiary care children's hospitals while the other requirements were met through rotations that treat both children and adults.8 In a related single‐site study, Chen et al. found that EM residents were sent to a tertiary pediatric hospital to increase their pediatric experience, only to encounter such a saturated learning environment that they averaged three patients per shift. Additionally, Chen et al. found that EM residents were more likely to treat older pediatric patients, perform only minor procedures, and see fewer critically ill patients compared to their non‐EM peers.9, 10 Similarly, Langhan et al.11 reports that EM residents feel uncomfortable with pediatric and neonatal resuscitations. While we recognize that these individual reports of deficiencies in EM resident education might be attributable to individual systems problems, we believe that collectively they demonstrate the need for a more rigorous core pediatrics curriculum to guide EM resident education.

Pediatric emergency care is taught to EM residents in a variety of clinical environments by a mix of both EM‐ and PEM‐trained faculty members. The goal of this study was to create a consensus pediatric EM curriculum that can be implemented by EM/PEM teaching faculty regardless of clinical training and practice site. By providing educational leaders with this framework, they can begin to develop competency and milestone‐based assessments, create didactics, and build simulations to minimize the gaps in the clinical experiences of their individual learners.

Aside from the EM Model of Clinical Practice, there is no up‐to‐date standard curriculum of PEM for the education of EM residents. In the late 1980s and early 1990s, some examples of standardized PEM curricula and objectives were proposed.12, 13, 14, 15 The Society for Academic Emergency Medicine through Council of Emergency Medicine Residency Directors regularly publishes recommended core content areas including pediatrics; however, these are limited to lists of general disease topics.16 Recently, a set of best practices for PEM education of EM residents was published which highlighted teaching a generalized approach to pediatric patients, focusing on the importance of child development and congenital illnesses. They also called for the establishment of a standard competency assessment.7 Interestingly, a consensus PEM clerkship curriculum for medical students was just recently published.17, 18, 19

The objective of this study was to engage an expert panel to establish a contemporary, consensus PEM curriculum and prioritize the broad spectrum of pediatric emergency care topics. This study is intended to supplement, not replace, the EM Model of Clinical Practice while providing an additional level of granularity and focus on important pediatric content. Furthermore, the effort was intended to be customized specifically for the education of EM residents. A consensus curriculum will establish foundational core knowledge and skills for EM learners and be the initial steps in the movement from process‐based to competency‐based education in pediatric emergency care.

Methods

The Delphi method is a well‐established method for generating curriculum content, solving problems, creating research agendas, conducting needs assessments, and other purposes.19, 20, 21, 22, 23, 24, 25 This process represents expert consensus and can be considered an evidence‐based process in educational research. For this study, we implemented a three‐round modified Delphi technique to generate a recommended core curriculum for EM residency programs–designed to teach care of the pediatric patient in the acute care setting. Our modified Delphi methods, including data processing, were modeled from those recommended by Witkin and Altschuld.21 Specific features included content generated by panelists, medium to small size groups of individuals with specialized knowledge (experts), up to four iterative rounds, and anonymity of panelist's contributions. This study was determined to be exempt research by the Nationwide Children's Hospital.

Selection of Expert Panelists

The original research team was composed of individuals from the Departments of Emergency Medicine at The Ohio State University and Nationwide Children's Hospital. This group recruited and selected participants who were representative of both 3‐ and 4‐year residency programs and academic programs housed in three different care‐delivery settings: free‐standing children's hospitals, children's emergency departments (EDs) housed within adult EDs, and EDs in community hospitals. Panelists were identified and recruited based on their individual expertise with both resident education and pediatric care (see Tables 1 and 2). Panelists included EM‐boarded educators and PEM physicians whose initial board certifications were a mix of EM and pediatrics. This selection was deliberate to ensure the voice of EM educators was not lost to the subspecialist's voice.

Table 1.

Demographic Profiles and Qualifications of Delphi Panelists

Panelist	Board Certifications	Institution Name	Roles and Responsibilities
Rebecca Fastle, MD	PEDS/PEM	University of New Mexico School of Medicine	PEM PD Asst. Prof.
Andrew M. King, MD	EM	The Ohio State University Wexner Medical Center & Nationwide Children's Hospital	EM APD Asst. Prof
Laura Hopson, MD	EM	University of Michigan Health System/St. Joseph Mercy Hospital	EM PD Asst. Prof.
John D. Hoyle, MD	EM/PEDS	Western Michigan University Homer Stryker School of Medicine‐Bronson/Borges Hospital	EMAPD Prof.
Kelly Levasseur, DO	PEDS/PEM	Oakland University–Beaumont Health System	PEM PD Asst. Prof.
Michael Mitchell, MD	PEDS/PEM	Wake Forest University School of Medicine–Baptist Medical Center	PEM APD Asst. Prof.
Jennifer Mitzman, MD	EM/PEM	The Ohio State University Wexner Medical Center & Nationwide Children's Hospital	Lead Pediatric Educator EM Residency Asst. Prof.
James O'Neill, MD	EM/PEM	Wake Forest University School of Medicine–Baptist Medical Center	Former–EM APD Current–PEDS/EM Fellowship PD Ass. Prof.
Philip Pazderka, MD	EM	Western Michigan University Homer Stryker School of Medicine–Bronson/Borges Hospital	Former–EM APD Current–EM PD Asst. Prof.
Marcia Perry, MD	EM	University of Michigan health System/St. Joseph Mercy Hospital	EM APD Asst. Prof
Payal Shah, MD	EM	Oakland University–Beaumont Health System	EM APD Asst. Prof.
Sara Skarbek‐Borowska, MD	PEDS/PEM	University of New Mexico School of Medicine	EM APD PEM Education Director Asst. Prof.
Rachel Stanley, MD	PEDS/PEM	The Ohio State University Wexner Medical Center & Nationwide Children's Hospital	Division Chair Asst. Prof.

APD = associate program director; PEM = pediatric emergency medicine.

Table 2.

Demographic Profiles of Delphi Participants

Institution Name	Institution Type	Type of Residency Program	Residency Program Size	Percent Time in Curriculum Allocated to Pediatrics (%)
Oakland University–Beaumont Health System	Pediatric unit within adult ED	3‐year program	14 residents per class	20.5%
The Ohio State University Wexner Medical Center & Nationwide Children's Hospital	Free‐standing children's hospital	3‐year program	16 EM and 2 EM‐IM residents per class	20.0%
University of Michigan health System/St. Joseph Mercy Hospital	Free‐standing children's hospital	4‐year program	16 residents per class	17.5%
University of New Mexico School of Medicine	Pediatric unit within adult ED	3‐year program	14 residents per class	20.0%
Wake Forest University School of Medicine–Baptist Medical Center	Pediatric unit within adult ED	3‐year program	15 residents per class	22.5%
Western Michigan University Homer Stryker School of Medicine–Bronson/Borges Hospital	Community hospital(s)	3‐year program	20 residents per class	19.5%

We gathered preliminary content material during modified Delphi round 1 through an open‐ended questionnaire that asked panelists to provide the core knowledge topics and clinical skills required of a resident to provide care to a child in the acute care setting. We also asked panelists to suggest the experiences that residents needed to achieve their recommended core knowledge and skills. And finally, we asked participants to provide the resources they used to generate their content materials (see Data Supplement S1, available as supporting information in the online version of this paper).

We aggregated the results of modified Delphi round 1 into a prioritization survey and fed this back anonymously to the participants during round 2. We provided the number of times the content topic or item had been “nominated” by the participants during round 1 and asked participants to use that information to rate each topic using a five‐point Likert‐type scale (with options labeled from 1 = not important, to 5 = very important). For organizational purposes, we presented the content material to participants in systems representing: organ systems; developmental, psychological, or sociologic typologies; or skill sets (see Data Supplement S1). Using a technique for prioritization recommended by Altschuld and Thomas, round 2 items were scored for strength of “importance” by multiplying the frequency of each rating by the rating value for each item.21, 25 For example, an item that was rated a “5″ by all 13 participants was scored a “65.” We also calculated the percentage of respondents who endorsed a topic by rating it “very important.” The items were then sorted by score and percentage of “very important” and assigned a rank.

We presented the results of modified Delphi round 2 to participants in a final modified Delphi round 3 survey. During this round, we presented the content topics by rank order. We also provided the other scoring information and additional comments gathered during round 2. The instructions for round 3 asked participants to sort the content material into categories: 1) highly recommended or must teach topics, that is, content that is highly recommended for an EM residency curriculum; 2) partially recommended or may teach topics, that is, content that is considered optional based on local needs and time in the curriculum; and 3) not recommended or don't teach topics, that is, content that is not recommended because it is material that is more appropriate for other levels of education (i.e., fellowship level training), can be taught in the context of adult care, or were felt to be irrelevant in contemporary medical practice (see Data Supplement S1).

Results

Twelve of 13 panelists contributed curriculum topics covering core knowledge, skills, and requisite experiences during the first modified Delphi round. All 13 panelists participated in the prioritization of topics during rounds 2 and 3. Participants represented academic faculty across six EM residency programs. Physician participants were involved in EM resident and/or PEM fellowship education or leadership positions in EM or PEM administration and have been involved in pediatric care. All participants were trained and board‐certified in EM or pediatrics with one participant in both. Some participants were PEM fellowship trained and board certified, which is consistent with the types of faculty teaching pediatric emergency medical care to residents.

Panelists generated 400 knowledge topics and 206 clinical skills during round 1. We reduced the original list of 400 knowledge items to 153 unique topics by eliminating redundancy and moving some of the topics to the list of core skills. In a similar fashion, the 206 clinical skills were reduced to 84. Participants responded to the lists of rank‐ordered revised items during round 3. Comments provided during round 3 contributed to item placement in one of the final three lists (highly recommended‐must teach topics‐recommended curriculum items, partially recommended‐may teach topics‐optional curriculum items, and not recommended‐don't teach topics‐items reserved for PEM or other specialists). The highly recommended or must teach curriculum content included 63 knowledge topics and 41 clinical skills (see Table 3). The partially recommend or may teach curriculum content included 65 knowledge topics and 21 clinical skills (see Table 4). The don't teach curriculum content included 25 knowledge topics and 22 clinical skills (see Data Supplement S1). Most of these items were eliminated due to the participant's belief that these topics could be covered through the regular EM curriculum or because their significance is reduced due to improved access to reference materials. A few items were combined or eliminated due to redundancy.

Table 3.

Highly Recommended Curriculum Content for Teaching Pediatrics to EM Residents: Both Knowledge and Skills Topics

Knowledge Topics
Recognize a sick child	Detecting physical abuse through history and physical examination	Diagnosis and management of Neisseria
Acute DKA and hyperglycemia	Major traumatic brain injury	Recognition of emergencies in febrile sickle cell disease
Intussusception	Assessing child for aspirated foreign bodies	Recognition of normal vital signs based on age and stage of development
The approach to the febrile or septic neonate	Respiratory distress	Indications for emergent blood transfusions in patients with shock
Malrotation/volvulus	Laws pertaining to medical personnel responsibility for child abuse and neglect	Ability to trouble shoot common pediatric medical devices–tracheostomy
Common signs and symptoms of physical abuse in children	Febrile seizures	Common traumatic conditions–head injury, blunt head trauma, concussion with return to play instructions
Neonatal congenital cardiovascular presentations	Retropharyngeal abscess*	Myocarditis
Pediatric sepsis	Altered mental status	Slipped capital femoral epiphysis
Meningitis	Minor head injury	Ingested foreign bodies
Asthma	SVT	Recognition of the “high‐stakes” milieu of pediatric emergencies
Recognition of fracture patterns that suggest abuse	Application of rules for fluid resuscitation in children 4.2.1 rule for maintenance of IV fluid resuscitation	Pharyngitis
Discriminate between patients who can be sent home and those who need admission to the hospital	Unique patterns of injury in the pediatric spine	Pyloric stenosis
Discrimination between common and deadly rashes	Fever and neutropenia	Epidural hematoma
Diagnosis and stabilization involving small dose ingestions dangerous or fatal to toddlers	Know signs and symptoms of Kawasaki's disease	Management of sickle cell pain crisis
Application of rules for fluid resuscitation in children, 20 mL/kg bolus	Preseptal/orbital cellulitis	Recognition of pediatric heart failure
Bronchiolitis	Intra‐abdominal surgical emergencies	Radiology–determination of when to use imaging: risks and benefits
Appendicitis	Diagnosis of children with a pediatric (or toddler) limp	Diagnosis and management of sexual abuse
CAH shock in neonates	Vomiting–by age group	Postoperative congenital heart disease child
Jaundice	Acute otitis media	Pediatric dosages acetaminophen (Tylenol)
Croup	Anaphylaxis	Postoperative tonsillectomy
Recognize patients who need higher levels of care than the ED, i.e., NICU or PICU (1)	Resources for evaluation of children suspected of suffering child abuse and neglect	Diagnosis and management of Rocky Mountain Spotted Fever
Clinical Skills Topics
Basic airway maneuvers, including appropriate positioning based on pediatric anatomy	Lumbar puncture	Laceration repair (suturing) with consideration for child's age
ET intubation of infants	Laryngeal mask airway	How to take a pediatric‐specific history including pertinent positives such as birth history/birthweight/loss
ET intubation of young children	Lumbar puncture in neonate	Establish rapport with children of different ages
Pediatric Advanced Life Support	How to successfully perform a physical examination on pediatric patients of varying ages	Interpretation of chest X‐ray
Place an intraossesous line	Needle crichothyrotomy	Immobilize common pediatric fractures using splinting
Pediatric trauma resuscitation	Reduction of radial head subluxation (nurse maid's elbow)	Foreign‐body removal–nose
Airway management for respiratory failure	Age appropriate neurologic assessment	Tracheostomy tube placement
Bag‐valve‐mask ventilation (stress over ET intubation)	Cervical spine clearance based on age	Mobilizing resources for nonaccidental trauma
Pediatric airway adjuncts	Pediatric burn management	Ventilator management
Resuscitation strategies involving blood, fluids, and glucose	Use of computerized tomography for scans of the head	Matching appropriate agent for the procedure
Calculate bolus and maintenance fluids for children based on age	Needle decompression of a pneumothorax	Foreign‐body removal–ear
Cardioversion/defibrillation	Neonatal resuscitation	Place an IV line
Noninvasive airway management HFNC, nasal cpap, optiflow, BiPAP	Pediatric Basic Life Support	Using succinylcholine for ketamine for laryngospasm rescue
Conversion of SVT	Effective communication with parents

BiPAP = bilevel positive airway pressure; CAH = congenital adrenal hyperplasia; CPAP = continuous positive airway pressure; DKA = ketoacidosis; ET = endotracheal; HFNC = high‐flow nasal cannula; NICU = neonatal intensive care unit; PICU = pediatric intensive care unit; SVT = supraventricular tachycardia; IV = intravenous.

Table 4.

Partially Recommended or Optional Curriculum Content for Teaching Pediatrics to EM Residents: Both Knowledge and Skills Topics

Knowledge Topics
Constipation	Anatomic and physiologic differences of pediatric patients based on developmental stages: neonate, infant, toddler, preschooler, grade schooler, adolescent/teenager	Pediatric dosing for adenosine
Use of fracture rules such as Salter‐Harris	Key decision rules–Kocher criteria for septic joint	Neurologic emergencies–stroke
Common traumatic conditions–blunt abdominal trauma	Chest pain	Manage of ocular emergencies–trauma
Pyelonephritis	Recognition of uncommon but serious hematologic disorders	Pediatric devices–gastrostomy tube
Use of head/cervical spine rules	Recognize and manage–viral exanthems	Manage of ocular emergencies–foreign bodies
Conditions/criteria for transfer to specialty care	Viral syndromes	Glomerulonephritis
Headache	Pediatric dosage of epinephrine (anaphylaxis)	Management of ocular emergencies–tips and tricks for examining a child's eyes
Musculoskeletal injuries by age group	Pediatric dosage of ketamine	Weakness or failure to thrive
Acute presentations–pneumonia, viral and bacterial	Pediatric dosage of epinephrine (code)	Treatment of acute presentations of cystic fibrosis
Neonatal hypoglycemia	Upper and lower urinary tract infections	ENT emergencies–epistaxsis
Gastroenteritis	Persistent fever over 7 days	Pediatric dosage of morphine
Suicide	Idiopathic hypertrophic subaortic stenosis (hypertrophic cardiomyopathy)	Antibiotic stewardship
Common traumatic conditions–penetrating trauma	Encephalitis	Common problems of NICU graduates– bronchopulmonary dysplasia: chronic lung disease from no surfactant
Stabilization of caustic ingestion (tide pods)	Initial management of metabolic diseases	Meckel's diverticulum
Higher risk for medical error in pediatric patients vs. adults	Manage special‐needs children–cerebral palsy	Knowledge of vaccination schedules and what illnesses children are vaccinated against
Leukemia	Syncope	Red stool
Ataxia	Biliary atresia	Pediatric dosages for polypharmaceutical ingestions in adolescents
Discrimination between MSK patients who need urgent consult vs. those who can be referred to outpatient care	Recognition of how pediatric emergencies provoke higher levels of anxiety among ED physicians	Pediatric devices–insulin pump
Common problems of NICU graduates–necrotizing enterocolitis: medical and surgical	Environmental emergencies–heat stroke/heat exhaustion	Psychosocial differences of pediatric patients based on developmental milestones
General administrative, legal, and ethical issues involved with treating children in an ED	Environmental emergencies–hypothermia	Manage special‐needs children–autism
Recognize and manage–Henoch‐Schonlein purpura	Stabilization involved with common pediatric overdose/poisoning–propofol	Pediatric dosing for amoxicillin (high dose)
EMS transport of children	Pediatric dosages for ibuprofen
Clinical Skills Topics
Incision and drainage of abscess	Foreign‐body removal from–soft tissue	Wound management
Reduction of paraphimosis	Diagnostic US–FAST scan	Complete eye examination (including slit‐lamp examination)
Pericardiocentesis	Chest tube placement on young children	Nasal packing
Interpretation of radiographs of MSK	Anticipatory guidance to parents	Nasal agents–fentanyl/versed
Delivering bad news	Install umbilical artery or vein catheters	Application of strategies for performing accurate PE on a difficult child
External cardiac pacing	Gastrostomy tube replacement	Chest tube placement on infants
Effective communication with consultants	Invasive airway rescue options‐transtracheal jet	Interpretation of radiographs of soft tissue neck

MSK = musculoskeletal and PE = physical examination.

Recommended Experiences

We asked panelists to provide experiences residents needed to learn to care for children. Their responses were classified into experience with children who have specific illness presentations; off‐service rotations; clinical or simulated experiences; and types of facilities, time allocation, and other miscellaneous suggestions. These experiences were rated during round 2, and because there was agreement among panelists, we did not ask about experiences during round 3.

The panel recommended that residents need to experience children of all age levels (newborns to adolescents) and all levels of acuity, including common, nonemergent conditions. They went on to recommend experience with specific patient presentations such as neonatal fever, septic shock, asthma, epilepsy, diabetic ketoacidosis, hematology/oncology patients with fever or neutropenia, and sickle cell disease. They also recommended specific sets of skill that should be covered.

In addition to receiving pediatric training from both EM and pediatric faculty, the panelists felt it important that residents also receive training from PEM‐boarded physicians and that skills training incorporate simulation. They emphasized specific skill sets that needed to be taught and practiced. These included airway management and medical and trauma resuscitation. Finally, the panelists suggested that all EM residents do ancillary rotations on pediatric specialty units like intensive care (pediatric intensive care unit), anesthesiology, and orthopedics (see Table 5).

Table 5.

Results from Delphi Rounds 1 and 2 on Resident Experiences for Learning How to Care for Pediatric Patients

Recommended Experiences (Nominations)*		Mean†	Strength Score‡	Pct. Endorsed§	Rank||	Status¶
Experience managing children with specific presentations or diseases
1.	Neonatal fever (1)	5.00	65	100	1.17	Must
2.	Septic shock (1)	5.00	65	100	1.17	Must
3.	Severe asthma (1)	5.00	65	100	1.17	Must
4.	Severe status epileptics (1)	5.00	65	100	1.17	Must
5.	DKA (1)	4.92	64	92.3	5	Must
6.	Hematology/oncology patients with fever and/or neutropenia (1)	4.69	61	69.2	6	Must
7.	Sickle cell disease (1)	4.31	56	38.5	7	Optional
8.	Chest pain (1)	3.77	49	23.1	8	Optional
Off‐service or ancillary clinical rotations: dedicated 1‐month clinical rotations on or with
1.	PICU (7)	4.92	64	92.3	1	Must
2.	Pediatric anesthesiology (2)	4.46	58	61.5	2	Must
3.	Pediatric orthopedics (3)	4.15	54	53.8	3	Must
4.	Pediatric morbidity and mortality cases by EM residents at educational conference (1)	3.92	51	38.5	4	Optional
5.	Child abuse response team (1)	3.15	41	7.7	5	Optional
6.	NICU (5)	3.25	40	7.7	6.5	Optional
7.	Inpatient pediatrics (1)	3.00	39	7.7	6.5	Optional
8.	Outpatient pediatrics (1)	4.00	34	0.0	7	Optional
Experience with dedicated topics of skills
1.	Airway experience (2)	5.00	65	100	1.5	Must
2.	Medical resuscitation (3)	5.00	65	100	1.5	Must
3.	Trauma resuscitation (3)	4.92	64	92.3	3	Must
4.	Neonatal resuscitation (2)	4.77	62	84.6	4	Must
5.	Establishing a comprehensive differential diagnosis (1)	4.38	57	46.2	5	Must
6.	History taking skills (1)	4.23	55	38.5	6.5	Must
7.	Physical examination (2)	4.23	55	38.5	6.5	Must
8.	Pain management (1)	4.15	54	30.8	8	Must
9.	Ordering labs and studies (1)	4.00	52	23.1	9	Must
10.	Learning techniques for distracting children (1)	3.62	47	15.4	10	Optional
Case mix
1.	Need to see patients in entire spectrum of ages (newborns to adolescents) (3)	4.77	62	76.9	1	Must
2.	Experience treating children across all levels of acuity (6)	4.69	61	76.9	2	Must
3.	Experience treating children for both common, nonemergent conditions and medical emergencies (1)	4.62	60	69.2	3	Must
Time allocation: PED rotations designed as follows
1.	Shifts or rotations scheduled different seasons to experience seasonal variation in illness	4.33	53	46.2	1	Must
2.	A minimum of 2–3 months working in a PED with PEM physicians	3.83	47	38.5	2	Optional
3.	A minimum of a 2‐month block of shifts for each year of residency	3.83	47	23.1	3	Optional
4.	Four or five shifts per month in the PED	3.83	47	15.4	4	Optional
5.	3‐month clinical rotations on PEM	3.58	44	15.4	5	Optional
6.	Equivalent numbers of shifts in a PED as residents would have in the adult ED	2.58	32	7.7	6	Optional
Experience at specific types of facilities
1.	Experience at a PED that serves a large population of pediatric patients (1)	4.46	58	69.2	1	Must
2.	Experience at a PED that is at least a Level II trauma center (1)	4.31	56	61.5	2	Must
Special certification courses
1.	Pediatric Advanced Life Support (1)	4.46	58	61.5	1	Must
2.	Neonatal resuscitation program (1)	3.85	50	30.8	2	Optional
Including additional subspecialists in training EM residents
1.	PEM‐boarded physicians (1)	4.31	56	69.2	1	Must
2.	SANE nurses (1)	2.85	37	15.4	2	Optional
Scope of training
1.	Pediatric rotations at each level of training (PGY1–3) with emphasis on building skills to attain mastery (1)	4.23	55	46.2	1	Must
2.	Competence at running an area or managing all pediatric patients who come through by senior year (1)	3.92	51	23.1	2	Optional
Miscellaneous experiences
1.	Simulation: practice pediatric specific skills through simulation (3)	4.54	59	53.8		Must
2.	Procedure heavy shifts so residents become adept at core procedural skills like lumbar puncture, incision and drainage, suture repair (3)	3.69	48	30.8		Optional

DKA = diabetic ketoacidosis; NICU = neonatal intensive care unit; PED = pediatric emergency department; PEM = pediatric emergency medicine; PICU = pediatric intensive care unit; SANE = sexual assault nurse examiner.

*Nominations = the frequency of times that item was suggested (nominated) during round 1.

†Mean = mean rating of items from round 2 from a Likert‐type scale labeled 5 = very important, 4 = considerable importance, 3 = moderate importance, 2 = minimal importance, 1 = not at all important.

‡Strength score = the sum of weighted frequencies (total points) resulting from multiplying the number of participants selecting a rating (frequency of occurrence) by the value of the rating from the Likert‐type scale.21

§Pct. Endorsed = the percentage of panelists out of 13 from round 2 and 12 from round 3 who endorsed the item by selecting the highest rating: “very important” from round 2 and “must teach” from round 3.

||Rank = the rank assigned to items based on the rank order of their strength score and percentage of panelists endorsing that item with the highest rating.

¶Status = recommendations from the panel: must = highly recommended experiences; optional = may be offered.

We asked panelists to estimate the proportion of curriculum time allocated to pediatric topics (see Table 2). The average percentage of learning time that programs dedicate to pediatric topics and experiences was about 20%. Participants said that within this 20%, the breakdown of educational experiences was 75% in the clinical environment and 25% in formal teaching sessions, i.e., didactics, small groups, simulations, or procedures training. With regard to how clinical time for pediatrics was allocated, the panelists had two specific recommendations. First, panelists highly recommended that learners experience pediatrics across all seasons so that they see the full spectrum of seasonal variation of illness. They also suggested that it was very important not to package clinical pediatric experiences into one level of education (program year) or into experiences within only one setting. In other words, panelists recommended that pediatric experiences be longitudinal across the entire residency program and that they experience care in places that treat high‐acuity as well as low‐acuity patients. They particularly emphasized a need to experience a pediatric ED that serves a large population of patients.

Literature Resources

We asked the expert panel to provide the literature they used to guide their work. Many of them mentioned their involvement with developing the EM pediatric curriculum for their own institution. Those individuals cited their institution's curriculum documents as a resource and suggested that they had used the ACGME program requirements,6 The EM Milestone Project26 and the content outline for the PEM subspecialty examinations.27 Also cited were two journal articles14, 17 and two textbooks, one on clinical procedures28 and one dedicated to PEM.29

Discussion

This curriculum project aimed to create a standard pediatric curriculum that EM program leaders can use to develop their own custom curriculum. This work is intended to supplement the EM Model of Clinical Practice with an additional level of granularity and focus on important pediatric related content. Variable clinical training environments inevitably lead to variability in learner education. A core consensus curriculum will assist educators in prioritizing the requisite pediatric content for an already dense EM curriculum. The core pediatric curriculum also enables training programs to critically evaluate their clinical environment and assess deficiencies in their current training programs. We hope that the results of this effort lays the foundation for subsequent efforts to develop competency‐based education covering pediatric content for EM residents.

We have categorized knowledge topics and clinical skills into recommended, optional, and unnecessary so that program leaders can integrate our standards with topics important to their circumstances to build the curriculum that best suits their needs. Our core consensus curriculum is applicable to learners taught by both EM and PEM faculty members. Although this curriculum was specifically designed for EM residents, there are likely components applicable to anyone caring for pediatric emergencies including pediatric residents and PEM fellows.

We intentionally did not address how to teach these topics as this will vary widely based on patient populations, resources, expert availability, and institutional practice. Panelists did, however, recommend the experiences they think residents need to achieve the knowledge and skills derived from this project. Future work should be performed to develop best practices for delivering this core content material, generating related competencies, and developing assessments for measuring competency achievement.

The lists we created are substantial and may be daunting upon first review. There was significant variability in the list of topics generated by panelists during the initial round of the modified Delphi. Although the final recommendations did not reach complete consensus, we feel that the final product is a step toward reducing the variability in pediatric education that currently exists throughout EM programs throughout the United States.

We also captured some disagreement, due to recent paradigm shifts in patient care that are represented by some of the topics that unexpectedly ended up on the dropped items list. For example, the last curriculum document took place before our current vaccination policies were in place.14 As a result, our standard curriculum outline contains far fewer items related to vaccine‐preventable illnesses than do previous curricula. Medication doses were another area of major shift. Generations of EM providers have memorized lifesaving dosages of acute resuscitation medications; however, many of our educators felt that in the current day of electronic resources, memorizing dosages is no longer necessary.

Through the use of the modified Delphi method solely through electronic communications, we were able to generate a standard consensus curriculum in a timely manner with limited expenditure of resources for travel and meeting facilities. The asynchronous participation yielded nearly 100% participation across all phases of the study. In retrospect, we believe one in‐person or electronically supported live meeting (such as a webinar or Skype meeting) to engage participants in more deliberate conversation about the curriculum topics as they were evolving would have been beneficial.

Limitations

The panelists generated and prioritized a large amount of content material. We did not receive any complaints; however, the possibility exists that fatigue was involved during the modified Delphi process. Additionally, the entire modified Delphi was completed through electronic communication. The lack of at least one face‐to‐face meeting may have contributed to the lack of consensus and wider variability in responses.

The study was limited by the number of individual experts we were able to involve. Front‐line experts with experience in both pediatric care and resident education were recruited from a cross‐section of training site types around the United States. The size of our panel was based on the quantity of learning material we anticipated receiving and on suggestions from the literature.20, 21 Consequently, we consider this to be a preliminary step toward drafting a core pediatric curriculum for EM residents and plan subsequent investigations to account for regional and demographic variation. Furthermore, we have merely provided the content outline and recommendations for experiences required to cover this content, leaving the task of instructional design up to individual programs.

Our basic objective was to provide a consensus curriculum outline for preparing EM physicians to treat children in the acute care setting. The panel represented considerable career expertise in EM and PEM. We believe that the resulting curriculum is slightly more ambitious or dense than can be easily covered in a 3‐year program. Accordingly, we have provided as much guidance as possible to help program leaders to prioritize topic coverage from most to least important.

Conclusions

The materials that accompany this article provide the basic structure and content for teaching EM residents about caring for the special population of children in the acute care setting. While the panel generally believed that there is some transference of skills and knowledge that is gleaned from experience with adult patients, the curriculum content and experiences presented here are considered to be most important for learning the nuances of caring for children.

Supporting information

Data Supplement S1. Pediatric emergency medicine curriculum for EM residents: complete results and statistics for all three Delphi rounds sorted by final ranking.

AEM Education and Training 2017;1:140–150