Design of a point‐of‐care ultrasound curriculum for pediatric emergency medicine fellows: A Delphi study

Abstract

Objectives

There has been a steady increase in the growth and utilization of point‐of‐care ultrasound (POCUS) in pediatric emergency medicine (PEM). POCUS has been established as an Accreditation Council for Graduate Medical Education (ACGME) core requirement for accreditation of PEM fellowship programs. Despite this requirement, training guidelines regarding POCUS knowledge and skills have yet to be developed. The purpose of this project was to develop a curriculum and a competency checklist for PEM fellow POCUS education.

Methods

We formed a core leadership group based on expertise in one or more key areas: PEM, POCUS, curriculum development, or Delphi methods. We recruited 29 PEM POCUS or ultrasound education experts from North America to participate in a three‐round electronic Delphi project. The first Delphi round asked experts to generate a list of the core POCUS knowledge and skills that a PEM fellow would need during training to function as an autonomous practitioner. Subsequent rounds prioritized the list of knowledge and skills, and the core leadership group organized knowledge and skills into global competencies and subcompetencies.

Results

The first Delphi round yielded 61 POCUS areas of knowledge and skills considered important for PEM fellow learning. After two subsequent Delphi rounds, the list of POCUS knowledge and skills was narrowed to 38 items that addressed elements of six global competencies. The core leadership group then revised items into subcompetencies and categorized them under global competencies, developing a curriculum that defined the scope (depth of content) and sequence (order of teaching) of these POCUS knowledge and skill items.

Conclusions

This expert, consensus‐generated POCUS curriculum provides detailed guidance for PEM fellowships to incorporate POCUS education into their programs. Our curriculum also identifies core ultrasound knowledge and skills needed by PEM fellows to perform the specific POCUS applications recommended in prior publications.

Tweet.

Experts in POCUS identified 38 specific areas of knowledge and skills needed for PEM fellows to incorporate POCUS into their practice.

INTRODUCTION

Point‐of‐care ultrasound (POCUS) is a well‐established clinical supplement to the practice of emergency medicine (EM). ¹  The American College of Emergency Physicians (ACEP) policy statement on POCUS, originally published in 2001, includes training recommendations for EM residents. ² Since 2012, the American Council on Graduate Medical Education (ACGME) has included POCUS as a core requirement and defined specific POCUS milestones as part of EM residency training. ³ , ⁴

In contrast, pediatric emergency medicine (PEM) has been slower to adopt POCUS for patient care. The American Academy of Pediatrics (AAP) generated a broad policy statement on POCUS for PEM physicians in 2015, when the American Board of Pediatrics included POCUS in the content specifications for the PEM subspeciality certification examination. ⁵ , ⁶ In 2020, the ACGME designated competence in POCUS as a core requirement for PEM fellowship graduates. ⁷ As emerging literature supports the use of POCUS in the pediatric emergency department (PED), PEM fellowships have gradually begun to incorporate POCUS education into training. ⁸ , ⁹ , ¹⁰

With increased integration of POCUS into PEM practice, POCUS‐specific competencies and learning objectives for PEM trainees must be defined. In 2013, consensus PEM POCUS education guidelines and a model curriculum were published, but these recommendations were heavily based on ACEP guidelines and not tailored specifically to the pediatric population. ¹¹  More recently, Shefrin et al. ¹² used the modified Delphi method to compile a set of core POCUS applications to include in PEM fellow curricula. This study focused on specific ultrasound applications and omitted the associated knowledge and skills required to operate an ultrasound machine at the bedside, such as ultrasound physics, operation of the ultrasound machine (knobology), image acquisition and interpretation, clinical integration, and quality assurance. Few pediatric residency programs in the United States provide POCUS education to prepare residents for PEM fellowships. ¹³  Therefore, despite the development of a strong framework of core POCUS applications, a carefully crafted curriculum consisting of specific ultrasonography knowledge and skills would be beneficial to PEM learners and educators alike.

While prior studies have laid the groundwork for adoption of POCUS education into PEM fellowships, a comprehensive catalog of specific global competencies, subcompetencies, knowledge, and skills that PEM trainees need to incorporate POCUS into their clinical practice has yet to be developed. A standard POCUS curriculum would provide PEM fellowship educators with a template for POCUS education that, if properly implemented, would culminate in the graduation of PEM physicians capable of effectively integrating POCUS into their care models.

The primary purpose of this project was to build upon the work of prior studies with the goal of defining the ideal PEM POCUS curriculum. ¹¹ , ¹² Using a modified Delphi method, we developed a comprehensive PEM fellow POCUS curriculum that included both fundamental and advanced competencies needed to perform the ultrasound applications identified by Shefrin and colleagues. A secondary purpose was to define suggested scope (breadth of knowledge) and sequence (order of learning) of PEM fellow POCUS education. Finally, from the global competencies and subcompetencies identified in this project, we developed a competency‐based assessment tool used to assist in the measurement and assessment of a PEM fellow's progress toward attainment of the recommended POCUS knowledge and skills.

METHODS

Participants

A team of seven physicians, nationally recognized for their scholarship or leadership roles in PEM or PEM POCUS education, served as the core leadership group for this project. This team developed criteria for Delphi panelist selection, synthesized and reported data to panelists after each Delphi round, and determined when the Delphi process was completed. From a curriculum standpoint, the core leadership group categorized data and processed panelist contributions into a structured format to create the final curriculum and checklist. Delphi panelists were selected based on their expertise in POCUS, fellowship education, ultrasound education and leadership in affiliated academic societies. These panelists were responsible for generating the list of knowledge and skills required of PEM fellows to become proficient POCUS practitioners.

Study design

The Delphi is a method for structuring communication into an effective process which allows a group of content experts to confront and solve complex problems. ¹⁴ , ¹⁵ , ¹⁶ , ¹⁷ For our study, we modified the conventional Delphi by communicating electronically with email and electronic survey platforms. ¹⁴  To capitalize on collective expertise, members of the core leadership group also served as participants on the Delphi panel. Our specific Delphi design was modeled from those recommended by Witkin and Altschuld ¹⁸ and consisted of content generated by panelists, a moderate‐sized group with representation from professionals with specialized knowledge (experts), up to four iterative rounds, and anonymity of panelists’ contributions. A priori, and consistent with prior Delphi studies, we deemed a minimum threshold of 85% agreement to retain items across Delphi rounds. ¹⁴ , ¹⁶ , ¹⁹  This study was deemed exempt by the institutional review board at Nationwide Children's Hospital.

The Delphi was conducted from January through September 2019. The first Delphi round was a survey that included background information leading to the project's purpose statement (Data Supplement S1, Appendix S1, available as supporting information in the online version of this paper, which is available at http://onlinelibrary.wiley.com/doi/10.1002/aet2.10700/full). We asked open‐ended questions designed to capture the panel's collective wisdom regarding which core knowledge and skills PEM fellows need to competently adopt POCUS into pediatric patient care. Panelists were encouraged to respond with free‐text descriptions and to include pertinent references with their contributions.

Round one panelist responses were then organized through thematic analysis into discrete categories that included both overarching global competencies and more detailed knowledge and skills. The core leadership group worked together to further interpret and organize those items for presentation to the entire panel. First, they eliminated items that referred to specific ultrasound applications such as “identify peritoneal free fluid in trauma” or “perform a focused assessment with sonography for trauma (FAST) scan,” because this information had been previously published and did not specify the knowledge or skills required for competent performance. ¹²  The core leadership team identified eight global POCUS competency categories among the remaining content. They also grouped similar remaining knowledge and skills or split items containing disparate content. A flowchart describing the Delphi rounds is provided (Figure 1).

FIGURE 1.

Flow diagram of the Delphi process

Panelists were asked to engage in two activities in the second Delphi round. First, they were asked to clarify or refine the proposed knowledge and skill items from round one by rating each topic as “important,” “not important,” or “needs changed.” When a panelist selected the last option, a proposed change was required in an adjacent free‐text box. Second, panelists were asked to rate the utility of the eight “global POCUS competencies” for categorizing discrete knowledge and skills items. In later rounds, these “global competencies” were used to create subcompetencies (Appendix S2).

After observing substantial disagreement during round two, the third and final Delphi round was deemed necessary for building consensus. For round three, the survey was divided into four sections: items that reached consensus (and therefore needed no further deliberation; n = 20), items that a third of the panelists rated as being more appropriate for a POCUS fellow than a PEM fellow (n = 12), items requiring agreement of modifications proposed during round two (n = 14), and items with substantial disagreement (n = 6). Panelists were asked to complete three tasks:

• 1) Rerate items panelists tagged as appropriate for a POCUS fellow, but inappropriate for a PEM fellow. Panelists selected from three options: “agree—NOT appropriate for PEM,” “disagree—appropriate for PEM,” or “outside my area of expertise.”

• 2) “Approve” the wording modifications panelists proposed during round two using the response options “agree” or “disagree.”

• 3) Rerate a third set of items that had not yet reached consensus. For the last task we used the original rating options “important” or “not important” (Appendix S3).

Upon completion of the Delphi rounds, the core leadership group organized items for which consensus was reached into global competencies and subcompetencies. They also organized global competencies and subcompetencies into a recommended curriculum by providing the sequence in which subcompetencies should be taught. For example, basic foundational knowledge and skills are introduced early in the PEM fellowship, with the gradual addition of more intermediate and advanced knowledge and skills later in the fellowship. Finally, to make these materials more practical for PEM fellowship program directors, the core leadership team also organized the final items into a checklist to be used to track fellows’ progress through the PEM POCUS curriculum.

Data collection and analysis

We used SurveyMonkey to distribute surveys and collect responses. Descriptive statistics (means, standard deviations, frequencies, and proportions) were used to summarize data collected during each Delphi round. All data analyses were performed with IBM‐SPSS Version 24.

RESULTS

The final Delphi panel of 29 included the seven core leadership team members and the additional 22 selected individuals with expertise in one or more of the following domains: POCUS, clinical practice of PEM, PEM education, fellowship program administration, leadership in academic societies with interest in PEM POCUS education, and/or experience in developing a curriculum within a competency‐based medical education framework (Table 1).

TABLE 1.

Demographic profile (frequencies and percentages) of 29 Delphi panelists

	N (%)
Total number of participants	29 (100)
Sex
Male	7 (24)
Female	22 (76)
Practice
Adult EM	6 (21)
Both EM/PEM	2 (7)
Pediatrics/PEM	21 (72)
Location
Canada	3 (10)
Ontario	3 (10)
United States	26 (90)
Northeast Region	7 (24)
Midwest Region	5 (17)
South Region	8 (26)
West Region	6 (21)
Title
Fellowship director	11 (38)
PEM	2 (7)
Emergency US (traditional)	3 (10)
Emergency US (pediatric)	6 (21)
Medical or unit director	18 (62)
EM US (adult or mixed‐service hospital)	6 (21)
EM US (children's hospital)	9 (31)
0PEM (children's hospital)	1 (3)
Residency program
Pediatrics	19 (66)
EM	10 (34)
Fellowship program
Both PEM and US	18 (62)
PEM	5 (17)
Ultrasound	6 (21)

Abbreviations: PEM, pediatric emergency medicine; US, ultrasound.

Round one

The first Delphi round continued for 3 weeks. Weekly email reminders were sent to encourage panelists’ completion of tasks. All 29 panelists (100%) completed responses to the round one questionnaire. Open‐ended questions about “core knowledge and skills” were distilled and organized into knowledge and skill items. After deliberation on these items and elimination of specific clinical ultrasound applications, the core leadership group identified eight global competency categories, 37 specific core knowledge items, and 24 specific skill items.

Round two

In round two, 29 of the 29 panelists (100%) completed responses to the questionnaire. Panelists reached the 85% agreement threshold for importance on six of eight global POCUS competencies and 23 of 61 core knowledge and skills items (subcompetencies). These items are listed as “retained” in Delphi round two on Table 2. Panelists rated 11 subcompetencies as “important,” but proposed revisions for clarification. These are listed in Table 2 with their original wording and proposed revisions. Nine items were discarded by the conclusion of round two (consolidating 10 items into four and eliminating three items thought to be redundant). These are listed as “discarded” or “combined” (Table 2).

TABLE 2.

Results from Delphi rounds two and three: mean ratings and number (and percentage) of panelists who endorsed the contributions made by POCUS experts regarding the required POCUS education content for PEM fellowships

Recommended competencies/subcompetencies		Mean a	N (%) endorsed b	Delphi round in which final status was determined	Final status
US physics
1.	Understands and is able to explain the theoretical safety risks involving the use of US	1.07	26 (89.7)	2	Retained
2a.	Uses color Doppler to identify neighboring vessels and absence of flow and to distinguish between arteries and veins	1.00	27 (93.1)	2	Revised
2b.	Identifies clinical uses for and principles of color Doppler (N = 28)	1.11	25 (89.3)	3	Retained
3.	Understands the relationship between imaging planes of the body and images on the screen	1.03	28 (96.6)	2	Retained
4.	Understands the relationship between frequency, penetration, and resolution	1.03	28 (96.6)	2	Retained
5.	Explains the physics of US wave generation	1.29	20 (69.0)	3	Discarded
6.	Understands the principle of pulse echo	1.39	17 (58.6)	3	Discarded
7.	Defines an US artifact	1.00	28 (96.6)	2	Retained
8.	Characterizes common US artifacts like shadowing, enhancement, mirror images, etc., that interfere with scan interpretation	1.00	29 (100)	2	Retained
Knobology (combined with US physics)
1.	Explains "depth" and how to adjust the US machine depth settings	1.00	28 (96.6)	2	Retained
2.	Explains "gain" and how to adjust the US machine gain settings	1.00	29 (100)	2	Retained
3.	Explains "resolution" and how to adjust the US machine resolution settings	1.16	21 (72.4)	3	Combined with #10
4.	Explain “contrast” and how to adjust the US machine contrast settings			1	Discarded
5.	Explains "color Doppler" and how to adjust the US machine color Doppler settings	1.00	29 (100)	2	Retained
6.	Explains "spectral Doppler" and how to adjust the US machine spectral Doppler settings	1.32	19 (65.5)	3	Discarded
7.	Explains "M‐mode" and how to adjust the US machine M‐mode settings	1.03	28 (96.6)	2	Retained
8.	Explains "B‐mode" and how to adjust the US machine B‐mode settings	1.11	24 (82.8)	3	Combined with #10
9.	Explains the different examination presets for each type of US examination	1.38	15 (51.7)	3	Combined with #10
10a.	Explains how to adjust machine settings for optimization of the image	1.04	25 (86.2)	2	Revised
10b.	Demonstrates how to adjust machine settings and presets for image optimization (N = 28)	1.04	27 (96.4)	3	Retained
11.	Identifies areas of interest and understand focal zones	1.14	24 (82.8)	3	Combined with #10
12.	Demonstrates proper cleaning and handling of US machine with attention to infection control standards	1.00	28 (96.6)	2	Retained
13.	Effectively adapts from one machine to another regardless of brand or vendor	1.21	22 (75.9)	3	Discarded
Image acquisition and interpretation
1.	Selects the correct transducer for specific US examinations	1.00	29 (100)	2	Retained
2.	Recognizes normal sonographic anatomy for specific US examinations	1.00	29 (100)	2	Retained
3.	Discriminates between normal anatomy and pathology on US	1.00	28 (96.6)	2	Retained
4.	Manipulates the transducer properly to obtain the desired image	1.00	27 (93.1)	3	Retained
5.	Obtains US images that are of sufficient quality for medical decision making and billing	1.00	29 (100)	2	Retained
6.	Is mindful when positioning patient and is considerate of their comfort	1.00	28 (96.6)	2	Combined with #7
7a.	Properly applies towels, gowns, and sheets for draping with consideration for patient modesty	1.03	28 (96.6)	2	Revised
7b.	Demonstrates consideration for patient comfort when positioning, draping, or applying gel	1.14	25 (86.2)	3	Retained
8a.	Properly applies US gel with attention to patient comfort and sufficient amounts for structure visualization	1.14	25 (86.2)	2	Revise
8b.	Demonstrates proper positioning of patient and application of gel and draping of materials for image optimization (N = 28)	1.11	25 (89.3)	3	Retained
9.	Acquires and saves appropriate and complete still images and video clips	1.00	28 (96.6)	2	Combined with #11
10.	Identifies the appropriate anatomic structures	1.00	28 (96.6)	3	Discarded
11a.	Acquires the necessary views for a complete/adequate US examination	1.00	28 (96.6)	2	Revised
11b.	Acquires the necessary views for an US examination and saves appropriate still and video images (N = 28)	1.07	26 (92.9)	3	Retained
12.	Labels images accurately	1.07	27 (93.1)	3	Retained
13.	Accurately documents POCUS image findings, including the necessary components needed for billing	1.00	29 (100)	2	Retained
14.	Effectively integrates knowledge of US physics with knowledge of human anatomy and physiology to apply this knowledge to the acquisition of practical US images	1.08	23 (79.3)	3	Discarded
POCUS for medical decision making
1.	Describes the clinical indications for using specific US studies	1.00	29 (100)	2	Retained
2.	Identifies the contraindications for specific US studies	1.07	27 (93.1)	2	Retained
3.	Properly integrates POCUS studies into medical decision making	1.03	28 (96.6)	2	Retained
4.	Describes the “pearls” (advantages) and “pitfalls” (cautions) when using US across specific applications	1.00	29 (100)	2	Retained
5a.	Defines the sensitivity, specificity, positive and negative likelihood ratios, and pretest probabilities for each specific application	1.21	19 (65.5)	2	Revised
5b.	Has awareness of the test characteristics of published data for specific US indications, e.g., sensitivity, specificity	1.03	28 (96.6)	3	Retained
6.	Identifies which US study to obtain based on patient presentation	1.07	27 (93.1)	3	Retained
7.	Accurately interprets the US image	1.00	27 (100)	2	Retained
8.	Translates the information gleaned from the US examination into the clinical examination for medical decision making	1.14	27 (93.1)	2	Discarded
Ethics, communication and professionalism (combined with administration & literature)
1a.	Effectively communicates the distinction between POCUS and radiology US to patients/families	1.00	27 (93.1)	2	Revised
1b.	Effectively communicates the distinction between POCUS and comprehensive US to patients/families	NA	NA	3	Retained
2.	Appropriately informs the patient and family as to the indications for and benefits/risks of a POCUS examination	1.00	28 (96.6)	2	Retained
3a.	Communicates US results with time efficiency and clarity to patients, families, and the primary medical team	1.00	26 (89.7)	2	Revised
3b.	Clearly and effectively communicates US results to patients, families, and the medical team	1.14	25 (86.2)	3	Retained
Administration of a POCUS program
1.	Outlines the proper considerations necessary for effective administration of a POCUS program	1.44	15 (51.7)	3	Discarded
2.	Understands and describes the roles and responsibilities of a POCUS director	1.36	18 (62.1)	3	Discarded
3a.	Recognizes the value of and advocate for a QA database for US examinations	1.14	25 (86.2)	2	Revised
3b.	Recognizes the value of a QA database for US examinations	1.17	24 (82.8)	3	Retained
4a.	Establishes pediatric emergency department policies that effectively detail the medical use of POCUS for diagnostic reasoning and guided procedures	1.33	18 (62.1)	2	Revised
4b.	Describes pediatric ED policies that effectively detail the medical use of POCUS for diagnostic reasoning and guided procedures	1.21	23 (79.3)	3	Discarded
5a.	Develops a standardized process for properly credentialing providers in US	1.48	13 (44.8)	2	Revised
5b.	Recognizes the need for a standardized process for properly credentialing providers in US	1.14	25 (86.2)	3	Retained
6.	Determines the role of an effective multidisciplinary POCUS program and the ability and importance of collaboration across divisions/departments to share expertise, resources, and responsibilities	1.38	16 (55.2)	3	Discarded
7.	Understands pathways to achieving and demonstrating proficiency in POCUS	1.17	24 (82.8)	3	Discarded
8.	Establishes methods for assessing provider's POCUS proficiency for credentialing	1.42	15 (51.7)	3	Discarded
9.	Recognizes the differences between ED POCUS and radiology US	1.04	26 (89.7)	2	Retained
10.	Develops a plan and monitoring system for proper and timely image review	1.41	16 (55.2)	3	Discarded
11.	Contributes to the good working order of US machines	1.04	27 (93.1)	3	Discarded
Literature, scholarship, education
1.	Has a working understanding of the basic relevant pediatric POCUS literature	1.00	29 (100)	2	Retained
2.	Demonstrates effective teaching of POCUS concepts and skills to other learners	1.11	24 (82.8)	3	Retained
3.	Design a curriculum for training learners up to mastery level performance in the pediatric ED	1.61	9 (31.0)	3	Discarded
4.	Complete of a set number of scans in training	1.08	23 (79.3)	3	Discarded
5a.	Describe the ACEP emergency medicine US guidelines	1.48	11 (37.9)	2	Revised
5b.	Describe the AAP POCUS guidelines for PEM physicians 5	1.14	25 (86.2)	3	Retained
Global competencies
1.	Demonstrates fundamental knowledge of US physics	1.10	26 (89.7)	3	Combined with #2
2a.	Demonstrates accurate technique for POCUS image acquisition (knobology)	1.00	29 (100)	2	Revised
2b.	Demonstrates knowledge of US physics and knobology	1.00	29 (100)	3	Retained
3.	Differentiates between normal and abnormal anatomy and physiology on US imaging	1.00	29 (100)	3	Combined with #4
4a.	Accurately interprets POCUS images	1.00	29 (100)	2	Revised
4b.	Image acquisition and interpretation: uses proper technique to acquire image; accurately interprets POCUS images; differentiates between normal and abnormal anatomy and physiology on US imaging	1.00	29 (100)	3	Retained
5.	Effectively utilizes POCUS in the medical decision making process	1.00	29 (100)	2	Retained
6.	Knowledge of POCUS indications, pearls, and pitfalls; utilizes POCUS in appropriate clinical scenarios (ethics, administration, communication, and professionalism)	1.00	28 (96.6)	2	Retained
7.	Demonstrates fundamental knowledge necessary for administration of a POCUS program	1.46	15 (51.7)	3	Combined with #6
8.	Engages in scholarly activities related to POCUS (literature, scholarship, and education)	1.46	14 (48.3)	3	Combined with #6

Contributions are presented as competencies and associated subcompetencies. The subcompetencies that were revised then retained and those that were discarded are indicated as such.

Abbreviations: AAP, American Academy of Pediatrics; ACEP, American College of Emergency Physicians; PEM, pediatric emergency medicine; POCUS, point‐of‐care ultrasound; QA, quality assurance; US, ultrasound.

^a Mean rating after round two from two‐point response option scale: 1 = important, 2 = not important. Items with mean ratings closer to “1.0” were considered the most important. Items with ratings closer to “2” were considered not important.

^b Number and percentage of panelists who endorsed the item as important during the round in which that item was retained.

Of note, during round two we observed substantial disagreements between the general EM and PEM POCUS experts related to specific item importance in the PEM fellow curriculum. EM POCUS experts rated items related to POCUS program administration, research and scholarship as important, while PEM POCUS experts rated these as unimportant. PEM POCUS experts further clarified that these items were more appropriate for POCUS fellowship programs designed for fellows’ academic career development. In addition, they suggested that fitting these topics into a PEM fellowship curriculum would be challenging. All panelists were then informed of frequencies, percentages, and comments related to items selected by the round two panelists and again asked to vote on items that received less than 85% agreement.

Round three

The response rate for this round was also 100% (29/29). The round three survey provided feedback to panelists on the performance of competencies and items from round two. During this round there were nine items approved with rewording that had been proposed during round two. There were also nine items discarded because they were considered appropriate for a POCUS fellow, but not appropriate for a PEM fellow. Finally, an additional five items were discarded because they did not reach the consensus threshold of 85%. The 38 items and six competencies retained after round three are listed in their final approved wording in Table 3.

TABLE 3.

Final product of the Delphi panel: a list of educational subcompetencies and competencies for teaching POCUS to PEM fellows

Specific knowledge, skills, and mindsets
CODE	Final wording
USP01	Understand and be able to explain the theoretical safety risks involving the use of US
USP02	Identify uses for, and principles of, color Doppler
USP03	Understand the relationship between imaging planes of the body and images on the screen
USP04	Understand the relationship between frequency, penetration, and resolution
USP07	Define an US artifact
USP08	Recognize common US artifacts like shadowing, enhancement, mirror images. etc., that interfere with scan interpretation
Knob1	Explain "depth" and how to adjust the US machine depth settings
Knob2	Explain "gain" and how to adjust the US machine gain settings
Knob4	Explain "color Doppler" and how to adjust the US machine color Doppler settings
Knob6	Explain "M‐mode" and how to adjust the US machine M‐mode settings
Knob9	Explain how to adjust machine settings and presets for image optimization
Knob11	Implement proper handling, cleaning, and maintenance of US machines
MDM1	Describe the clinical indications for using specific US studies
MDM2	Identify the contraindications for specific US studies
MDM3	Properly integrate POCUS studies into medical decision making
MDM4	Describe the “pearls” (advantages) and “pitfalls” (cautions) when using US across specific applications
MDM5	Have awareness of the test characteristics of published data for specific US indications, e.g. sensitivity, specificity.
MDM6	Identify which US study to obtain based on patient presentation
MDM7	Accurately interpret the US image
AQIN1	Select the correct transducer for specific US examinations
AQIN2	Recognize normal sonographic anatomy for specific US examinations
AQIN3	Discriminate between normal anatomy and pathology on US
AQIN4	Manipulate the transducer properly to obtain the desired image
AQIN5	Obtain US images that are of sufficient quality for medical decision making and billing
AQIN6	Demonstrates consideration for patient comfort when positioning, draping, or applying gel
AQIN7	Demonstrate proper positioning of patient, application of gel, and draping of materials for image optimization
AQIN11	Acquire the necessary views for an US examination and save appropriate still and video images
AQIN12	Label images accurately
AQIN13	Accurately document POCUS image findings, including the necessary components needed for billing
ETPR1	Effectively communicate the distinction between POCUS and radiology US to patients/families
ETPR2	Appropriately inform the patient and family as to the indications for and benefits/risks of a POCUS examination
ETPR3	Clearly and effectively communicate US results to patients, families, and the medical team
LSE1	Have a working understanding of the basic relevant pediatric POCUS literature
LSE2	Demonstrate effective teaching of POCUS concepts and skills to other learners
LSE5	Describe policy statements pertinent to PEM POCUS, e.g., ACEP emergency US guidelines, AAP Policy Statement on Point‐of‐Care Ultrasonography by PEM physicians
ADM3	Recognize the value of a QA database for US examinations
ADM5	Recognize the need for a standardized process for properly credentialing providers in US
ADM9	Recognize the differences between ED POCUS and radiology US
Global competencies
COMP_1	Demonstrates fundamental knowledge of US physics
COMP_2	Demonstrates accurate technique for POCUS image acquisition (knobology)
COMP_3	Differentiates between normal and abnormal anatomy and physiology on US imaging
COMP_4	Accurately interprets POCUS images
COMP_5	Effectively utilizes POCUS in the medical decision making process
COMP_6	Knowledge of POCUS indications, pearls, and pitfalls; utilizes POCUS in appropriate clinical scenarios

Abbreviations: AAP, American Academy of Pediatrics; ACEP, American College of Emergency Physicians; ADM, administration; AQIN, image acquisition and interpretation; COMP, competency; ETPR, ethics and professionalism; Knob, knobology; LSE, literature, scholarship, and education (teaching); MDM, medical decision making; POCUS, point‐of‐care ultrasound; QA, quality assurance; USP, ultrasound physics.

Organization of curriculum content and creation of the competency checklist

The standard educational curriculum consists of not only educational content but also scope (breadth and depth of coverage) and sequence (the order in which content is taught). ²⁰ Upon conclusion of the Delphi, the core leadership group convened to discuss and organize final outcomes into a curriculum. The group decided that the scope of the curriculum was intrinsically defined by wording of the items in “behavioral objective” terminology. However, they believed that the curriculum would be more useful to program leaders if organized into subcompetencies and presented under the global competencies panelists had designated as useful.

In the process of creating a competency checklist, the core leadership group deliberated and subsequently decided to simplify the curriculum into a more practical document. Accordingly, they distilled the six global competencies down to four. First, they combined ultrasound physics and knobology, with the rationale that at this stage of learning it made sense for learners to think of the two competencies together; i.e., one needs to have knowledge of ultrasound physics to develop the skill of knobology (see Figure 2, competency 1). They also combined image interpretation and medical decision making for a similar reason, since the two skills work in concert (see Figure 2, competency 2). Image acquisition remained its own competency (see Figure 2, competency 3), but the remaining competencies were grouped together under an umbrella competency termed administration (see Figure 2, competency 4). The core leadership group then organized the subcompetencies into “basic,” “intermediate,” and “advanced” skills under each competency heading to create the final competency checklist (Figure 2).

FIGURE 2.

POCUS curriculum checklist for PEM fellows: a list of core competencies and subcompetencies organized from basic to advanced. Some subcompetencies involve content that crosses levels of performance. For example, the ability to “define an ultrasound artifact” is considered a basic skill that shows a rudimentary understanding of ultrasound physics, whereas “characterizes common ultrasound artifacts like shadowing, enhancement, mirror images, etc.” is considered a slightly higher level of knowledge and skill, since it involves the application of knowledge to recognize and provide examples of artifacts and being able to explain how they interfere with scan interpretations. AAP, American Academy of Pediatrics; ACEP, American College of Emergency Physicians; PEM, pediatric emergency medicine; POCUS, point‐of‐care ultrasound

DISCUSSION

Using a Delphi panel of POCUS experts, we developed a comprehensive and detailed curriculum to guide PEM fellowship program directors in their implementation of PEM fellow POCUS education. This POCUS curriculum contains specific global competencies and subcompetencies that PEM fellows should achieve to become proficient POCUS practitioners in their eventual PEM practice. Our curriculum, if integrated effectively with the PEM POCUS applications generated by Shefrin et al. and the educational guidelines recommended by Vieira et al. ¹¹ , ¹² is likely to produce a PEM fellowship graduate who is proficient at operating an ultrasound machine at the bedside of a child to acquire and interpret the appropriate scans for diagnosing pediatric pathology or guiding a clinical procedure.

One interesting feature of this Delphi project was how the nature of curriculum adjustments varied from round to round. After round one, most of the item editing was accomplished through removing redundancies, combining similar ideas, and adjusting language to reach panelist agreement on an item's meaning. After the second round, most item editing resulted from elimination of items considered more appropriate for a POCUS fellow. One item of interest that did not reach the final round agreement threshold was establishing a required number of POCUS scans during fellowship (79.3% agreement). Although most panelists were in favor of this item, those who were opposed made a convincing argument to the core leadership group that this type of “number‐based criteria” was not an accurate measure of POCUS competency and was antithetical to competency‐based education. ²¹ , ²²

Another observation of this Delphi process was the contrast between PEM POCUS and EM POCUS panelists’ contributions. The EM POCUS representatives believed that POCUS administration, research, and scholarship were important features of a POCUS education program, while PEM POCUS representatives did not. We believe this difference originates from the foundational purpose of the PEM fellowship compared to that of an EM POCUS (or academic EM ultrasound) fellowship. POCUS has been a core competency for EM residencies for decades. ²³ , ²⁴ PEM fellowships draw applicants from both EM and pediatrics residencies. Since pediatric residencies do not typically incorporate POCUS into training, ²⁵ , ²⁶  many pediatric residents enter PEM fellowship without significant POCUS knowledge and need to first develop skills in core aspects of POCUS image acquisition and interpretation. Only then can fellows incorporate POCUS into medical decision making for the duration of their training, with limited ability of all to develop administrative, research, and scholarship skills.

As POCUS continues to permeate PEM, there is a need for a standardized curriculum to ensure appropriate allocation of expertise, time, and resources. Prior studies have recommended the creation and implementation of a formal POCUS curriculum for PEM fellows, and a recent survey study of PEM fellowships examined compliance with this recommendation. ⁷ The 2020 survey study by Acuña et al. evaluated the current state of POCUS education and training in PEM fellowships. ¹⁰ Of the 48 programs that responded to the study (80%), 77% reported a working POCUS curriculum in place. This study did not, however, investigate the educational content of these curricula nor their associated educational methods. The creation of a PEM POCUS curriculum standard such as the one derived from our study can be used to provide guidance for programs without POCUS education as well as those with an established curriculum.

The competency checklist we designed provides a practical framework for educators as well as a checklist for program directors to track fellow's progress through PEM‐specific POCUS competencies. This checklist can also help programs integrate POCUS into current fellowship curricula through sequencing of both the complex psychomotor skills needed to perform POCUS and the clinical knowledge needed to act on POCUS images. Additionally, with the relatively low prevalence of pathology in pediatric patients compared to adult patients, simulation and image repositories can supplement the knowledge requirements suggested by our work. ²⁷ , ²⁸ Implementation of this proposed POCUS curriculum, including validation of a time frame in which it should be taught and routes of implementation such as objective structured clinical examinations for POCUS, will be important for future investigations. ²⁹ , ³⁰ , ³¹ , ³²

LIMITATIONS

The limitations of our study relate to those inherent to any modified Delphi study, which is reliance on panelists with whom face‐to‐face contact is limited or nonexistent. While we identified and enlisted panelists on the basis of their accomplishments in the field of POCUS and/or medical education, we were unable to directly observe how they processed our feedback to make adjustments to their voting. The lack of face‐to‐face interactions also made it impossible to gauge the time and rigor individual panelists invested in this project. However, the panel response rate of 100% across all rounds demonstrate a high level of interest and commitment to the study. The possibility exists that panelist efforts varied greatly, and yet we treated their contributions as equal. A second limitation is that the opinion of these 29 panelists may not necessarily reflect that of all the experts in the field. A final limitation was the greater influence that the core leadership group had on the final project outcomes, particularly in providing the sequence in which subcompetencies should be taught, i.e., basic to advanced knowledge and skills. Due to their direct involvement with PEM fellowship programs and PEM POCUS, we felt this was appropriate given our objectives to design a practical standard curriculum for PEM fellow POCUS education.

CONCLUSIONS

Through a modified Delphi process, we created an expert‐derived point‐of‐care ultrasound curriculum that can be incorporated into pediatric emergency medicine fellowship training. This point‐of‐care ultrasound curriculum may provide guidance for fellowships who have yet to adopt point‐of‐care ultrasound education and standards for those who have an existing curriculum. As the field of point‐of‐care ultrasound continues to grow in general pediatrics and pediatric emergency medicine, we anticipate that future iterations of our curriculum will require modifications to include more advanced material.

CONFLICT OF INTEREST

The authors have no potential conflicts to disclose.

AUTHOR CONTRIBUTIONS

Delia Gold conceived of the study and supervised data acquisition. David P. Way contributed to study design and provided statistical analysis, data management, and advisement of the Delphi process. Deborah Hsu contributed to the conceptual framework and organized the Delphi results into global competencies and subcompetencies. Jennifer R. Marin provided oversight of data interpretation and manuscript production, including important intellectual content. Delia Gold, Marla Levine, Deborah Hsu, David P. Way, Allan E. Shefrin, Samuel H. F. Lam, Resa Lewiss, and Jennifer R. Marin served as members of the Core Leadership Group, organized Delphi survey results for feedback to the Delphi Panel, and contributed to data interpretation and drafting of critical revisions of the manuscript. Members of the PEM POCUS Curriculum Task Force generated the content of the final PEM POCUS Curriculum through the Delphi method and contributed to critical revisions of the manuscript.

Supporting information

Data Supplement S1. Supplemental material.

Gold D, Levine M, Hsu D, et al; the PEM POCUS Curriculum Task Force . Design of a point‐of‐care ultrasound curriculum for pediatric emergency medicine fellows: A Delphi study. AEM Educ Train. 2021;5:e10700. doi: 10.1002/aet2.10700