Consensus Core Point‐of‐care Ultrasound Applications for Pediatric Emergency Medicine Training

Abstract

Background

Pediatric emergency medicine (PEM) physicians have variably incorporated point‐of‐care ultrasound (POCUS) into their practice. Prior guidelines describe the scope of POCUS practice for PEM physicians; however, consensus does not yet exist about which applications should be prioritized and taught as fundamental skills for PEM trainees. Members of the PEM POCUS Network (P2Network) conducted a consensus‐building process to determine which applications to incorporate into PEM fellowship training.

Methods

A multinational group of experts in PEM POCUS was recruited from the P2Network and greater PEM POCUS community if they met the following criteria: performed over 1,000 POCUS scans and had at least 3 years of experience teaching POCUS to PEM fellows, were a local academic POCUS leader, or completed a formal PEM POCUS fellowship. Experts rated 60 possible PEM POCUS applications for their importance to include as part of a PEM fellowship curriculum using a modified Delphi consensus‐building technique.

Results

In round 1, 66 of 92 (72%) participants responded to an e‐mail survey of which 48 met expert criteria and completed the survey. Consensus was reached to include 18 items in a PEM fellowship curriculum and to exclude two items. The 40 remaining items and seven additional items were considered in round 2. Thirty‐seven of 48 (77%) experts completed round 2 reaching consensus to include three more items and exclude five. The remaining 39 items did not reach consensus for inclusion or exclusion.

Conclusion

Experts reached consensus on 21 core POCUS applications to include in PEM fellowship curricula.

Point‐of‐care ultrasound (POCUS) has been established in the practice of emergency medicine for over two decades.1, 2 More recently, its use has expanded into the practice of pediatric emergency medicine (PEM).3 Most PEM fellowship programs now offer formal ultrasound training to their fellows and incorporate POCUS into the care of the pediatric patient.4, 5 In 2015, the American Academy of Pediatrics (AAP), the American College of Emergency Physicians (ACEP), the Society for Academic Emergency Medicine (SAEM), and the World Interactive Network Focused on Critical Ultrasound (WINFOCUS) issued a joint policy statement on the use of POCUS by PEM physicians thus supporting and further reinforcing its role in the clinical practice of PEM.6, 7

In 2001, with revisions in 2008 and 2016, ACEP published POCUS training guidelines in its policy statement for POCUS use by emergency physicians though these were not specific to PEM and do not suggest the applications in which PEM physicians should be competent.8 Further, the Accreditation Council of Graduate Medical Education (ACGME) has made recommendations for ultrasound education of emergency medicine residents through their milestones project.9, 10 To date, no similarly comprehensive recommendations exist for PEM fellows.11 Objectives of PEM training include minimal POCUS applications within the American Board of Pediatrics core content for PEM training, certification, and maintenance of certification. Understanding the role of POCUS in the assessment of the trauma victim and sonographic skills related to focused assessment with sonography for trauma, focused cardiac ultrasound, assessment of suspected ectopic pregnancy, and foreign body localization and removal are the only ultrasound‐related content included to date.12 Likewise, the Royal College of Physicians and Surgeons of Canada recommends understanding the utility, applications, and limitations of POCUS in its PEM training objectives; however, no specific recommendations exist to guide PEM fellowship directors regarding what POCUS content should be included in fellowship training.13

As PEM physicians have incorporated POCUS into practice the number of described uses for pediatric POCUS has rapidly increased creating a need for detailed educational guidelines for PEM fellowship programs. Consensus guidelines for implementing ultrasound education for PEM fellows have been published by a small group of POCUS leaders, although these are better thought of as a consensus statement on which applications are within the scope of PEM training and not what should form the core content of training.14, 15

The P2Network (www.p2network.com) was formed in 2014 by PEM POCUS leaders to further PEM POCUS collaboration. P2Network's main objectives are to collaborate in the areas of PEM POCUS education, administration, research, and mentorship. The need to arrive at more standardized curricula for PEM fellows and PEM POCUS fellows was recognized as top priorities to better define the skill set, training expectations, and expectations for PEM POCUS performance in practice.

In this study, we set out to establish consensus guidelines for core applications to include in a POCUS curriculum for PEM fellows by leveraging a large diverse group of experts from within and outside the P2Network using a modified Delphi survey.

Patients and Methods

The research protocol was reviewed and approved by the Children's Hospital of Eastern Ontario Research Ethics Board.

Study Design

A modified Delphi method was used to reach consensus core applications. This method involves two or more iterations of a data collection instrument, in this case an electronic survey, to a group of stakeholders to reach consensus.16, 17 In designing this study consensus was defined to be reached when ≥80% of respondents agreed on a given item. This level of agreement has been used in previous studies, including ultrasound studies.16, 17, 18, 19, 20 Between each stage, data was analyzed and then presented in an anonymized fashion in the subsequent iteration to help influence decision making. The need for future iterations was based on whether there were significant changes in ratings on nonconsensus items between iterations.

Participants

Experts from the P2Network and the greater PEM POCUS community were invited to participate in this study. Participants were considered experts if they had performed at least 1,000 pediatric POCUS scans and met at least one of the following criteria: 3 or more years of experience in teaching POCUS to PEM fellows, PEM POCUS directorship, or leadership at an academic or tertiary pediatric emergency department or PEM POCUS fellowship trained.

At the time the survey was conducted, the P2Network had 87 members. To reach as many experts as possible, we identified PEM POCUS leaders who were not members of the P2Network from previous publications, listservs, and personal relationships. This subject pool was deemed sufficient as the acceptable number of participants to reach a consensus using the Delphi technique ranges from four to 200.21, 22, 23, 24, 25

Survey

The Delphi survey was developed by the investigators, all of whom are local, national, and/or international experts in PEM POCUS. All known PEM POCUS applications were identified by reviewing the current literature and expert opinion of the study team and included in the survey.14, 15 All members of the study team reviewed and edited each iteration of the survey and pilot tested it to identify errors and points of confusion and ensure ease of administration. The authors deliberately broke down protocols to their individual applications, as in the case of the extended focused assessment with sonography in trauma (E‐FAST), which incorporates abdominal, cardiac, and pulmonary components, with the understanding that educators could teach these together, but each application has value in being able to be performed and understood independent of the other applications.

First Iteration

The round 1 survey was sent to all P2Network members by e‐mail. Experts who were not part of the P2Network were e‐mailed directly to invite them to participate. A unique identifier allowed each participant to access their survey. The first series of questions assessed the respondent's eligibility and did not allow the participant to proceed if they did not meet the expert eligibility criteria. Baseline demographics including level of training, institution type, and educational structures were collected in this iteration of the survey. Following this, participants were asked to rate each ultrasound application based on their perceived level of importance for PEM fellows. Importance was ranked on a Likert‐type scale ranging from 1 to 4 (1 = not important to 4 = extremely important) as has been used in previous Delphi studies.26, 27, 28, 29 Participants could add free‐text comments including additional applications, justification of choices, and requests for clarification. The round 1 survey is attached included in Data Supplement S1 (available as supporting information in the online version of this paper, which is available at http://onlinelibrary.wiley.com/doi/10.1002/aet2.10332/full).

Further Iterations

A second iteration was deemed necessary after many items did not reach consensus for inclusion or exclusion. This survey was distributed only to those who completed the first iteration. POCUS applications that did not reach 80% agreement were included in the second survey. Round 1 results for each POCUS application, in the form of means, were provided to the respondents. Respondents rerated each application's level of importance, as with the first iteration. Additionally, seven applications were added to round 2 based on round 1 free‐text comments. In round 2, respondents were asked, in yes or no format, whether they felt that these new items should be included in a future round and if the majority of respondents agreed they should, a third round would be triggered.

Data Collection and Analysis

REDCap (v6.14.1, Vanderbilt University), a secure Web application for building and managing online research surveys and databases, was used to distribute the surveys and collect responses. Ratings were analyzed using SPSS (v22, IBM Corp.), providing a mean frequency for each POCUS application.

Results

In round 1 a total of 62 of 87 (71%) P2Network members and four of five (80%) non‐P2Network members responded to the e‐mail survey. Of those respondents, 48 met expert criteria and completed round 1 (44 P2Network members, four non‐P2Network members). Thirty‐seven of the 48 respondents from round 1 (77%) completed round 2 (33 P2Network, four non‐P2Network members). The 11 participants who completed round 1 who did not complete round 2 were contacted multiple times by e‐mail to solicit completion of the second round but were lost to follow‐up.

The majority of responders were working in the United States and Canada at the time of study (Table 1). The expert panel was primarily made up of physicians who completed a pediatric residency followed by a PEM fellowship. More than half of the respondents completed a formal PEM or emergency medicine POCUS fellowship.

Table 1.

Profile of Expert Respondents Completing Round 1 (N = 48)

Characteristic	No. (%)
Country
United States	33 (69)
Canada	12 (25)
Othera	3 (6)
Years of practice outside training, median (IQR)	5.5 (6.25)
Postgraduate training (selected all that applied)
Pediatrics	41 (85)
Emergency medicine	5 (10)
PEM	43 (90)
Other	1 (2)
Ultrasound training (selected all that applied)
PEM POCUS fellowship	17 (35)
EM POCUS fellowship	13 (27)
PEM fellowship rotation	12 (25)
Longitudinal US training within PEM fellowship	14 (29)
Longitudinal US training within EM residency	1 (2)
1 or 2‐day course	20 (42)
Self‐trained	15 (31)
Other	6 (13)
RDMS/RDPS certified	12 (25)
Involved in ultrasound education of PEM/PEM POCUS fellows	46 (96)
Number of years, median (IQR)	4.8 (1.8‐7.8)
Fellowships available at expert's institution
PEM	42 (88)
PEM POCUS	20 (42)
EM POCUS	16 (33)

IQR = Interquartile range; PEM = pediatric emergency medicine; POCUS = point‐of‐care ultrasound; US = ultrasound.

^aOne respondent each from Brazil, Israel, and Jamaica.

Consensus for inclusion was reached if 80% of responses fell within the “extremely important” or “very important” categories. After two Delphi rounds, consensus was reached to include 21 of the 60 possible PEM POCUS applications (Table 2). The recommended applications to include in a POCUS curriculum for PEM fellowship are listed in Table 3. Consensus for exclusion was determined if 80% of responses fell within the “not important” or “somewhat important” categories. After two Delphi rounds, consensus was reached to exclude seven of the 60 possible PEM POCUS applications (Table 2). The recommended applications to exclude from the PEM fellowship curriculum are listed in Table 4. There was no expert consensus on the remaining 32 applications (Table 5).

Table 2.

Number of POCUS Applications by Delphi Round

	Round 1 (60 Applications)	Round 2 (47a Applications)	Final Applications
Include	18 (30)	3 (6.4)	21
Exclude	2 (3.3)	5 (10.6)	7
No consensus	40 (66.7)	39a (83)	39a

Data are reported as n (%).

POCUS = point‐of‐care ultrasound.

^aIncludes seven additional items added by respondents in round 1.

Table 3.

POCUS Applications to Include by Delphi Technique

Application	No. of Experts Who Ranked “Extremely/Very Important” (%)
Round 1
Identify free peritoneal fluid in trauma	100
Identify nontraumatic pericardial effusion	100
Identify pericardial effusion in trauma	100
Identify hemothorax	98
Identify pleural fluid/effusion	98
Identify pneumothorax	98
Identify cardiac standstill	96
Abscess incision and drainage	94
Identify abscess	94
Central line placement	91
Evaluate cardiac function	88
Identify cellulitis	88
Identify intussusception	87
Identify intrauterine pregnancy	85
Identify soft tissue foreign body	85
Assess bladder volume	83
Identify lung consolidation	83
Peripheral IV access	81
Round 2
Foreign body localizations and removal	89
Identify pulmonary edema	84
Pericardiocentesis	84

POCUS = point‐of‐care ultrasound.

Table 4.

POCUS Applications to Exclude by Delphi Technique

Application	No. of Experts Who Ranked “Not/Somewhat Important” (%)
Round 1
Identify abdominal aortic aneurysm	90
Identify myositis	83
Round 2
Assess for ovarian torsion	92
Identify epididimoorchitis	84
Identify vitreous detachment	84
Identify vitreous hemorrhage	84
Identify testicular mass	81

POCUS = point‐of‐care ultrasound.

Table 5.

POCUS Applications With No Consensus to Include or Exclude by Delphi Technique

	Percentage of Experts Who Ranked “Extremely/Very Important” (%)
Identify hydronephrosis	78
Assess IVC for volume status	76
Evaluate optic nerve for papilledema	76
Nerve blocks	76
Identify joint effusions	73
Identify adenitis	70
Identify appendicitis	62
Identify bone fractures	62
Identify skull fracture	62
Identify cholelithiasis	59
Arthrocentesis	57
Endotracheal tube position confirmation	57
Identify cholecystitis	57
Suprapubic bladder aspiration	54
Identify pyloric stenosis	51
Lumbar puncture	51
Identify testicular torsion	49
Pleurocentesis	49
Determine viability of intrauterine pregnancy	46
Fracture reduction	43
Assess IVC to aorta ratio	41
Identify retinal detachment	41
IO needle confirmation	40
Arterial lines	35
Identify deep vein thrombosis	32
Identify peritonsillar abscess	32
Identify necrotizing fasciitis	30
Paracentesis	30
Identify adnexal abscess	27
Assess for ovarian cyst/mass	22
Identify lens dislocation	22
Peritonsillar abscess drainage	21

IO = intraosseous; IVC = inferior vena cava; POCUS = point‐of‐care ultrasound.

In round 1 experts suggested seven applications, via free text, to be added to the Delphi process including: identify severe cardiac valvular disease, assess congenital neck abnormalities, identify corneal foreign bodies, identify ocular drusen, identify bowel obstruction, identify hydrocele, and identify varicocele. Experts were asked in round 2 whether these should be added to future rounds and none received greater than 50% yes votes to trigger a third round. The Delphi process ended after two rounds since no significant changes were observed between the first two rounds on applications that had not achieved consensus.

Discussion

In this study we used a modified Delphi technique to determine which POCUS applications should be taught to PEM fellows. PEM POCUS experts identified 21 applications that should be considered core applications and included in all PEM curricula. These core applications include traumatic applications (identification of free peritoneal fluid, pericardial effusion, hemothorax, and pneumothorax), focused cardiac assessment (identification of nontraumatic pericardial effusion and cardiac standstill and evaluation of global cardiac function), soft tissue applications (identification of cellulitis, abscess, and foreign bodies), pulmonary applications (identification of pleural fluid and effusions, lung consolidation, and pulmonary edema), other diagnostic applications (identifying first trimester pregnancy, diagnosing intussusception, and assessing bladder volume), and ultrasound‐guided procedures (central and peripheral vascular access, abscess incision and drainage, foreign body localization and removal, and pericardiocentesis). Participants were asked only to rate applications and not items related to ultrasound physics, image generation, common artifacts, and knobology. We recommend that any POCUS educational program include these components.

Employing a modified Delphi method, as implemented in this study, to achieve consensus has several benefits, especially when performed electronically. It allows for a large group of people to come together to reach consensus in a cost‐effective manner, without geographic limitations. It can be delivered anonymously to allow for pressure‐free responses.30 The Delphi method is a surrogate for a face‐to‐face meeting where the opinions of others are freely known and can influence the decision making of others. Hence, it is common in the Delphi method to share results from previous iterations.

Reaching consensus on the core applications will hopefully lead to the incorporation of these recommendations by stakeholders such as local training programs, professional organizations, and training accreditation bodies. PEM POCUS educators are encouraged to structure their curricula to, at a minimum, include these applications. Institutions who are unable to meet these core objectives are encouraged to seek out ways to ensure that their fellows learn these applications whether by external training or via recruitment of physicians who are able to teach these applications. This consensus can serve as the baseline for what PEM fellows should learn during their PEM training and what can be expected of new graduates when they enter the workforce. Training programs are encouraged to go beyond this core set of applications based on local expertise and the future practice needs of their trainees. Applications that did not reach consensus might be impactful in the right hands, in the right setting, and at the right time.

This consensus does not direct educators how to teach the applications only what they should include in a POCUS curriculum for PEM fellows. Educational milestones should be created that mirror this consensus much like the Council of Emergency Medicine Residency Directors (CORD) has established for emergency medicine residency.10 Once these education milestones are developed, curricula can be developed by PEM POCUS educators to meet these objectives. Curricula should include didactic training in ultrasound concepts, recognizing normal and pathological anatomy, indications for use, how to incorporate into medical decision making, and hands‐on training to assist in the development of the motor skills necessary for image acquisition. Ultrasound simulators and image data banks can be used to assist learners, especially for less frequent applications, in the development of pattern recognition necessary for POCUS use in PEM.

Other stakeholders that would benefit from this consensus include accreditation bodies and professional organizations. The ACGME, Royal College of Physicians and Surgeons of Canada, and other similar accreditation bodies can include these applications as the POCUS objectives for PEM fellowship. Professional organizations such as the AAP, ACEP, and Canadian Association of Emergency Physicians can also consider incorporating this work into their POCUS position statements.

As the development of PEM POCUS education progresses, dedicated 1‐year fellowships in PEM POCUS have been developed to offer higher education in the areas of program oversight, leadership, research, and education. Experts should clearly define via consensus the POCUS objectives for these PEM POCUS fellowships as this would help clearly define the scope of POCUS training required for PEM fellows and PEM POCUS fellows.

We expect our consensus recommendations to change over time as the PEM POCUS field grows and expertise increases. Physicians will be finding new applications for POCUS as ultrasound machine technology, price, and portability improve and these may warrant consideration for inclusion in curricula in the future. Further, POCUS teaching is permeating medical education and we expect the future PEM fellow to begin fellowship with a broader scope of POCUS knowledge and skills than their predecessors. As curricula are introduced and users achieve greater competence and comfort, we expect some of the applications that at present did not reach consensus for inclusion will in the future. As such, it is our plan to repeat this consensus‐building project in 5 to 7 years.

Limitations

Our study has several limitations. We did our best to identify leaders both inside and outside of the P2Network, and it is possible that we missed some. We expect future iterations to include more participants as the P2Network continues to grow and the amount of PEM POCUS users who meet expert criteria will grow. We will also continue to identify experts outside of the P2Network by surveying worldwide POCUS networks, scanning publications, reviewing conference speaker lists, and leveraging personal relationships. Our project only surveyed PEM POCUS experts using expert criteria to best standardize POCUS training for PEM fellows. We did not survey non‐POCUS using PEM physicians or POCUS users in the general emergency medicine community and it is possible that our results would differ if they participated.

Conclusions

A modified Delphi method was used by the ultrasound collaborative P2Network to reach a consensus on 21 core applications for inclusion in a point‐of‐care ultrasound curriculum for pediatric emergency medicine fellowship. This consensus establishes a baseline for education and expectations of graduating fellows.

The authors acknowledge Dr. Nick Barrowman, Senior Statistician, Clinical Research Unit at the Children's Hospital of Eastern Ontario, for his assistance with survey design and statistical analysis and Ms. Kendra Sikes, Research Coordinator, University of Louisville, for her assistance with the manuscript preparation.

Supporting information

Data Supplement S1. Delphi Round 1 PEM POCUS Curriculum Consensus Survey.

AEM Education and Training 2019;3:251–258