Success of implementation of a systemwide point‐of‐care ultrasound privileging program for emergency medicine faculty

Abstract

Objectives

Point‐of‐care ultrasound (POCUS) is widely used in the emergency department (ED). Not all practicing emergency physicians received POCUS training during residency, leaving a training gap that is reflected in POCUS privileging. The purpose of this study was to evaluate the success of meeting privileging criteria as well as associated factors, following implementation of a basic POCUS training and privileging program within a large emergency medicine department.

Methods

We implemented a POCUS training and privileging program, based on national guidelines, for faculty physicians who worked at one of the following EDs staffed by the same emergency medicine department: a pediatric tertiary site, two tertiary academic sites, and seven community sites. POCUS examinations included aorta, cardiac, first‐trimester obstetrics (OB), and extended focused assessment with sonography in trauma. Pediatric emergency medicine faculty were taught soft tissue and thoracic US instead of aorta and OB. Completion of the program required 16 h of didactics, ≥25 quality‐assured US examinations by examination type, and passing a series of knowledge‐based examinations. Descriptive statistics were calculated. Associations between physician characteristics and successfully becoming privileged in POCUS were modeled using Firth's logistic regression.

Results

A total of 176 faculty physicians were eligible. A total of 145 (82.4%) achieved basic POCUS privileging during the study period. Different pathways were used including 86 (48.9%) practice‐based, nine (5.1%) fellowship‐based, and 82 (46.9%) residency‐based. POCUS privileging was lower for those working in a community versus academic setting (odds ratio 0.3, 95% confidence interval 0.1–0.9). A greater number of scans completed prior to the privileging program was associated with greater success.

Conclusions

Implementation of a POCUS training and privileging program can be successful in a large emergency medicine department that staffs hospitals in a large‐scale health care system composed of both academic and community sites. Faculty physicians with at least some prior exposure to POCUS were more successful.

INTRODUCTION

Point‐of‐care ultrasound (POCUS) is increasingly used within emergency medicine at the bedside to answer a focused, clinical question. ¹  The use of POCUS in the emergency department (ED) has been shown to decrease ionizing radiation, increase patient satisfaction, reduce time to diagnosis, guide management, and improve disposition times. ² , ³ , ⁴ , ⁵ , ⁶ , ⁷ POCUS can also improve success rates and decrease complications when incorporated into bedside procedures. ⁸ , ⁹ , ¹⁰ , ¹¹ US guidance during invasive procedures is recommended by a number of specialty societies and considered by many to be the standard of care, especially for guiding central vascular access. ¹² , ¹³ , ¹⁴

Within the United States, a push has been made to incorporate POCUS into medical school training, ¹⁵ and emergency medicine residencies have been expected to provide US training to residents as a core skill in their education for well over a decade. ¹⁶ In 2015, a policy supporting the use of POCUS by pediatric emergency medicine physicians was announced by the American Academy of Pediatrics. ¹⁷

The American College of Graduate Medical Education did not require incorporation of POCUS as a core component of emergency medicine residency training until 2008. ¹⁸ As a result, emergency physicians (EPs) who completed residency training prior to that time received little to no US education during residency. Some EPs have adopted its use by utilizing continuing medical education training opportunities, but many have identified barriers to learning US independently and feel uncomfortable doing this without a formal training process. ¹⁹ , ²⁰ , ²¹ , ²² Because of these factors, many EPs working in EDs today lack POCUS training and privileging to use POCUS in daily practice. The purpose of this study was to evaluate the success of meeting privileging criteria, as well as associated factors, following implementation of a basic POCUS training and privileging program within a large emergency medicine department.

METHODS

Study design, setting, and participants

This was a prospective cohort study of faculty physicians who belonged to a large emergency medicine department that staffed both academic and community EDs throughout a large network of hospitals. Sites included 10 different hospitals: two large urban teaching hospitals with 88,000–106,000 patient visits each, one pediatric urban academic hospital with >60,000 annual patient visits, and seven community EDs with volumes ranging from 6000–55,000. All practicing physicians in the ED were eligible to be included in the POCUS privileging program, regardless of prior US training. This included physicians with backgrounds in emergency medicine, internal medicine, family medicine, pediatrics, or a combination thereof who were credentialed and employed by the emergency medicine department. A list of faculty physicians was provided by the ED administration. Supplemental faculty were excluded.

Implementation of the POCUS privileging program began in fall 2016 at the three academic sites. Beginning in 2018, a financial incentive was tied to the completion of the program. In 2018, 2019, and 2020 the community sites were added in a staggered fashion. Data were collected through June 2021. This study was deemed exempt by the institutional review board.

POCUS privileging program

The basic POCUS privileging program was based on the American College of Emergency Physicians emergency US guidelines. ¹⁶ Consistent with these guidelines, we established three pathways for EPs to gain privileging to perform POCUS: (1) a residency‐based, (2) a practice‐based, and (3) a fellowship‐based pathway. The guidelines further recommend that practice‐based pathways require physicians to obtain both didactic and hands‐on education and demonstrate proficiency by completing a minimum number of examinations that have been reviewed and graded for accuracy. As such, physicians who had US training incorporated into their residency program and could provide a letter stating that they had successfully completed their residency's POCUS training program were placed in the residency‐based training pathway, as long as their program required the completion of didactics and a minimum of 150 reviewed POCUS examinations. EPs who did not receive POCUS training in residency, did not meet the minimum requirement of 150 examinations, or could not provide a letter attesting to completion of such training were placed within the practice‐based pathway.

The educational objectives of the training program included understanding the indications for POCUS, developing the hands‐on skills needed to acquire adequate POCUS images, recognizing normal and abnormal pathology, and understanding how to incorporate POCUS into patient care. This multimodal approach has since been recognized as important in establishing competency in clinical US. ²³  The basic program covered four core modalities—abdominal aorta, cardiac, first‐trimester obstetrics (OB), and extended focused assessment with sonography in trauma. The program also included instruction on basic US physics, artifact recognition, and basics of operating an US machine. EPs practicing in a pediatric‐only setting had soft tissue and thoracic US modalities swapped for abdominal aorta and OB US examinations since these applications are rarely indicated and not in the routine scope of practice for pediatric emergency medicine. ²⁴

A minimum of 16 h of POCUS‐specific continuing medical education was required, with at least 5 h coming from an in‐person basic US course. Faculty within the emergency US division sponsored several hands‐on courses throughout the year that were available for department faculty at no cost. A minimum of 25 quality‐assured (QA'd) POCUS scans in each of the core four modalities was required for program completion. All images were reviewed and QA'd by a member of the US division. Participants were required to pass a series of postknowledge assessments, correlating to each of the core imaging modalities, with a score of at least 70%. If a passing grade was not achieved on the first try, participants were encouraged to study the topic and, when ready, were allowed unlimited attempts to pass the examination. When these requirements were met, participants were considered to have successfully completed the program.

Data analysis

Data were described using means with standard deviation (SD), medians with interquartile range (IQR), or percentages, where appropriate. Logistic regression (LR) was used to model the associations between clinician characteristics and the primary outcome of becoming credentialed for participants on the practice pathway. Model fit was assessed using Akaike's information criteria (AIC) and accuracy via area under the receiver operating characteristic curve (AUC). A priori, it was assumed that prior US experience would overall be the best predictor of becoming privileged in POCUS but whether having US training as part of residency (yes vs. no), subjective report of prior experience (none vs. some vs. moderate vs. large amount), or self‐report of number of US examinations performed prior to the study period (0–25 vs. 26–100 vs. >100) would be the strongest predictor was not known. Therefore, univariable LR was used to identify the strongest of these predictors using AIC and AUC. The best performing predictor was then entered into a multivariable model, with other candidate predictors: part‐time employment versus full‐time (defined as >0.75 full‐time equivalent), patient type (adult only vs. pediatric only or mixed), work site (academic vs. community), type of residency training (EM vs. internal medicine vs. family practice vs. pediatrics vs. pediatric EM), and years of work experience. Owing to the low overall number of events and data sparsity, penalized LR using Firth's method was used; to avoid overfitting, the maximum degrees of freedom in the multivariable model was set at 3 (not including the intercept). Significance level was set at alpha = 0.05. Statistical analysis was completed using SAS version 9.4 (SAS Institute).

RESULTS

We analyzed data on 176 faculty physicians. Physicians were a median of 10 years (SD 8.9) post–residency graduation (range 1–39 years). Demographics for all faculty are listed in Table 1. At the onset of the POCUS privileging program, 91 (51%) of 176 emergency medicine faculty within the academic and community hospitals included in this study had previously achieved the needed requirements for basic POCUS privileges; 82 (46.9%) faculty were trained during residency and nine (5.1%) faculty were US fellowship trained or had registered diagnostic medical sonographer certification. Eighty‐six of 176 faculty (48.9%) were placed in the practice‐based pathway. Over the course of the program, 22 faculty left due to retirement, death, or a new job including 11 in the practice pathway. At the completion of the program, 131 of 154 faculty (85.1%) achieved basic POCUS privileges, and 23 (14.9%) did not.

TABLE 1.

Emergency physician demographics

	All (n = 176)	Practice pathway (n = 86)
Gender, male	108 (61.4%)	57 (66%)
Years post–residency training, median (IQR)	10 (8.9)	19 (9)
Training
Emergency medicine	126 (71.6%)	50 (58.1%)
Internal medicine	7 (4.0%)	7 (8.2%)
Family medicine	9 (5.1%)	9 (10.5%)
EM/peds, pediatric emergency medicine	30 (17.0%)	16 (18.6%)
Pediatric	4 (2.3%)	4 (4.65%)
ED site
Academic	112 (63.6%)	51 (59.3%)
Community	54 (30.7%)	45 (40.7)
Split (academic and community)	10 (5.7%)	0
General ED	146 (83.0%)	70 (81.4%)
Pediatric only	21 (11.9%)	15 (17.4%)
Split (adult and pediatric)	9 (5.1%)	1 (1.2%)
FTE
Full time	157 (89.2%)	17(83.7%)
Part‐time (<0.75)	19 (10.8%)	14 (16.3%)
Prior US use	n = 161	n = 79
None to very little	12 (7.5%)	12 (15.9%)
Some	31 (19.3%)	27 (34.1%)
Moderate	50 (31%)	32 (40.5%)
Large	68 (42.2%)	8 (10.1%)
Number of USs previously performed	n = 159	n = 78
0–25	35 (22.0%)	33 (42.3%)
26–100	39 (24.5%)	25 (32.1)
>100	85 (53.5%)	20 (25.6)

Abbreviation: US, ultrasound.

After the 11 faculty who left during the credentialing period were excluded, 53 (70.7%) of 75 faculty in the practice pathway completed the requirements needed for basic POCUS privileges, while 22 (29.3%) did not. The majority of the practice‐based pathway faculty, 58 (77%) of 75, reported some prior experience with POCUS, while 11 (16%) reported little to no prior experience, while data were missing for six subjects. Sixty‐three (84%) faculty reported no US training during residency, and 29 (43%) had performed 0–25 POCUS examinations before the study period (see Table 1). During the study period, faculty in the practice pathway submitted a total of 11,246 POCUS examinations for QA; 7842 of those examinations were submitted and QA'd using our internal POCUS workflow solution and 3404 were from hands‐on scanning sessions outside of the clinical setting (see Tables 2 and 3 and Figures 1, S1 and S2 for number and types of POCUS examinations). The number of scans increased significantly between years 2017 and 2018.

TABLE 2.

POCUS Examinations submitted for QA by practice‐pathway faculty, Median ± SD

Scan type	2016	2017	2018	2019	2020	2021	Total
Aorta	36, 0 ± 1	18, 0 ± 6	495, 0 ± 10	166, 0 ± 4	70, 0 ± 2	18, 0 ± 1	803, 2 ± 14
Cardiac	68, 0 ± 2	37, 0 ± 1	673, 0 ± 12	428, 1 ± 7	262, 0 ± 5	168, 0 ± 4	1,636, 9 ± 24
E‐FAST	79, 0 ± 3	70, 0 ± 2	691, 0 ± 12	603, 1 ± 11	469, 1 ± 9	207, 0 ± 5	2,119, 9 ± 34
OB	114, 0 ± 3	52, 0 ± 2	420, 0 ± 10	492, 0 ± 9	311, 0 ± 6	225, 0 ± 5	1,614, 2 ± 30
ST a	0, 0 ± 0	2, 0 ± 1	442, 11 ± 28	461, 31 ± 20	258, 18 ± 11	105, 2 ± 8	1,268, 83 ± 54
Thoracic a	0, 0 ± 0	0, 0 ± 0	195, 2 ± 13	164, 11 ± 11	43, 3 ± 2	0, 0 ± 0	402, 28 ± 18
Total	297	179	2916	2314	1413	723	7842

Abbreviations: E‐FAST, extended focused assessment with sonography in trauma; OB, obstetrics; POCUS, point‐of‐care ultrasound.

^a Faculty practicing at a pediatric‐only site.

TABLE 3.

POCUS examinations performed during hands‐on workshops by practice‐pathway faculty

Scan type	2016–2018	2019	2020	2021	Total
Aorta	0	642, 1 ± 12	103, 0 ± 3	146, 0 ± 5	891, 3 ± 15
Cardiac	0	709, 2 ± 13	122, 0 ± 3	125, 0 ± 4	956, 6 ± 16
E‐FAST	0	726, 2 ± 12	153, 0 ± 4	119, 0 ± 4	998, 5 ± 16
OB	0	432, 0 ± 9	35, 0 ± 1	27, 0 ± 1	494, 1 ± 10
ST a	0	26, 0 ± 4	0, 0 ± 0	0, 0 ± 0	26, 0 ± 4
Thoracic a	0	30, 0 ± 4	9, 0 ± 2	0, 0 ± 0	39, 0 ± 5
Total	0	2565	422	417	3404

Data are reported as median ± SD.

Abbreviations: E‐FAST, extended focused assessment with sonography in trauma; OB, obstetrics; POCUS, point‐of‐care ultrasound.

^a Faculty practicing at a pediatric‐only site.

FIGURE 1.

POCUS examinations performed by practice‐pathway faculty. POCUS, point‐of‐care ultrasound

For the 53 faculty in the practice‐based training pathway who achieved privileging, median years in practice was 19 years (IQR 8), 33 (62%) were male, 47 (89%) worked full‐time, 48 (68%) worked in an academic‐only setting, 11 (21%) worked in a pediatric‐only setting, and 43 (81%) reported no US training during residency. For the number of POCUS examinations performed prior to the privileging program, 13 subjects performed 0–25 ("low" group), 19 performed 26–100 ("moderate" group), and 16 performed >100 ("high" group); data were missing for five subjects in this group.

Of the 22 faculty in the practice‐based pathway who did not achieve basic POCUS privileges during the study period, nine (41%) faculty worked at an academic site only, 13 (59%) at a community site only, and none split clinical time between both. Nineteen (86%) faculty worked in a general ED, three (14%) at a pediatric ED, and none split between both. Sixteen (73%) were full‐time faculty and six (27%) were part‐time. Median (IQR) years in practice was 21 (10) years, 18 (82%) were male, and 20 (91%) reported no US training during residency. In terms of prior US experience, eight (40%) reported “no to little US experience,” seven (35%) reported “some” experience, four reported “moderate” experience, and one reported “large” prior experience; data were missing for two subjects in this group. For the number of POCUS examinations performed prior to the privileging program, 16 subjects performed 0–25 (“low” group), none performed 26–100 (moderate group), and four performed >100 (“high” group); data were missing for two people in this group.

On univariable LR, number of reported prior POCUS examinations was the strongest predictor of achieving POCUS privileging, with an odds ratio (OR) for success of high group versus low group of 4.5 (95% confidence interval [CI] 1.2–16.3) and moderate versus low of 47.7 (95% CI 2.4–930.1), AUC 0.80 (95% CI 0.71–0.89). For some versus “no” US training during residency, the OR for obtaining credentialing was 1.9 (95% CI 0.43–9.1), AUC 0.55 (95% CI 0.47–0.63). For self‐reported prior US experience, compared to the “none‐to‐little” group, the OR for achieving POCUS privileges for the some, moderate, and large amount of experience groups were 5.3 (95% CI 1.11–25.7), 12.7 (95% CI 2.4–66.5), and 12.1 (95% CI 1.3–117.4), respectively, with AUC 0.73 (95% CI 0.59–0.86). Model fit, assessed by AIC, was best for reported prior number of US examinations. In the final multivariable model, after full‐ versus part‐time work was adjusted for, the OR for achieving POCUS privileges for the high and moderate groups, compared to the low number of prior US examinations performed group, was 4.9 (95% CI 1.2–19.7) and 60.1 (95% CI 2.8–999), AUC 0.83 (95% CI 0.74–0.92).

DISCUSSION

POCUS has developed into a powerful tool in emergency medicine and has been a required part of residency training for over a decade. However, there still exists a significant training gap in EPs currently in practice. Physicians who trained before POCUS was widely available are increasingly expected to be able to perform and incorporate POCUS into their practice, particularly in locations where trainees are learning the practice of emergency medicine. As is the case in our department and similar to national survey data, up to half of EPs lack the training needed for basic POCUS privileges, even in departments housing emergency medicine training programs. ²⁵ To provide uniformly excellent care, emergency medicine departments as well as health care organizations at large, face the challenge of overcoming this training gap and systematically training the experienced physician workforce in this skill set. To meet this challenge, our department implemented a multiyear POCUS training program that featured structured didactic opportunities, in‐person mentored scanning sessions, and ongoing feedback on images obtained through independent scanning within the ED. ²⁶  This approach successfully trained and privileged the majority of physicians in our department who were previously unfamiliar with US within a 5‐year period.

An important element in the success of this training program was the addition of a financial incentive in 2018. Prior to this incentive, participation in the program was small. After this incentive was instituted, the majority of the faculty in the department began work toward completion of the training and privileging program. While this incentive was modest compared to the annual physician salary, it was associated with a significant increase in completing the required didactics, hands‐on training, and scan numbers required for POCUS privileging. We speculate that the opportunity cost of losing this small bonus combined with the social pressure of uniform privileging expectations may have outweighed prior reluctance to complete POCUS training. We also believe the department leadership's willingness to offer a financial incentive made a profound statement as to the importance of achieving this training as well as recognizing that practicing physicians should have some compensation as to the amount of time needed to achieve privileging. The positive impact of financial incentive on POCUS training and documentation is consistent with prior reports. ²⁷ , ²⁸ Similar to our model, Budhram et al. ²⁷ built in a financial incentive into their POCUS training and privileging program from the beginning. While they did not directly measure the effect that this financial incentive played, they theorized that the incentive as well as deadlines were a key part of successfully privileging a small cohort of EPs. Similarly, Melton et al. ²⁸ found that providing both positive and negative financial incentives resulted in improved POCUS documentation among EPs who had already achieved POCUS privileges, ultimately improving the rate and number of POCUS studies that were completed for clinical care. While a financial incentive seems to be a reliable motivator for POCUS education and credentialing, there are examples of successful educational programs without a financial component. Cormack et al. ²⁹ demonstrated an educational platform that trained and privileged a significant portion of a faculty group without financial incentives, albeit at a much lower rate than demonstrated in ours and prior studies.

We found several intrinsic factors positively associated with achieving POCUS privileging, including previous training within residency and use of POCUS as a practicing attending. Interestingly, having moderate prior POCUS experience was associated with the highest likelihood of completing the privileging program compared to “none/low” or high POCUS experience. We speculate that this moderate experience group may have had enough prior exposure to appreciate the utility of POCUS, but not so much that the privileging training seemed unnecessary. Separate messaging may be important in engaging the remaining groups. To improve the successful completion of a privileging program among faculty with “low or minimal” prior experience, future efforts may focus on demonstrating the diagnostic and prognostic value of POCUS to faculty who may be less aware of the importance and utility of POCUS. To improve privileging among faculty with large prior POCUS exposure, future efforts may focus on demonstrating that standardized training may identify and remediate knowledge gaps that may remain from on‐the‐job informal training.

While the number of POCUS scans completed trended lower after peaking in 2018, they have plateaued at a much higher rate compared to the preintervention baseline. This was expected because many faculty were required to complete 100 scans each during the initial training phase in 2018 but had lower required threshold in subsequent years. The postprivileging plateau is a more realistic expectation that suggests ongoing POCUS use and skill retention among faculty members who completed the POCUS privileging requirements. In addition, the number of “hands‐on” scans drop dramatically in 2020 and 2021 from restrictions related to the COVID‐19 pandemic; some workshops were canceled and all had attendance capped.

Even though health care organizations vary in size and structure, keeping physician knowledge base and skill sets in line with the standard of care is a uniform challenge. Here we describe a successful approach for training physician staff in a new essential skill, with a combination of opportunistic didactics, personalized learning sessions, and financial incentives that can be replicated at other institutions and in other departments interested in privileging their faculty in POCUS. Future research should focus on the progression from learning basic POCUS applications to more advanced techniques. In addition, studying the utilization of POCUS by these newly privileged physicians will be important.

LIMITATIONS

There are several limitations that may limit the generalizability of this training to other departments and institutions. This study was based within a large department that staffed several different hospitals. However, we examined a large sample of faculty and included faculty practicing in varied settings. It is important to note that not all faculty started the privileging program at the same time due to the large amount of resources needed to implement the curriculum, in addition to participant scheduling conflicts and other academic commitments. Nevertheless, all faculty participants received the same training. Departmental support for experienced faculty instructors to train novice faculty and other necessary resources such as equipment may vary from institution to institution and could be a significant barrier. Our department's financial incentive greatly contributed to successful completion by faculty and overall feasibility of the curriculum. Lastly, as the sample size for the faculty on the practice‐based pathway who did not achieve privileging was small, this limited our ability to determine factors associated with not meeting privileging, because we had a limited number of events/outcome and that limited our ability to adjust for confounding factors.

CONCLUSIONS

Implementation of a point‐of‐care ultrasound privileging program was feasible within a large‐scale Department of Emergency Medicine that staffed a combination of both academic and community hospitals. Physicians with a variety of training backgrounds who worked in these EDs with little to no prior point‐of‐care ultrasound experience were successful in completing the practice‐based pathway described by American College of Emergency Physicians.

CONFLICTS OF INTEREST

FMR and BN have been in the last year a consultant for GE Healthcare. RMF is or has been in the past year a consultant for and/or received Honoria from Vave Healthcare and 3rd Rock Ultrasound. AH, SKK, MAR, LKR, GSZ, RRE, and DB have no conflicts of interest.

AUTHOR CONTRIBUTIONS

Sarah K. Kennedy, Frances M. Russell, Robinson M. Ferre, and Loren K. Rood conceived the study. Sarah K. Kennedy and Loren K. Rood supervised the conduct of the trial and data collection. Sarah K. Kennedy, Frances M. Russell, Robert R. Ehrman, Daniel Brenner, and Benjamin Nti drafted the manuscript. Sarah K. Kennedy, Frances M. Russell, Robert R. Ehrman, Daniel Brenner, Benjamin Nti, Loren K. Rood, Robinson M. Ferre, and Matt A. Rutz contributed substantially to its revision. Frances M. Russell takes responsibility for the paper as a whole.

Supporting information

Fig S1

Fig S2

Kennedy SK, Ferre RM, Rood LK, et al. Success of implementation of a systemwide point‐of‐care ultrasound privileging program for emergency medicine faculty. AEM Educ Train. 2022;6:e10744. doi: 10.1002/aet2.10744