Abstract
Background
Point‐of‐care ultrasound (POCUS) is increasingly utilized in emergency medicine (EM). While residents are required by the Accreditation Council for General Medical Education to complete a minimum of 150 POCUS examinations before graduation, the distribution of examination types is not well‐described. This study sought to assess the number and distribution of POCUS examinations completed during EM residency training and evaluate trends over time.
Methods
This was a 10‐year retrospective review of POCUS examinations across five EM residency programs. The study sites were deliberately selected to represent diversity in program type, program length, and geography. Data from EM residents graduating from 2013 to 2022 were eligible for inclusion. Exclusion criteria were residents in combined training programs, residents who did not complete all training at one institution, and residents who did not have data available. Examination types were identified from the American College of Emergency Physicians guidelines for POCUS. Each site obtained POCUS examination totals for every resident upon graduation. We calculated the mean and 95% confidence interval for each procedure across study years.
Results
A total of 535 residents were eligible for inclusion, with 524 (97.9%) meeting all inclusion criteria. The mean number of POCUS examinations per resident increased by 46.9% from 277 in 2013 to 407 in 2022. All examination types had stable or increasing frequency. Focused assessment with sonography in trauma (FAST), cardiac, obstetric/gynecologic, and renal/bladder were performed most frequently. Ocular, deep venous thrombosis, musculoskeletal, skin/soft tissue, thoracic, and cardiac examinations had the largest percentage increase in numbers over the 10‐year period, while bowel and testicular POCUS remained rare.
Conclusions
There was an overall increase in the number of POCUS examinations performed by EM residents over the past 10 years, with FAST, cardiac, obstetric/gynecologic, and renal/bladder being the most common examination types. Among less common procedures, increased frequency may be needed to ensure competence and avoid skill decay for those examination types. This information can help inform POCUS training in residency and accreditation requirements.
INTRODUCTION
Point‐of‐care ultrasound (POCUS) was introduced to emergency medicine (EM) in the 1980s and quickly became a core skill for EM physicians, with training courses beginning in 1991 and the first EM ultrasound fellowship starting in 1993. 1 The rapid expansion of POCUS prompted the creation of the first national curriculum in 1994 2 as well as POCUS being included in national residency training requirements in 1996. 3 In 2012, the Accreditation Council for General Medical Education (ACGME) designated POCUS as one of the Milestone competencies for graduating EM residents. 3
As POCUS indications and use expanded, the need to ensure competence among EM physicians became apparent. POCUS has subsequently been included as a core component of EM, with both diagnostic and procedural ultrasound being included in the American Board of Emergency Medicine Model of Clinical Practice in EM. 4 This has further translated to residency training, where the ACGME lists POCUS as one of the Key Index Procedures. 5 While no longer explicitly included in Milestones 2.0, 6 the ACGME Key Index Procedures guidelines still require that residents complete a minimum of 150 POCUS examinations to complete residency training, though the distribution of specific examination types is not defined. 5 Alternatively, the American College of Emergency Physicians (ACEP) Emergency Ultrasound Guidelines recommend that residents complete 150–300 POCUS examinations and at least 25–50 in a given application. 7 These variations in training recommendations between accreditation bodies and specialty professional organizations create ambiguity for institutions regarding optimal strategies to educate trainees in POCUS.
Prior research has described the total numbers of POCUS performed during training but were older and limited to single sites with small sample sizes and an emphasis on only the total numbers of examinations. 8 , 9 , 10 No studies have sought to quantify the distribution of POCUS examinations per user. Moreover, it is unclear whether the number of examinations performed has changed over time as the field has evolved. As the ACGME reevaluates its numeric requirements for the Key Index Procedures and programs determine their local goals, there is a need to provide benchmarks on examination performance in specific POCUS applications. The objective of this study was to assess the number of POCUS examinations completed during EM residency training and evaluate these trends over time.
METHODS
Study design
We conducted a retrospective review of specific POCUS examination totals among graduating EM residents at five ACGME‐accredited residency programs from 2013 to 2022. The institutional review boards at all five sites reviewed the study and deemed it either exempt or approved without required consent.
Study population
Ultrasound examination data from all categorical EM residents graduating in 2013–2022 were eligible for inclusion. We excluded data from residents that did not complete their full training at that institution (i.e., transferred to/from another residency program), residents in combined training programs (e.g., EM and internal medicine/family practice/critical care), and residents without ultrasound data available. We deliberately selected study sites based on access to reliable POCUS data while seeking to ensure breadth of geographic location, program length, and program type. Study site characteristics are described in Table S1.
Study protocol
We determined the list of POCUS examination types based on the 2016 ACEP guidelines for core and advanced applications. 7 Each site provided anonymous, resident‐specific totals for each POCUS examination based on logged totals in their image archival software and resident self‐report via procedure logs (when not stored in their image archival software).
Data analysis
We calculated the mean and 95% confidence intervals (CIs) for each specific POCUS examination type. Data were presented in total as well as by year of graduation. All analyses were performed in Microsoft Excel 365 and Stata 16.
RESULTS
We collected data from 535 total residents across all five programs, with data from 524 (97.9%) meeting inclusion criteria. Of the 11 residents excluded, eight were due to missing data and three had transferred in or out of the residency program. There were 324 (60.6%) men, 210 (39.3%) women, and one (0.1%) nonbinary resident.
The mean overall number of POCUS examinations performed during residency was 348, which increased from 277 in the class of 2013 to 407 in the class of 2022 (Table 1). The most frequently performed POCUS examinations were focused assessment with sonography in trauma (FAST), cardiac, obstetric/gynecologic, and renal/bladder. Most sites reported obstetric/gynecologic examinations as a single category. Among those reporting transabdominal versus transvaginal obstetric/gynecologic ultrasound separately (n = 157 residents), transabdominal (mean 13, 95% CI 11–14) was performed significantly more often than transvaginal (mean 2, 95% CI 2–3). Ocular, deep venous thrombosis, musculoskeletal, skin/soft tissue, thoracic, and cardiac examinations had the largest percentage increase in numbers over the 10‐year period, while bowel and testicular POCUS remained rare (Figure S1).
TABLE 1.
Distribution of ultrasound numbers by graduation year.
| Year of graduation | FAST | Cardiac | OB/GYN | Renal/Bladder | Biliary | Thoracic a | Skin/soft tissue a | Aorta | DVT b | Musculo‐skeletal a | Ocular a | Bowel a | Testicular a | Total |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 2013 | 67 (57–78) | 38 (29–48) | 41 (30–51) | 26 (19–32) | 25 (18–34) | 13 (7–18) | 11 (7–14) | 22 (16–27) | 9 (6–13) | 7 (4–10) | 4 (3–6) | 0 (0–0) | 0 (0–0) | 277 (221–334) |
| 2014 | 65 (51–79) | 31 (24–37) | 36 (27–45) | 23 (17–29) | 22 (17–27) | 11 (7–15) | 11 (7–15) | 16 (12–20) | 11 (7–14) | 9 (5–12) | 7 (4–9) | 0 (0–0) | 0 (0–1) | 243 (197–289) |
| 2015 | 80 (68–92) | 67 (20–113) | 36 (29–43) | 31 (24–37) | 28 (23–32) | 22 (17–28) | 13 (10–16) | 23 (19–27) | 15 (10–20) | 10 (7–13) | 7 (5–9) | 2 (1–4) | 0 (0–1) | 332 (258–406) |
| 2016 | 77 (64–91) | 58 (46–69) | 39 (31–47) | 33 (27–39) | 33 (28–39) | 21 (16–27) | 14 (10–19) | 22 (19–26) | 15 (9–20) | 10 (6–14) | 10 (7–13) | 1 (0–1) | 1 (0–1) | 335 (286–384) |
| 2017 | 54 (44–65) | 46 (36–56) | 29 (22–35) | 27 (20–35) | 21 (16–27) | 15 (12–18) | 12 (9–16) | 16 (12–20) | 11 (8–14) | 9 (7–12) | 7 (5–9) | 1 (0–1) | 0 (0–0) | 277 (229–326) |
| 2018 | 70 (57–83) | 76 (55–98) | 30 (23–37) | 28 (23–32) | 26 (21–31) | 27 (20–35) | 26 (17–35) | 18 (14–22) | 13 (7–18) | 9 (3–14) | 11 (8–15) | 2 (0–3) | 1 (0–1) | 341 (281–402) |
| 2019 | 92 (73–111) | 103 (82–124) | 52 (43–60) | 42 (34–51) | 28 (21–34) | 23 (16–29) | 51 (42–61) | 23 (18–28) | 19 (10–27) | 25 (15–35) | 17 (14–20) | 1 (0–2) | 1 (0–1) | 463 (396–530) |
| 2020 | 77 (64–89) | 88 (76–100) | 32 (27–38) | 32 (28–36) | 28 (25–31) | 36 (28–44) | 26 (19–32) | 20 (17–23) | 16 (13–20) | 11 (7–16) | 12 (10–14) | 2 (1–2) | 0 (0–1) | 378 (335–422) |
| 2021 | 81 (70–93) | 77 (68–87) | 36 (31–41) | 33 (31–36) | 30 (27–33) | 33 (25–41) | 28 (23–33) | 20 (18–23) | 19 (14–24) | 12 (9–16) | 15 (13–18) | 2 (2–3) | 0 (0–1) | 388 (352–425) |
| 2022 | 73 (62–84) | 90 (79–101) | 41 (35–46) | 31 (28–34) | 29 (27–32) | 32 (25–39) | 28 (23–34) | 22 (19–24) | 23 (17–28) | 17 (13–21) | 15 (13–18) | 2 (1–3) | 0 (0–1) | 407 (368–445) |
| Average | 74 (70–78) | 69 (63–75) | 37 (34–39) | 31 (29–32) | 27 (26–29) | 25 (23–27) | 22 (20–24) | 20 (19–21) | 15 (14–17) | 12 (10–13) | 11 (10–12) | 1 (1–2) | 0 (0–1) | 348 (331–365) |
Note: Data are reported as mean (95% CI).
Abbreviations: DVT, deep venous thrombosis; FAST, focused assessment with sonography in trauma; OB/GYN, obstetric or gynecologic ultrasound.
Data not available for 87 residents.
Data not available for 46 residents.
DISCUSSION
Our study reported the mean number of overall and specific POCUS examinations over a 10‐year period, finding that the mean overall EM resident ultrasound numbers have increased by nearly 50% from 2013 to 2022. This is important, as the increase in numbers has far exceeded the number recommended by the ACGME by a factor of 3. There are several reasons for this potential increase. First, ultrasound training has advanced as a specialty, with increased applications and expanded incorporation into training. 11 , 12 It has also become more ubiquitous within EM, possibly stemming from data demonstrating benefits in patient care across several applications. 13 Moreover, ultrasound fellowship–trained EM faculty are also increasing, with a positive effect on resident education. 14 , 15 Increased billing for POCUS may have further increased clinical use.
We also identified several interesting findings with regard to specific ultrasound examination types. Over the 10‐year period, we found increases in ocular, deep venous thrombosis, skin/soft tissue, musculoskeletal, thoracic, and cardiac ultrasound examinations performed. This may have been influenced by the transition of ocular, deep venous thrombosis, soft tissue/musculoskeletal, and thoracic from newer applications to core applications in the more recent 2016 ACEP ultrasound guidelines. 7 Among the common indications, FAST was the most frequently utilized, with cardiac being the only other indication to exceed an average of 50 scans per resident. These may be due to the broader range of indications for these as opposed to some more narrow examinations (e.g., bowel, testicular). Biliary, obstetric, renal, and thoracic ultrasound also exceeded the minimum recommendation of 25 scans. 7 Interestingly, many of the other indications fell below the ACEP recommended 25 scans. In particular, a few examinations (e.g., transvaginal, testicular, bowel ultrasound) remained rather low, and residents are unlikely to demonstrate competence for these indications at the current thresholds. Bowel ultrasound has been an emerging application and may rise in use as comfort with this grows. 16 In contrast, transvaginal and testicular ultrasound may be more challenging to obtain numbers due to perceived invasiveness of the examination and greater potential for a repeat radiologic examination if the initial is nondiagnostic.
We also noticed a peak in the total number of examinations performed in the 2019 graduation year. This was followed by a rapid decline in the class of 2020 and a subsequent slow recovery. One potential reason for this decline may have been the impact of COVID‐19 reducing the number of POCUS examinations that could be obtained due to increased patient care needs in other areas, canceled POCUS rotations, COVID‐19 exposures leading to more sick days and call coverage, and reduced overall POCUS imaging to limit clinician exposure. 17 , 18 , 19 The impact of COVID‐19 may also explain the paradoxical spike in thoracic imaging during that time.
While many programs currently track learner's progression to graduation based on the total number of scans performed overall, the actual learning curves in ultrasound skill attainment can vary by examination type. Performance plateaus for image quality on soft tissue examinations, for example, are estimated at 18 examinations, whereas right upper quadrant scans are closer to 90 examinations. 20 Although one can estimate that 25–50 quality reviewed examinations are adequate, we suggest that ultrasound assessment and learning is not one size fits all. Similar to the way that the ACGME tailors minimum examination numbers for core procedures in EM training we argue that ultrasound assessment should be similarly tailored for different examination types. For example, in the ACGME Key Index Procedures, resuscitations are not viewed as all the same. Programs must track trauma resuscitations differently than medical resuscitations and within each, pediatric cases are differentiated from adults. 4 Consequently, we propose that programs should similarly tailor their ultrasound numbers to the specific learning curve data for each examination rather than grouping all ultrasound examinations as a single category.
Although many programs currently utilize a numerical approach as a surrogate for proficiency, it is important to note that ensuring competency in emergency ultrasound includes multiple components. The I‐AIM model suggests that operators need to identify when to perform the scan, acquire the images properly, interpret the obtained images, and incorporate these interpretations into medical decision making. 21 These components cannot be accurately captured using numerical data alone. Recent work from the Ultrasound Competency Work Group advocates for the use of multimodal tools for assessment of ultrasound competence. 22 Recommended assessments include written examinations, image review, objective structured clinical evaluations, standardized direct observation tools, and direct clinical observation. In the era of competency‐based medical education, educators will need specific tools to assess entrustment of skills and procedures like POCUS, such as the Ultrasound Competency Assessment Tool. 23 , 24 Ultimately, given that learning curves do require a variable amount of practice based on examination type and learner, these tools will likely merge with numerical data to provide a defensible summative assessment of a learner's acquisition of competence.
LIMITATIONS
This study was limited to five institutions over a 10‐year period and may not fully reflect other institutions. However, we intentionally selected programs from different geographic locations and with varying program types. We also utilized the 2016 ACEP guidelines to inform our list of POCUS examinations. 7 As the field has advanced, some applications have transitioned from advanced or emerging to core, which may have influenced the number of examinations performed. Moreover, some examination types (e.g., contrast‐enhanced ultrasound, transcranial Doppler, procedural guidance) were not routinely collected at any site, limiting the ability to evaluate these applications. Other examinations (e.g., ocular, thoracic, transvaginal ultrasound) were not universally tracked across all sites. Another limitation is the reliance on image logs and self‐report. As such, this is subject to reporting bias and may have missed ultrasound examinations that were not saved or reported. We were also limited by the types of examinations reported. Finally, while this study presented data on numeric trends, we were unable to ascertain the quality of the associated images.
CONCLUSIONS
This study described the mean number of ultrasound examinations by specific application among emergency medicine residents and identified trends in the distribution over a 10‐year period. While many examination types increased in frequency, some remained uncommon. Among less common procedures, increased frequency may be needed to ensure competence and avoid skill decay for those examination types. Future work should examine this among different sites and identify factors associated with increased use.
CONFLICT OF INTEREST
The authors declare no potential conflict of interest.
Supporting information
FIGURE S1 Mean and 95% confidence intervals for each exam type from 2013 to 2022.
TABLE S1 Program demographics.
ACKNOWLEDGMENTS
The authors thank Dr. Petra Duran‐Gehring and Christine Carlson for their assistance with obtaining the ultrasound data.
Gottlieb M, Cooney R, King A, et al. Trends in point‐of‐care ultrasound use among emergency medicine residency programs over a 10‐year period. AEM Educ Train. 2023;7:e10853. doi: 10.1002/aet2.10853
Supervising Editor: Dr. Daniel Egan.
REFERENCES
- 1. von Kuenssberg JD, Wilson C. Some recollections of early work with bedside ultrasound in emergency medicine: the first 10 years. J Am Coll Emerg Physicians Open. 2020;1(5):871‐875. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2. Mateer J, Plummer D, Heller M, et al. Model curriculum for physician training in emergency ultrasonography. Ann Emerg Med. 1994;23(1):95‐102. [DOI] [PubMed] [Google Scholar]
- 3. Lewis R, Pearl M, Nomura J, et al. CORD‐AEUS: consensus document for the emergency ultrasound milestone project. Acad Emerg Med. 2013;20:740‐745. [DOI] [PubMed] [Google Scholar]
- 4. Beeson MS, Ankel F, Bhat R, et al. The 2019 model of the clinical practice of emergency medicine. J Emerg Med. 2020;59(1):96‐120. [DOI] [PubMed] [Google Scholar]
- 5. Accreditation Council for Graduate Medical Education . Emergency Medicine Defined Key Index Procedure Minimums. Accessed January 23, 2023. https://www.acgme.org/globalassets/pfassets/programresources/em_key_index_procedure_minimums_103117.pdf
- 6. The Accreditation Council for Graduate Medical Education . Emergency Medicine Milestones. Accessed January 23, 2023. https://www.acgme.org/globalassets/pdfs/milestones/emergencymedicinemilestones.pdf
- 7. American College of Emergency Physicians . Ultrasound Guidelines: Emergency, Point‐of‐care, and Clinical Ultrasound Guidelines in Medicine. Accessed January 23, 2023. https://www.acep.org/globalassets/new‐pdfs/policy‐statements/ultrasound‐guidelines‐‐‐emergency‐point‐of‐care‐and‐clinical‐ultrasound‐guidelines‐in‐medicine.pdf
- 8. Langdorf MI, Montague BJ, Bearie B, Sobel CS. Quantification of procedures and resuscitations in an emergency medicine residency. J Emerg Med. 1998;16(1):121‐127. [DOI] [PubMed] [Google Scholar]
- 9. Hayden SR, Panacek EA. Procedural competency in emergency medicine: the current range of resident experience. Acad Emerg Med. 1999;6(7):728‐735. [DOI] [PubMed] [Google Scholar]
- 10. Bucher JT, Bryczkowski C, Wei G, et al. Procedure rates performed by emergency medicine residents: a retrospective review. Int J Emerg Med. 2018;11(1):7. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11. Gottlieb M, Arno K, Kuhns M, Chan TM. Distribution of clinical rotations among emergency medicine residency programs in the United States. AEM Educ Train. 2018;2(4):288‐292. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12. Alerhand S, Situ‐Lacasse E, Ramdin C, Gottlieb M. National Survey of point‐of‐care ultrasound scholarly tracks in emergency medicine residency programs. West J Emerg Med. 2021;22(5):1095‐1101. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13. Agency for Healthcare Research and Quality . Evidence Report/Technology Assessment: Number 43. Making Health Care Safer. A Critical Analysis of Patient Safety Practices: Summary. Accessed January 23, 2023. http://archive.ahrq.gov/clinic/ptsafety/pdf/ptsafety.pdf [Google Scholar]
- 14. Adhikari S, Raio C, Morrison D, et al. Do emergency ultrasound fellowship programs impact emergency medicine residents' ultrasound education? J Ultrasound Med. 2014;33(6):999‐1004. [DOI] [PubMed] [Google Scholar]
- 15. Gottlieb M, Duran‐Gehring P, Coneybeare D, Lema P. Creation of a novel course for the advanced emergency medicine ultrasound focused practice designation examination. AEM Educ Train. 2022;6(5):e10810. doi: 10.1002/aet2.10810 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 16. Gottlieb M, Peksa GD, Pandurangadu AV, Nakitende D, Takhar S, Seethala RR. Utilization of ultrasound for the evaluation of small bowel obstruction: a systematic review and meta‐analysis. Am J Emerg Med. 2018;36(2):234‐242. [DOI] [PubMed] [Google Scholar]
- 17. Ford TR, Fix ML, Shappell E, et al. Beyond the emergency department: effects of COVID‐19 on emergency medicine resident education. AEM Educ Train. 2021;5(3):e10568. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 18. Tsyrulnik A, Gottlieb M, Coughlin RF, et al. Socially distanced, virtually connected: faculty and resident perceptions of virtual didactics. AEM Educ Train. 2021;5(3):e10617. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 19. Gottlieb M, Landry A, Egan DJ, et al. Rethinking residency conferences in the era of COVID‐19. AEM Educ Train. 2020;4(3):313‐317. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 20. Blehar DJ, Barton B, Gaspari RJ. Learning curves in emergency ultrasound education. Acad Emerg Med. 2015;22:574‐582. [DOI] [PubMed] [Google Scholar]
- 21. Bahner DP, Hughes D, Royall NA. I‐AIM: novel method for teaching and performing focused sonography. J Ultrasound Med. 2012;31:295‐300. [DOI] [PubMed] [Google Scholar]
- 22. Damewood SC, Leo M, Bailitz J, et al. Tools for measuring clinical ultrasound competency: recommendations from the ultrasound competency work group. AEM Educ Train. 2019;4(suppl 1):S106‐S112. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 23. Gottlieb M, Duanmu Y. Beyond the numbers: assessing competency in point‐of‐care ultrasound. Ann Emerg Med. 2023. doi: 10.1016/j.annemergmed.2023.01.020 [DOI] [PubMed] [Google Scholar]
- 24. Bell C, Hall AK, Wagner N, Rang L, Newbigging J, McKaigney C. The ultrasound competency assessment tool (UCAT): development and evaluation of a novel competency‐based assessment tool for point‐of‐care ultrasound. AEM Educ Train. 2020;5(3):e10520. [DOI] [PMC free article] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.
Supplementary Materials
FIGURE S1 Mean and 95% confidence intervals for each exam type from 2013 to 2022.
TABLE S1 Program demographics.