The Surgeon-Performed Ultrasound: A Curriculum to Improve Residents’ Basic Ultrasound Knowledge

Abstract

Background

Despite the development of ultrasound courses by the American College of Surgeons two decades ago, many residencies lack formal ultrasound training. The aim of this study was to assess the previous ultrasound experience of residents and the efficacy of a new ultrasound curriculum by comparing pre- and post-course tests.

Methods

A pre-course survey and test were sent to all residents at the University of Texas Southwestern Medical Center. Pre-interns and junior residents received a didactic lecture on ultrasound basics and the extended focused assessment with sonography for trauma (EFAST) and were given hands-on practice. Finally, a post-course test and survey were sent to the pre-interns and junior residents.

Results

Only 11.3% of the residents reported having previous exposure to a formal ultrasound curriculum, and only 12.7% were taught by faculty. On the pre-course test, there was no difference in performance among senior residents, junior residents and pre-interns (p = 0.114). After taking the course, the pre-interns improved their performance and their average increased from 44.3% (SD = 12.4%) to 66.1% (SD = 12.2%; p < 0.001). The junior residents also had an improvement in their performance on the test after the course (p < 0.001). Junior residents performed better than pre-interns on the post-course test (p = 0.001).

Conclusion

The knowledge of surgical residents in ultrasound basics and EFAST can be improved with the establishment of an ultrasound curriculum. We believe that such an educational endeavor should be encouraged by all surgical residencies.

Introduction

The use of ultrasound has gained acceptance across many disciplines and has proven to be valuable in a variety of clinical settings. Several specialties have recognized the importance of developing a rigorous ultrasound training curriculum, and some have mandated this be completed during residency training. For example, emergency medicine residencies require that every resident take a series of lectures and participate in a 2- to 4-week rotation with dedicated hands-on practice in ultrasound [1,2]. Many critical care fellowships have started to embed an ultrasound curriculum in their training [3]. Ultrasound training is not isolated to residents and fellows. Some medical schools recognize the importance of this emerging skill set and have integrated ultrasound into their curriculum to better prepare students for the clinical setting [4,5].

General surgeons use ultrasound as a diagnostic tool in the office, emergency department, and operating room. They also use it for procedural guidance in the outpatient setting, operating room, intensive care unit, and the emergency department. The American College of Surgeons (ACS) recognized, early on, the necessity of providing ultrasound training for surgeons. In 1996, the first ultrasound course, entitled “Ultrasound for the General Surgeon,” was given at the 24^th spring meeting of the ACS. This effort was led by an “Ultrasound Users Group” under the leadership of the chairman of the college’s Committee on Emerging Surgical Technologies and Education and the director of the ACS Education and Surgical Services Department [6]. In 1998, a voluntary “verification program” was implemented by the ACS board of regents providing certification for the trainee and describing the level of proficiency attained [7]. The program has evolved and the ACS currently offers both online and live ultrasound courses through its website for both residents and practicing general surgeons. This course covers a wide range of ultrasound applications, including vascular, endocrine, breast, and abdominal. Despite the presence of a rigorous and well-established ultrasound curriculum developed by the ACS, formal ultrasound teaching is not emphasized in many surgical residencies. A survey of directors of 255 general surgery residency programs showed that up to 49% of university programs and 43% of community programs did not teach their residents about ultrasound basics and knobology. These percentages may underestimate the reality, as nearly half of the programs didn’t participate in the survey [8]. While the cost of the ACS program may make it prohibitive for nationwide adoption, several other factors may be present for the lack of ultrasound training in surgical residencies.

Our program has developed a tiered ultrasound curriculum with the initial course focusing on the basics of ultrasound, image interpretation, and clinically applying these skills using the extended focused assessment with sonography for trauma (EFAST) to junior residents. The EFAST examination has been shown to be easily applied by novice learners, giving them a proficiency and accuracy comparable to that of an ultrasound-trained radiologist [9–12]. In addition, we wanted to teach the learners a clinically important skill. The EFAST exam has been shown to be a necessary adjunct in dictating the appropriateness for surgical intervention in unstable patients with blunt thoraco-abdominal trauma [13–16]. The extended part of the EFAST exam incorporates assessing the pleural space for a pneumo and hemothorax. The sensitivity of ultrasound in the detection of a pneumothorax in the emergency setting is higher than supine chest x-ray; however it continues to be underutilized in clinical practice [17].

The goal of this study was twofold. First, we sought to explore baseline ultrasound knowledge at a large academic general surgery program. Second, we examined the effectiveness of an EFAST-focused curriculum in improving US knowledge, EFAST application, and image interpretation.

Methods

The study was approved by the University of Texas Southwestern institutional review board, the department chairman, the assistant dean for graduate medical education, and the department of surgery resident education research advisory committee.

A survey and a pre-course test were sent to all residents at the University of Texas Southwestern Medical Center (PGY1 to PGY5) on May 2016 and all incoming interns (referred to as pre-interns) on June 2016. The course was only given to junior residents (PGY-1 and PGY-2) as a group of 6–8 residents with two instructors, and to the pre-interns the week prior to the start of residency training. The course was followed by a post-course survey and test. Senior residents did not participate in the course or the post-course test. The surveys and tests were sent electronically and the responses were anonymous. None of the data collected could be linked to individual participants and the results could not be used to impact individual resident evaluation. While the participation in the survey/test was voluntary, the ultrasound course was integrated into the educational curriculum and was mandatory for all junior residents.

The survey evaluated the subjective assessment of the residents of their own knowledge and skills in taking and interpreting ultrasound images. The residents were asked to rate their knowledge of the basic physics of ultrasound, taking adequate images, interpreting the images, and diagnosing free intraperitoneal fluid and pneumothorax using ultrasound. The rating ranged from 0 (poor) to 100 (excellent) with a rating of 50 being moderate. In addition, the survey contained questions regarding the residents’ previous experience in ultrasound and course feedback.

The pre-course test consisted of 20 questions and the post-course consisted of 23 questions. This test assessed the residents’ knowledge in three main sections: 1) ultrasound physics and knobology (referred to as ultrasound basics), 2) image interpretation, and 3) the definition, indications, and limitations of EFAST (referred to as applied questions). The questions for the pre- and post-course were different but had a similar ratio for the three areas. Alternate forms of the exam were used to minimize retesting effects, taking additional care to maintain a similar level of difficulty between both tests.

The course consisted of a two-hour didactic presentation and an hour practical session. The didactic presentation was developed by the first and second author and reviewed by the senior author. This presentation was split into two sections with the first one focusing on ultrasound basics, physics, and knobology. It covered the frequency of ultrasound, its speed in the human body, the piezoelectric effect, image formation, acoustic characteristics (including reflection, refraction, transmission, scatter, and attenuation), ultrasound echogenicity, resolution, modes of display, artifact, probes, spatial orientation, and machine functions. The second part of the lecture focused on EFAST definition, components, indications, advantages, disadvantages, limitations, and sensitivity and specificity compared to CT and x-ray imaging and image interpretation.

The practical session was provided by trauma and surgical critical fellows and consisted of residents performing supervised EFAST examinations on healthy volunteers. The residents watched others perform the skill and were allowed to practice until achieving proficiency.

Descriptive statistics were calculated and presented as a mean and standard deviation for normally distributed continuous variables, and numbers with percentages for the survey items. The pre-test and post-test scores were compared using independent sample t-tests as we were not able to pair the scores of individual residents since the responses were anonymous. We also compared results across years in training with a t-test or analysis of variance (ANOVA) when appropriate. These results were divided into pre-interns, junior (PGY1–2), and seniors (PGY3–5). A two-way ANOVA was used to determine the interaction between the performance on the post-course test compared to the pre-course test and the resident level. Chi-square or Fischer’s exact tests were used to compare categorical data, as appropriate. A Pearson correlation was used to assess the correlation between the residents’ subjective assessment of their own knowledge of ultrasound basics and image interpretation with their objective scores on the pre-test for both fields. All statistical analyses were performed using SPSS version 23 with the significance set at p < 0.05.

Results

The rate of participation was 64.5% (71/110) for the pre-course survey and test and 61.5% (43/70) for the post-course survey and test. The distribution of the residents’ level is summarized in Figure 1.

Figure 1.

Distribution of residents for the pre- and post-course test

Nearly all (94.4%, 67/71) of the trainees agreed that a formal ultrasound curriculum would make them more comfortable taking and interpreting EFAST images. All (43/43) thought that the course was useful and would recommend that their colleagues take the course. Only 11.3% (8/71) reported having previous exposure to a formal ultrasound curriculum with the pre-interns having the highest exposure at 25.9% (7/27) compared to junior residents (4%, 1/25) and senior residents (0%, 0/19; p = 0.017). Overall, 33.8% of the trainees (24/71) had not already learned how to perform an EFAST. The highest proportion of residents without previous EFAST exposure was among pre-interns (59.3%, 16/27) compared to junior residents (20.0%, 5/25) and senior residents (15.8%, 3/19; p = 0.002). Only 12.7% (9/71) had been taught by a faculty member while 12.7% (9/71) were taught by junior residents and 36.6% (26/71) were taught by senior residents. Up to 46.5% (33/71) of the trainees had not performed an EFAST exam with the highest proportion being among pre-interns (74.1%, 20/27) compared to juniors (32.0%, 8/25) and seniors (26.3%, 5/19; p = 0.001; Table 1).

Table 1.

Results of the pre- and post-course survey

Questions	Responses	Pre-Interns (N = 27)	Juniors (N = 25)	Seniors (N = 19)	Total (N = 71)	Fisher’s Exact P Value
Pre-course survey
Have you taken an ultrasound course?	Yes	7 (25.9%)	1 (4.0%)	0 (0%)	8 (11.3%)	0.017
	No	20 (74.1%)	24 (96.0%)	19 (100%)	63 (88.7%)
Would completing a formal ultrasound curriculum make you more comfortable performing an EFAST and interpreting ultrasound images in the clinical setting?	Yes	26 (96.3%)	24 (96%)	17 (89.5%)	67 (94.4%)	0.533
	No	1 (4.0%)	1 (4.0%)	2 (10.5%)	4 (5.6%)
How did you learn to perform an EFAST?	a) I have not learned how to perform an EFAST	16 (59.3%)	5 (20.0%)	3 (15.8%)	24 (33.8%)	0.002*
	b) From a junior resident (PG1–2)	3 (11.0%)	5 (20.0%)	1 (5.3%)	9 (12.7%)
	c) From a senior resident (PG3–5)	2 (7.4%)	13 (52.0%)	11 (57.9%)	26 (36.6%)
	e) From an attending	4 (14.8%)	1 (4.0%)	4 (21.1%)	9 (12.7%)
	f) Others (example: emergency medicine resident or attending)	2 (7.4%)	1 (4.0%)	0 (0%)	3 (4.2%)
Have you performed an EFAST?	Yes	7 (25.9%)	16 (68.0%)	14 (73.7%)	38 (53.5%)	0.001Φ
	No	20 (74.1%)	8 (32.0%)	5 (26.3%)	33 (46.5%)
Post-course survey
Was the course useful?	Yes	26 (100%)	17 (100%)		43 (100%)
Would you recommend that your classmate take the course?	Yes	26 (100%)	17 (100%)		43 (100%)

^*Variable was dichotomized into have not learned vs. learned

^ΦChi-square test

When asked to rate their level of knowledge of ultrasound basics (Figure 2), there was a significant difference between the groups (p = 0.022), with pre-interns and seniors reporting higher knowledge ratings compared to juniors. In addition, there was a significant difference in the confidence of residents in their ability to interpret ultrasound images (p = 0.034) and to diagnose intraperitoneal free fluid (p = 0.001) with seniors having a higher level of confidence compared to pre-interns and juniors. All three groups rated their ability to diagnose a pneumothorax using ultrasound as low, and there was no difference between them on this point (p = 0.250).

Figure 2.

Residents’ rating of their own knowledge and skills in ultrasound

On the pre-course test, there was no difference in performance among the three groups, with seniors averaging 49.2% of correct responses (SD = 12.5%), juniors 42.0% (SD = 8.8%) and pre-interns 44.2% (SD = 12.3%; p = 0.114). In addition, there was no difference in performance among the three groups across the different sections of the test (ultrasound basics: p = 0.409; image interpretation: p = 0.199; and applied questions: p = 0.380; Table 2).

Table 2.

Pre-course performance

Groups	N	Total Score (%)			Ultrasound Basics (%)			Image Interpretation (%)			Applied Questions (%)
		Mean	(SD)	P value	Mean	(SD)	P value	Mean	(SD)	P value	Mean	(SD)	P value
Pre-interns	27	44.26	(12.38)	0.114	44.44	(18.23)	0.409	47.22	(28.02)	0.199	42.33	(19.20)	0.380
Juniors	25	42.00	(8.89)		41.33	(19.38)		35.00	(27.95)		46.86	(20.83)
Seniors	19	49.21	(12.5)		49.12	(19.71)		47.37	(24.85)		50.38	(18.05)

There was a correlation between the rating of the residents of their knowledge in ultrasound basics with their test score on this section (r = 0.248, p = 0.044). However, there was no correlation between the subjective assessment of the residents about their own skill level at image interpretation and their actual test score on this section (r = −0.054, p = 0.669; Figure 3).

Figure 3.

Correlation between residents’ ratings and performance on the test

After taking the course, the pre-interns improved their performance from 44.3% (SD = 12.4%) to 66.1% (SD = 12.2%; p < 0.001). This improvement was across all three sections of the test including ultrasound basics (pre-course average: 44.4%, SD = 18.2%; compared to a post-course average of 58.1%, SD = 16.4%; p = 0.006) image interpretation (pre-course average: 47.2%, SD = 28.0%; compared to a post-course average of 60.6%, SD = 18.9%; p = 0.048) and applied questions (pre-course average: 42.3%, SD = 19.2%; compared to a post-course average of 75.3%, SD = 14.2%; p < 0.001).

The junior residents also had an improvement in their performance on the post-course test (pre-course average: 42.0%, SD = 8.9%; compared to a post-course average of 80.8%, SD = 12.2%; p < 0.001). The improvement was also present across all sections of the test including ultrasound basics (pre-course average: 41.3%, SD = 19.4%; compared to a post-course average of 71.2%, SD = 14.2%; p < 0.001), image interpretation (pre-course average: 35.0%, SD = 27.9%; compared to a post-course average of 77.9%, SD = 21.4%; p < 0.001), and applied questions (pre-course average: 46.9%, SD = 20.8%; compared to a post-course average of 90.6%, SD = 13.9%; p < 0.001).

There was interaction between the performance on the post-course test compared to the pre-course test and the level of residents, with junior residents performing better than pre-interns on the post-course test (p = 0.001). This was true for the ultrasound basics (p = 0.028) and image interpretation section (p = 0.005) but not the applied questions (p = 0.149).

Discussion

The ACGME requires that general surgery residencies “ensure that residents have required experience with evolving diagnostic and therapeutic methods” [18]. While it does not specify and define evolving diagnostic methods, ultrasound meets this requirement. Over the last decades, the role of ultrasound has been integrated into different surgical clinical settings. Ultrasound is a vital diagnostic and interventional modality in trauma, surgical oncology, critical care, breast, pediatrics, endocrine, and vascular surgery [3,9,13–15,19–21]. More recently it has also been incorporated into minimally invasive surgery and in endoscopic exams [22,23]. The ACS recognizes the importance of ultrasound in surgical practice and has developed both practical and didactic courses that cover important aspects of ultrasound use in surgery. These resources became popular among surgeons and increased the use of ultrasound in clinical practices [6]. In a survey sent to surgeons who have participated in the ACS course, 65% of them used ultrasound in their practice with breast ultrasound being the most commonly used followed by ultrasound in acute care and vascular surgery [6]. Despite the importance of ultrasound and its wide use by surgeons, a formal ultrasound training curriculum is lacking in many surgical residencies. In a survey to ACGME-accredited general surgery residency programs with a 51% response rate, didactic training was absent in 42% of the programs for ultrasound basics, 66% for abdominal and laparoscopic ultrasound, 59% for breast ultrasound, and 22% for FAST exam. Our study shows that the surgical resident’s knowledge of ultrasound basics and EFAST is not optimal in the absence of formal teaching. This was true across all classes as the average of performance on the pre-course test was similar between pre-interns, juniors and seniors. Despite multiple rotations where ultrasound is routinely used and the limited exposure to the use of the FAST during the advanced trauma life support course, the knowledge of the residents did not appear to improve in basic ultrasound and EFAST as they become more senior. Interestingly, 15.8% of seniors did not learn how to perform an EFAST. This is mainly due to a lack of teaching of the extended part of the EFAST since all residents are required to perform FAST exams during their trauma rotations; however this experience is not standardized. This highlights the heterogeneity of learning among residents in the absence of a standardized curriculum. More concerning than the heterogeneity of resident education is that senior residents had a relatively high confidence in their image interpretation abilities, although they did not perform as such on the test. This would indicate that we are training surgeons to be confident in ultrasound skills that perform poorly when it becomes necessary to apply this skill set.

With the establishment of a formal course, residents had a significant improvement in their performance on the post-course test compared to the pre-course test. In addition, all residents who participated in the course agreed to the importance of such an educational endeavor. In this study, juniors benefited more from the course compared to pre-interns; however they had similar pre-course results. This might be secondary to pre-exposure to the ultrasound machine and its use during their clinical rotation. Testing the retention of knowledge gained from the course is necessary and subsequently reinforcement of this learning experience may be needed.

An interesting finding in the survey is the higher exposure of the new generation of residents to previous formal ultrasound courses. Pre-interns had the highest exposure with 25.9% compared to junior residents (4%) and senior residents (0%). This may reflect the new integration of ultrasound training in medical school curricula. Studies have shown that ultrasound teaching in medical school has become more popular; however it is not required [5,24–26]. Early exposure to this technology, reinforcement from surgical residents, and its application by faculty in the clinical setting can make medical students better equipped to apply this technology in resident training.

Despite these important findings, there are a number of limitations in this study. One of these is the fairly small sample size and use of only one program. There may also have been selection bias as not all participants completed the survey and tests. Inclusion of more trainees across a sampling of programs would make these data more robust. Additionally, as a single armed intervention, the improvement of the resident’s performance may be due to exposure to the material during the pre-course test despite our efforts to minimize any testing effects. Finally, this study did not test the ability of the residents to take adequate images or the implication of such educational activity on clinical outcomes. Furthermore, the human models used for hands-on teaching were all healthy residents. The different pathologies that can be diagnosed with an EFAST were taught showing representative images during the didactic sessions. Many studies have shown that the use of simulators is effective in teaching novices, and thus can be used to teach residents about the pathologies in a non-stressful environment [27,28]. Future work should continue to explore these methodologies.

Despite these potential concerns, the study’s aim is to raise awareness that the education of residents about ultrasound may not be adequate in the absence of a rigorous curriculum. This calls for the need to implement a curriculum in every surgical residency, an effort that was welcomed by the residents in this study. The ACS ultrasound courses can be potentially used as part of a curriculum for resident education. One obstacle to widespread implementation is the cost. In a large program like ours, with approximately 70 residents, it would cost $5750 ($500 for the content + $75/resident × 70 residents) to export the course to our institution. With medical educational budgets becomingly increasingly stringent, this may be a seriously prohibitive obstacle for many institutions.

Conclusions

The knowledge of surgical residents in ultrasound basics and EFAST can be improved with the establishment of a short and easy to implement ultrasound curriculum. We believe that such an educational endeavor should be a requirement by the ACGME for all surgical residencies. Studies such as this show the low cost and utility of such an endeavor.

Figure 4.

Interaction between the performance on the post-course test compared to pre-course test and the level of residents

Table 3.

Comparison between pre-and post-course performance

Groups	Test (N)	Total Score (%)			Ultrasound Basics (%)			Image Interpretation (%)			Applied Questions (%)
		Mean	(SD)	P value	Mean	(SD)	P value	Mean	(SD)	P value	Mean	(SD)	P value
Pre-interns	Pre (27)	44.26	(12.38)	<0.001	44.44	(18.23)	0.006	47.22	(28.02)	0.048	42.33	(19.20)	<0.001
	Post (26)	66.05	(12.24)		58.12	(16.42)		60.58	(18.94)		75.38	(14.21)
Juniors	Pre(25)	42.00	(8.89)	<0.001	41.33	(19.38)	<0.001	35.00	(27.95)	<0.001	46.86	(20.83)	<0.001
	Post(17)	80.82	(12.21)		71.23	(14.20)		77.94	(21.44)		90.59	(13.91)

Abbreviations

ACS

American College of Surgeons

EFAST

extended focused assessment with sonography for trauma